File size always considered changed when downloading compressed image

[Afnarel]

Describe the bug
When trying to download a compressed ISO image (using proxmox_virtual_environment_download_file), it seems the size from the downloaded compressed image is compared against the size of the already present local uncompressed image (if there is one).

So Terraform always considers the image size has changed and wants to download the image again and recreate the VM using this ISO every time.

To Reproduce
Steps to reproduce the behavior:

1. Create a proxmox_virtual_environment_download_file resource using a

resource "proxmox_virtual_environment_download_file" "opnsense_25_1_template_iso" {
  content_type            = "iso"
  datastore_id            = "local"
  node_name               = "console"
  url                     = "https://mirror.ams1.nl.leaseweb.net/opnsense/releases/25.1/OPNsense-25.1-dvd-amd64.iso.bz2"
  checksum                = "68efe0e5c20bd5fbe42918f000685ec10a1756126e37ca28f187b2ad7e5889ca"
  checksum_algorithm      = "sha256"
  upload_timeout          = 3600
  decompression_algorithm = "bz2"
  file_name               = "OPNsense-25.1-dvd-amd64.iso"
}

2. Use this resource to create a VM
3. Run terraform apply
4. Run terraform apply (or even just terraform plan) a second time without changing anything
5. Observe that Terraforms detects the size of the image is different and plans to download it again and recreate the VM, even though the image has not changed

Expected behavior
On a second run, if the image is already present in the local storage, it should not be downloaded again.

Additional information

This issue was raised a few months ago but in this other case the image was not compressed: #1440

When an image is uncompressed and its size changes I understand that the image should be considered different, leading to recreate the VM with the newly downloaded image.

However in this case the problem seems to be a different one. Not sure how to solve it.

Also, using a lifecycle block to ignore disk changes doesn't feel like a viable solution: I still want to be able to change disk size and other disk parameters.

  lifecycle {
    ignore_changes = [disk,]
  }

Screenshots

• Single or clustered Proxmox: single
• Proxmox version: 8.3.3
• Provider version (ideally it should be the latest version): 0.70.0
• Terraform/OpenTofu version: Terraform v1.9.8
• OS (where you run Terraform/OpenTofu from): Windows 11
• Debug logs (TF_LOG=DEBUG terraform apply):

[bpg]

Thanks for the report, @Afnarel!

You can set overwrite to false on the proxmox_virtual_environment_download_file resource to ignore size differences for the file.

That said, we could technically store the original remote file size in the Terraform state and use it for comparison during state refresh. However, this might have other implications, so we'll need to think it through 🤔

[rafsaf]

From what i remember, this feature (recreate file resource when file size has changed) is pretty simple and assumes that size from content-length header from url is the same as file size after download.

^ Obviously this is not always the case as per this issue

• I think introducing new "remote_size" to the state will be non breaking and will fix it and also work fine when no decompression algorithm is used, we should perform two comparisons: size vs size and remote_size vs remote_size, both trigger replace but with different warning message
• TBH i would really consider making overwrite false by default and document more this option what user should expect by choosing true, looking at this issue and especially CloudImage size changes forces replacement of VM #1440, current default behaviour is rather unexpected (EDIT: actually overwrite=false is used in mentioned issue but there is yet another bug 🐛 there and it's not respected properly)
• edit: one more nitpick: change of decompression_algorithm should probably recreate resource, not tried it but i believe PVE will crash on update anyway, or even if not, it makes no sense not to recreate

[Afnarel]

Thank you @bpg and @rafsaf!

I had tried adding the overwrite option to false and thought it was not working because running terraform plan still showed that the file would be downloaded again (and VM recreated).

Actually, after seeing your comments, I tried running terraform apply after adding overwrite = false and, after the apply has finished, any future run of plan/apply does not detect a change anymore.

About documentation, I was not 100% sure what overwrite was supposed to do when set to false. The documentation states "If false, there will be no checks." and I wasn't completely sure what doing no check would lead to: would it always download the file or would it never do so.
The setting being called overwrite, the "never" option was the most probable one, however maybe it could be clarified to leave no doubt, as well as clarifying that apply must be run once after changing the setting to take effect?

Using overwrite works for my use case. Of course it would be best to not have to use it, probably with this "remote size" solution you proposed, but in the meantime my problem is "resolved", so thank you!