Creating a VM from a large qcow2 image taking too long and then failing

[dandaolrian]

Hi,
We have a use case where we’re deploying a windows qcow2 image of around 20Gb and were running into problems deploying to one of our nodes (others were OK)
When we took a closer look at what the provisioner we saw it was running the following commands:

terraform-provider-proxmox/proxmoxtf/resource/vm.go

Lines 2790 to 2796 in a96a3a9

	`source_image=$(pvesm path "$file_id")`,
	`cp "$source_image" "$file_path_tmp"`,
	`qemu-img resize -f "$file_format" "$file_path_tmp" "${disk_size}G"`,
	`imported_disk="$(qm importdisk "$vm_id" "$file_path_tmp" "$datastore_id_target" -format $file_format | grep "unused0" | cut -d ":" -f 3 | cut -d "'" -f 1)"`,
	`disk_id="${datastore_id_target}:$imported_disk${disk_options}"`,
	`qm set "$vm_id" "-${disk_interface}" "$disk_id"`,
	`rm -f "$file_path_tmp"`,

So if we understand correctly, we copy the image, resize that import it and then throw the resized file away.
On the Proxmox node we saw multiple qemu-img convert processes and these take a long time when importdisk is running as we do that on the resized (larger) image

We thought logically it would make more sense to import the image as is (i.e smaller) and then resize. So like this

`imported_disk="$(qm importdisk "$vm_id" "$source_image" "$datastore_id_target" -format $file_format | grep "unused0" | cut -d ":" -f 3 | cut -d "'" -f 1)"`,
`disk_id="${datastore_id_target}:$imported_disk${disk_options}"`,
`qm set "$vm_id" "-${disk_interface}" "$disk_id"`,
`qm resize "$vm_id" "${disk_interface}" "${disk_size}G"`,

With this we no longer have such long times for deployment and were down from 55mins and failing to 10mins and working 😊

We would start a pull request but weren’t sure if maybe it had been done as above for a reason and wanted to understand that before doing so
This seems to similar if not the same as #675

Incidentally when it failed we got the same error as in #675
Post "https://dehx-pa-001:8006/api2/json/nodes/proxmox/qemu/437/status/start": context deadline exceeded
And we never managed to track down why the start would fail. We could start manually but it failed everytime here after taking 55+ mins to create the VM

Would be interested to hear your thoughts

[bpg]

Hi @dandaolrian,

That's a good investigation!
I don't know for sure why the resizing was implemented this way. The code is mostly unchanged since it was added in the original provider. I can only guess that qm resize command was probably not available at that time?

Anyway, I'll be happy to accept a PR for the change you're proposing, seems totally reasonable to me.

Speaking about the context deadline exceeded error -- that's definitely a timeout, but it's hard to say where exactly. The provider's default timeout_start_vm is 1800sec = 30min is shorter than the time you see. It could be that the execution context is cancelled after 30min, but the provider does not terminate the request and simply waits until it gets the response back, then discards the response and throws that error. Definitely there is something to check and improve in that part.

[bpg]

PR with the proposed fix

• fix(vm): resize image once imported  #753