Backup of big machines fails with a 'Broken pipe'

[furby72]

Hi there,

i am using vzborg on a Proxmov 7.1-5 and it works perfectly with small containers an virtual machines. Thank you for the good work. But one of my VM's disk is filled with about 1,1 TB of Data. This backup always fails with a broken pipeline with the following errors:
.
.
.
INFO: 31% (357.1 GiB of 1.1 TiB) in 1h 40m 34s, read: 18.4 MiB/s, write: 18.4 MiB/s
INFO: 32% (368.7 GiB of 1.1 TiB) in 1h 48m 57s, read: 23.5 MiB/s, write: 23.5 MiB/s
INFO: 32% (377.8 GiB of 1.1 TiB) in 1h 55m 46s, read: 22.8 MiB/s, write: 22.7 MiB/s
ERROR: vma_queue_write: write error - Broken pipe
INFO: aborting backup job
INFO: stopping kvm after backup task
ERROR: Backup of VM 101 failed - vma_queue_write: write error - Broken pipe
INFO: Failed at 2021-11-21 19:14:33
INFO: Backup job finished with errors
job errors
/usr/local/bin/vzborg: line 188: 148722 Exit 255 vzdump ${VM_ID} --dumpdir ${temp_dir} --quiet ${vzdump_quiet} --compress 0 --mode ${vzdump_mode} --stdout
148723 Killed | borg create ${borg_extra} --files-cache=disabled --stdin-name=vzborgdump --chunker-params ${default_chunker} -s "${backup_compression}" "${backup_comment}" ${vzborg_repo}::${backup_name} - 2>&1
148724 Done | tee -a ${detail_file}
.
.
.

Do you have a suggestion what i could do?

Thank you

[furby72]

I've searched the web for a cure to this problem. It seems, that for many people the target is the problem. So i've changed the target and tried internal and external targets, but all fail at some point or later with the 'broken pipe' problem. Some people reported back missing space at the temporary filesystem, but i've monitored that during the backup-process and that could not be the problem. I've always had enough space. Perhaps there is a way to increase the buffer between vzdump and borg somehow?

[juanep97]

I doubt that you are still looking for a solution, but I just ran into the problem. The borg process was being killed by the OOM. Changing the chunker parameters to default Borg 1.4.0 parameters solves it

DEFAULT_CHUNKER='buzhash,19,23,21,4095'

See juanep97/vzborg.