borg check --verify-data performance suggestions?

[dahjelle]

I'd like to verify my backups with borg check --verify-data, but it's rather slow (around 4 hours). I'm running it on the same server, not over SSH or anything. The deduplicated size of all archives is around 270 GB, and about 20 archives.

It appears that --max-duration does not work for --verify-data, otherwise that option would suit my needs admirably.

In this case, I'm primarily concerned about the data integrity of the most recent archive and of the repository.

Would adding the --last 1 flag to the command mean only the last archive is checked, thus speeding things up around 20x? Am I "only" missing the check of the other 19 archives in that case?

[ThomasWaldmann]

borg check does 2 things (see docs for more details):

• a low-level repository check (read and check the crc32 of all data in the repo)
• a high-level archives check (normally only the metadata, but if you give --verify-data it checks also the content data - this is a cryptographic check, much stronger than the crc32 check)

The low-level check will always take the same time.

The high-level metadata check depends on the archives count checked, because each archive is checked separately.

The high-level "--verify-data" content data check does not depend on the amount of archives, because it uses Repository.scan() to look at each chunk only once, in the order as the chunks are found in the segment files.