Skip to content

Commit

Permalink
metamorphic: tweak L0StopWritesThreshold in case of small files
Browse files Browse the repository at this point in the history
Small sstables lead to slower compactions as more time is spent
in compaction picking and opening/closing files and iterators than
iterating over sstables. A combination of a tiny target file size
and a low L0 stop writes threshold can cause a deep write stall
that makes some operations time out. This change tweaks L0StopWritesThreshold
to let us wait longer in those instances without timing out.

Partly seen in #3947.
  • Loading branch information
itsbilal committed Oct 24, 2024
1 parent efa5401 commit faa0ea5
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions metamorphic/options.go
Original file line number Diff line number Diff line change
Expand Up @@ -519,6 +519,8 @@ func standardOptions() []*TestOptions {
index_block_size=1
`,
15: `
[Options]
l0_stop_writes_threshold=100
[Level "0"]
target_file_size=12
`,
Expand Down Expand Up @@ -740,6 +742,12 @@ func RandomOptions(
lopts.BlockSizeThreshold = 50 + rng.IntN(50) // 50 - 100
lopts.IndexBlockSize = 1 << uint(rng.IntN(24)) // 1 - 16MB
lopts.TargetFileSize = 1 << uint(rng.IntN(28)) // 1 - 256MB
if lopts.TargetFileSize < 1<<12 {
// We will generate a lot of files, which will slow down compactions.
// Increase L0StopWritesThreshold to reduce the number of write stalls
// that could cause operation timeouts.
opts.L0StopWritesThreshold = 100
}
// The EstimatedSize of an empty table writer is 8 bytes. We want something a
// little bigger than that as the minimum target.
lopts.TargetFileSize = max(lopts.TargetFileSize, 12)
Expand Down

0 comments on commit faa0ea5

Please sign in to comment.