Persist write buffer when creating a snapshot #478

wenkokke · 2024-11-22T12:07:00Z

This is a WIP PR that serialises the write buffer when creating a snapshot.

I'd love some feedback :)

jorisdral

Looks promising!

I think this works for writing out the write buffer. It would be good to add a test that exercises writeWriteBuffer directly. Maybe we could test that the WriteBufferWriter and RunBuilder produce the same keyops and blob files?

src/Database/LSMTree/Internal.hs

src/Database/LSMTree/Internal/Paths.hs

src/Database/LSMTree/Internal/WriteBufferWriter.hs

wenkokke · 2024-11-25T15:47:28Z

Looks promising!

I think this works for writing out the write buffer. It would be good to add a test that exercises writeWriteBuffer directly. Maybe we could test that the WriteBufferWriter and RunBuilder produce the same keyops and blob files?

Is there any testing infrastructure that would allow me to easily set up the required inputs?

wenkokke · 2024-11-25T15:51:58Z

Looks promising!

I think this works for writing out the write buffer. It would be good to add a test that exercises writeWriteBuffer directly. Maybe we could test that the WriteBufferWriter and RunBuilder produce the same keyops and blob files?

Do you have any initial feelings regarding the amount of duplication?

jorisdral · 2024-11-25T15:53:20Z

Looks promising!
I think this works for writing out the write buffer. It would be good to add a test that exercises writeWriteBuffer directly. Maybe we could test that the WriteBufferWriter and RunBuilder produce the same keyops and blob files?

Is there any testing infrastructure that would allow me to easily set up the required inputs?

https://github.com/IntersectMBO/lsm-tree/blob/d11c709cb3c3b91d9f66e94620017e0edd997528/src-extras/Database/LSMTree/Extras/RunData.hs

^ contains some definitions for generating data for runs, which is equivalent to data for write buffers. It's just sorted key/value data without duplicate keys

lsm-tree/test/Test/Database/LSMTree/Internal/Run.hs

Lines 179 to 212 in d11c709

    
           -- | Loading a run (written out from a write buffer) from disk gives the same 
        
           -- in-memory representation as the original run. 
        
           -- 
        
           -- @openFromDisk . flush === flush@ 
        
           prop_WriteAndOpen :: 
        
                FS.HasFS IO h 
        
             -> FS.HasBlockIO IO h 
        
             -> RunData KeyForIndexCompact SerialisedValue SerialisedBlob 
        
             -> IO Property 
        
           prop_WriteAndOpen fs hbio wb = 
        
               withRun fs hbio (simplePath 1337) (serialiseRunData wb) $ \written -> 
        
               withTempRegistry $ \reg -> do 
        
                 let paths = Run.runRunFsPaths written 
        
                     paths' = paths { runNumber = RunNumber 17} 
        
                 hardLinkRunFiles reg fs hbio paths paths' 
        
                 loaded <- openFromDisk fs hbio CacheRunData (simplePath 17) 
        
                 (RefCount 1 @=?) =<< readRefCount (runRefCounter written) 
        
                 (RefCount 1 @=?) =<< readRefCount (runRefCounter loaded) 
        
                 runNumEntries written @=? runNumEntries loaded 
        
                 runFilter written @=? runFilter loaded 
        
                 runIndex written @=? runIndex loaded 
        
                 writtenKOps <- readKOps Nothing written 
        
                 loadedKOps <- readKOps Nothing loaded 
        
                 assertEqual "k/ops" writtenKOps loadedKOps 
        
                 -- make sure runs get closed again 
        
                 removeReference loaded 
        
                 -- TODO: return a proper Property instead of using assertEqual etc. 
        
                 return (property True)

^ this test could give you an idea of how to use the run data.

jorisdral · 2024-11-25T15:56:40Z

Looks promising!
I think this works for writing out the write buffer. It would be good to add a test that exercises writeWriteBuffer directly. Maybe we could test that the WriteBufferWriter and RunBuilder produce the same keyops and blob files?

Do you have any initial feelings regarding the amount of duplication?

Oops, yes, I meant to write about that but I forgot. I think it's okay to have the duplication for now, as long as we

Put down a TODO (maybe in the module header?) to investigate whether we can reduce the amount of duplication. I might have some ideas, but it would mean a larger refactoring effort which I don't think is worth it to do right away
Establish some correspondence between RunBuilder and WriteBufferWriter, hence the suggestion to add a test

wenkokke · 2024-11-25T16:11:04Z

Looks promising!

I think this works for writing out the write buffer. It would be good to add a test that exercises writeWriteBuffer directly. Maybe we could test that the WriteBufferWriter and RunBuilder produce the same keyops and blob files?

Do you have any initial feelings regarding the amount of duplication?

Oops, yes, I meant to write about that but I forgot. I think it's okay to have the duplication for now, as long as we

Put down a TODO (maybe in the module header?) to investigate whether we can reduce the amount of duplication. I might have some ideas, but it would mean a larger refactoring effort which I don't think is worth it to do right away

Establish some correspondence between RunBuilder and WriteBufferWriter, hence the suggestion to add a test

Fair! I still think the generalisation of the run infrastructure to permit empty filters and indices might be worthwhile, especially with SPJ's recent "specialise with value" extension.

jorisdral · 2024-11-25T17:27:01Z

Looks promising!

I think this works for writing out the write buffer. It would be good to add a test that exercises writeWriteBuffer directly. Maybe we could test that the WriteBufferWriter and RunBuilder produce the same keyops and blob files?

Do you have any initial feelings regarding the amount of duplication?

Oops, yes, I meant to write about that but I forgot. I think it's okay to have the duplication for now, as long as we

Put down a TODO (maybe in the module header?) to investigate whether we can reduce the amount of duplication. I might have some ideas, but it would mean a larger refactoring effort which I don't think is worth it to do right away

Establish some correspondence between RunBuilder and WriteBufferWriter, hence the suggestion to add a test

Fair! I still think the generalisation of the run infrastructure to permit empty filters and indices might be worthwhile, especially with SPJ's recent "specialise with value" extension.

True, though we wouldn't be able to take advantage of that on older GHCs... unless that stuff is backported

wenkokke · 2024-11-25T17:35:02Z

Looks promising!

I think this works for writing out the write buffer. It would be good to add a test that exercises writeWriteBuffer directly. Maybe we could test that the WriteBufferWriter and RunBuilder produce the same keyops and blob files?

Do you have any initial feelings regarding the amount of duplication?

Oops, yes, I meant to write about that but I forgot. I think it's okay to have the duplication for now, as long as we

Put down a TODO (maybe in the module header?) to investigate whether we can reduce the amount of duplication. I might have some ideas, but it would mean a larger refactoring effort which I don't think is worth it to do right away

Establish some correspondence between RunBuilder and WriteBufferWriter, hence the suggestion to add a test

Fair! I still think the generalisation of the run infrastructure to permit empty filters and indices might be worthwhile, especially with SPJ's recent "specialise with value" extension.

True, though we wouldn't be able to take advantage of that on older GHCs... unless that stuff is backported

We could do something a bit unhinged and encode the flag using instances of a type class. That way it's easily backwards compatible.

wenkokke · 2024-11-29T16:07:03Z

Added the test prop_WriteRunEqWriteWriteBuffer that compares serialisation as Run versus serialisation as WriteBuffer, which fails:

lsm-tree
  Database.LSMTree.Internal.Run
    Write buffer to disk
      prop_WriteRunEqWriteWriteBuffer: FAIL (0.01s)
        *** Failed! (after 4 tests and 2 shrinks):
        Exception:
          Assertion failed
          CallStack (from HasCallStack):
            assert, called at src/Database/LSMTree/Internal/RawBytes.hs:157:5 in lsm-tree-0.1.0.0-inplace:Database.LSMTree.Internal.RawBytes
        WB {unWB = fromList [(SerialisedKey [],Delete)]}
        Use --quickcheck-replay="(SMGen 4977141403849015630 6713696729127803493,3)" to reproduce.

jorisdral · 2024-12-02T12:17:48Z

Added the test prop_WriteRunEqWriteWriteBuffer that compares serialisation as Run versus serialisation as WriteBuffer, which fails:

The assertion at the specified line of the code asserts that keys should be longer than 8 bytes. The Arbitrary instance for WriteBuffer probably does not satisfy this requirement... Your best bet is probably to create a WriteBuffer from RunData. I can help with that if needed

wenkokke · 2024-12-05T18:29:01Z

I've added an implementation for readWriteBuffer. However, my test fails with a double free, which I cannot seem to hunt down.

wenkokke · 2024-12-09T19:04:11Z

@jorisdral This is ready for review.

wenkokke · 2024-12-10T15:59:00Z

This is ready for review but not for merge. There appear to be some non-deterministic test failures.

wenkokke · 2024-12-10T17:27:11Z

Here's my assessment of the CI failures, for reference when I continue work on LSM next week:

# REPRODUCED with GHC 9.8.2
#
# propLockstep_RealImpl_MockFS_IOSim: FAIL (0.69s)
#   *** Failed! Falsified (after 1 test and 24 shrinks):
#   do action $ New (PrettyProxy @((Key,Value,Blob))) (TableConfig {confMergePolicy = MergePolicyLazyLevelling, confSizeRatio = Four, confWriteBufferAlloc = AllocNumEntries (NumEntries 1), confBloomFilterAlloc = AllocFixed 10, confFencePointerIndex = CompactIndex, confDiskCachePolicy = DiskCacheNone, confMergeSchedule = Incremental})
#      pure ()
#   FailureEvaluation RefNeverReleased RefId 4653
#   Allocation site: CallStack (from HasCallStack):
#     newRefWithTracker, called at src-control/Control/RefCount.hs:226:7 in lsm-tree-0.1.0.0-inplace-control:Control.RefCount
#     newRef, called at src/Database/LSMTree/Internal/WriteBufferBlobs.hs:140:5 in lsm-tree-0.1.0.0-inplace:Database.LSMTree.Internal.WriteBufferBlobs
#   Use --quickcheck-replay="(SMGen 3745502568310153392 9447962285702497153,91)" to reproduce.
cabal run lsm-tree-test -- -p '/propLockstep_RealImpl_MockFS_IOSim/' --quickcheck-replay="(SMGen 3745502568310153392 9447962285702497153,91)"

# REPRODUCED with GHC 9.8.2
#
# propLockstep_RealImpl_RealFS_IO: FAIL (11.33s)
#   *** Failed! Assertion failed (after 1 test and 158 shrinks):
#   do var1 <- action $ New (PrettyProxy @((Key,Value,Blob))) (TableConfig {confMergePolicy = MergePolicyLazyLevelling, confSizeRatio = Four, confWriteBufferAlloc = AllocNumEntries (NumEntries 2), confBloomFilterAlloc = AllocFixed 10, confFencePointerIndex = CompactIndex, confDiskCachePolicy = DiskCacheNone, confMergeSchedule = Incremental})
#      action $ Updates [(Key (KeyForIndexCompact {getKeyForIndexCompact = [121,62,225,0,0,0,0,0]}),Insert (Value (SerialisedValue [])) (Just (Blob (SerialisedBlob [0]))))] (unsafeMkGVar var1 (OpFromRight `OpComp` OpId))
#      action $ CreateSnapshot (SnapshotLabel "Key Value Blob") "snap1" (unsafeMkGVar var1 (OpFromRight `OpComp` OpId))
#      var6 <- action $ OpenSnapshot (SnapshotLabel "Key Value Blob") "snap1"
#      var10 <- action $ Duplicate (unsafeMkGVar var6 (OpFromRight `OpComp` OpId))
#      action $ Updates [(Key (KeyForIndexCompact {getKeyForIndexCompact = [0,0,0,0,0,0,0,0]}),Insert (Value (SerialisedValue [])) (Just (Blob (SerialisedBlob [1]))))] (unsafeMkGVar var10 (OpFromRight `OpComp` OpId))
#      var15 <- action $ NewCursor (Just (Key (KeyForIndexCompact {getKeyForIndexCompact = [121,62,224,96,53,92,252,220,175,239,140,57,8,65,169,5,100,247,111,202,35,151,202,139,199,203,174,214,40,157,156,189,98,28,59,226,206,234,147,114,163,92,201,7,143,220,140,176,239,130,40,54,57,93,139,13,136,97,182,85,161,139,223,24,216,231,106,90,66,136,24,105,62,156]}))) (unsafeMkGVar var10 (OpFromRight `OpComp` OpId))
#      var23 <- action $ ReadCursor 59 (unsafeMkGVar var15 (OpFromRight `OpComp` OpId))
#      action $ RetrieveBlobs (unsafeMkGVar var23 (OpQueryResults `OpComp` OpFromRight `OpComp` OpId))
#      pure ()
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 0,fromList []))], cursors = fromList [], nextID = 1, snapshots = fromList []}) (Stats {snapshotted = fromList [], numLookupsResults = (0,0,0), numUpdates = (0,0,0,0), successActions = ["New"], failActions = [], numActionsPerTable = fromList [(0,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0)], dupTableActionLog = fromList []})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 1,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 1, snapshots = fromList []}) (Stats {snapshotted = fromList [], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 1, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 2, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 3, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 3, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["Updates","Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [(3,<<Cursor Key Value Blob>>)], nextID = 4, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["NewCursor","Updates","Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,2)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [(3,<<Cursor Key Value Blob>>)], nextID = 4, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["ReadCursor","NewCursor","Updates","Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,2)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [(3,<<Cursor Key Value Blob>>)], nextID = 4, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["RetrieveBlobs","ReadCursor","NewCursor","Updates","Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,2)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2])]})
#   System under test returned: OEither (Right (OVector [OBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [1])})])) (Right [WrapBlob {unwrapBlob = Blob (SerialisedBlob [1])}])
#   but model returned:         OEither (Right (OVector [OBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [0])})])) (MEither (Right (MVector [MBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [0])})])))
#   Use --quickcheck-replay="(SMGen 6681553406898521959 16697549380326312555,88)" to reproduce.
cabal run lsm-tree-test -- -p '/propLockstep_RealImpl_RealFS_IO/' --quickcheck-replay="(SMGen 6681553406898521959 16697549380326312555,88)"

# REPRODUCED with GHC 9.8.2
#
# propLockstep_RealImpl_RealFS_IO: FAIL (73.11s)
#   *** Failed! Assertion failed (after 1 test and 285 shrinks):
#   do var27 <- action $ New (PrettyProxy @((Key,Value,Blob))) (TableConfig {confMergePolicy = MergePolicyLazyLevelling, confSizeRatio = Four, confWriteBufferAlloc = AllocNumEntries (NumEntries 2), confBloomFilterAlloc = AllocFixed 10, confFencePointerIndex = CompactIndex, confDiskCachePolicy = DiskCacheNone, confMergeSchedule = Incremental})
#      var37 <- action $ Duplicate (unsafeMkGVar var27 (OpFromRight `OpComp` OpId))
#      action $ Updates [(Key (KeyForIndexCompact {getKeyForIndexCompact = [72,0,0,0,0,0,0,0]}),Insert (Value (SerialisedValue [])) (Just (Blob (SerialisedBlob [0]))))] (unsafeMkGVar var37 (OpFromRight `OpComp` OpId))
#      action $ CreateSnapshot (SnapshotLabel "Key Value Blob") "snap3" (unsafeMkGVar var37 (OpFromRight `OpComp` OpId))
#      var59 <- action $ OpenSnapshot (SnapshotLabel "Key Value Blob") "snap3"
#      action $ Updates [(Key (KeyForIndexCompact {getKeyForIndexCompact = [0,0,0,0,0,0,0,0]}),Insert (Value (SerialisedValue [])) (Just (Blob (SerialisedBlob [1]))))] (unsafeMkGVar var59 (OpFromRight `OpComp` OpId))
#      var63 <- action $ RangeLookup (FromToExcluding (Key (KeyForIndexCompact {getKeyForIndexCompact = [71,146,85,251,201,63,13,171,108,104,235,95,72,169,31,247,251,180,104,35,142,30,40,142,208,251,88,106,217,209,67,69,8,54,185,129,193,219,230,17,30,138,26,139]})) (Key (KeyForIndexCompact {getKeyForIndexCompact = [224,160,131,208,135,90,169,220,1,9,97,98,40,28,141,128,196,117,26,157,178,207,140,15,244,111,77,4,244,27,67,169,231,191,131,248,144,5,185,46,19,6,84,160,63,255,190,249,47,109,124,98,204,187,249,98,174,182,56,126,219,98,247,211,6,56,196,196,37,200,90,195,244,206,206,146,53,111,225,29,22,70,150,191,106,130,227,204,110,36,187,39,138,10,59,67,51,177,186,184,98,108,33,87,5,68,147,155]}))) (unsafeMkGVar var59 (OpFromRight `OpComp` OpId))
#      action $ RetrieveBlobs (unsafeMkGVar var63 (OpQueryResults `OpComp` OpFromRight `OpComp` OpId))
#      pure ()
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 0,fromList []))], cursors = fromList [], nextID = 1, snapshots = fromList []}) (Stats {snapshotted = fromList [], numLookupsResults = (0,0,0), numUpdates = (0,0,0,0), successActions = ["New"], failActions = [], numActionsPerTable = fromList [(0,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0)], dupTableActionLog = fromList []})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 0,fromList [])),(1,(UpdateCounter 0,fromList []))], cursors = fromList [], nextID = 2, snapshots = fromList []}) (Stats {snapshotted = fromList [], numLookupsResults = (0,0,0), numUpdates = (0,0,0,0), successActions = ["Duplicate","New"], failActions = [], numActionsPerTable = fromList [(0,0),(1,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,0)], dupTableActionLog = fromList []})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 0,fromList [])),(1,(UpdateCounter 1,fromList [("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 2, snapshots = fromList []}) (Stats {snapshotted = fromList [], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["Updates","Duplicate","New"], failActions = [], numActionsPerTable = fromList [(0,0),(1,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,0)], dupTableActionLog = fromList [(0,[1])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 0,fromList [])),(1,(UpdateCounter 2,fromList [("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 2, snapshots = fromList [("snap3",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap3"], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["CreateSnapshot","Updates","Duplicate","New"], failActions = [], numActionsPerTable = fromList [(0,0),(1,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,0)], dupTableActionLog = fromList [(0,[1])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 0,fromList [])),(1,(UpdateCounter 2,fromList [("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 0,fromList [("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 3, snapshots = fromList [("snap3",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap3"], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["OpenSnapshot","CreateSnapshot","Updates","Duplicate","New"], failActions = [], numActionsPerTable = fromList [(0,0),(1,1),(2,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,0),(2,2)], dupTableActionLog = fromList [(0,[1])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 0,fromList [])),(1,(UpdateCounter 2,fromList [("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 3, snapshots = fromList [("snap3",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap3"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["Updates","OpenSnapshot","CreateSnapshot","Updates","Duplicate","New"], failActions = [], numActionsPerTable = fromList [(0,0),(1,1),(2,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,0),(2,2)], dupTableActionLog = fromList [(0,[1]),(2,[2])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 0,fromList [])),(1,(UpdateCounter 2,fromList [("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 3, snapshots = fromList [("snap3",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap3"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["RangeLookup","Updates","OpenSnapshot","CreateSnapshot","Updates","Duplicate","New"], failActions = [], numActionsPerTable = fromList [(0,0),(1,1),(2,2)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,0),(2,2)], dupTableActionLog = fromList [(0,[1]),(2,[2])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 0,fromList [])),(1,(UpdateCounter 2,fromList [("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 3, snapshots = fromList [("snap3",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("H\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap3"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["RetrieveBlobs","RangeLookup","Updates","OpenSnapshot","CreateSnapshot","Updates","Duplicate","New"], failActions = [], numActionsPerTable = fromList [(0,0),(1,1),(2,2)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,0),(2,2)], dupTableActionLog = fromList [(0,[1]),(2,[2])]})
#   System under test returned: OEither (Right (OVector [OBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [1])})])) (Right [WrapBlob {unwrapBlob = Blob (SerialisedBlob [1])}])
#   but model returned:         OEither (Right (OVector [OBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [0])})])) (MEither (Right (MVector [MBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [0])})])))
#   Use --quickcheck-replay="(SMGen 856599901754634088 14991145271356155781,87)" to reproduce.
cabal run lsm-tree-test -- -p '/propLockstep_RealImpl_RealFS_IO/' --quickcheck-replay="(SMGen 856599901754634088 14991145271356155781,87)"

# REPRODUCED with GHC 9.8.2
#
# propLockstep_RealImpl_RealFS_IO: FAIL (11.40s)
#   *** Failed! Assertion failed (after 1 test and 158 shrinks):
#   do var1 <- action $ New (PrettyProxy @((Key,Value,Blob))) (TableConfig {confMergePolicy = MergePolicyLazyLevelling, confSizeRatio = Four, confWriteBufferAlloc = AllocNumEntries (NumEntries 2), confBloomFilterAlloc = AllocFixed 10, confFencePointerIndex = CompactIndex, confDiskCachePolicy = DiskCacheNone, confMergeSchedule = Incremental})
#      action $ Updates [(Key (KeyForIndexCompact {getKeyForIndexCompact = [121,62,225,0,0,0,0,0]}),Insert (Value (SerialisedValue [])) (Just (Blob (SerialisedBlob [0]))))] (unsafeMkGVar var1 (OpFromRight `OpComp` OpId))
#      action $ CreateSnapshot (SnapshotLabel "Key Value Blob") "snap1" (unsafeMkGVar var1 (OpFromRight `OpComp` OpId))
#      var6 <- action $ OpenSnapshot (SnapshotLabel "Key Value Blob") "snap1"
#      var10 <- action $ Duplicate (unsafeMkGVar var6 (OpFromRight `OpComp` OpId))
#      action $ Updates [(Key (KeyForIndexCompact {getKeyForIndexCompact = [0,0,0,0,0,0,0,0]}),Insert (Value (SerialisedValue [])) (Just (Blob (SerialisedBlob [1]))))] (unsafeMkGVar var10 (OpFromRight `OpComp` OpId))
#      var15 <- action $ NewCursor (Just (Key (KeyForIndexCompact {getKeyForIndexCompact = [121,62,224,96,53,92,252,220,175,239,140,57,8,65,169,5,100,247,111,202,35,151,202,139,199,203,174,214,40,157,156,189,98,28,59,226,206,234,147,114,163,92,201,7,143,220,140,176,239,130,40,54,57,93,139,13,136,97,182,85,161,139,223,24,216,231,106,90,66,136,24,105,62,156]}))) (unsafeMkGVar var10 (OpFromRight `OpComp` OpId))
#      var23 <- action $ ReadCursor 59 (unsafeMkGVar var15 (OpFromRight `OpComp` OpId))
#      action $ RetrieveBlobs (unsafeMkGVar var23 (OpQueryResults `OpComp` OpFromRight `OpComp` OpId))
#      pure ()
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 0,fromList []))], cursors = fromList [], nextID = 1, snapshots = fromList []}) (Stats {snapshotted = fromList [], numLookupsResults = (0,0,0), numUpdates = (0,0,0,0), successActions = ["New"], failActions = [], numActionsPerTable = fromList [(0,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0)], dupTableActionLog = fromList []})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 1,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 1, snapshots = fromList []}) (Stats {snapshotted = fromList [], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 1, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 2, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 3, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 3, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["Updates","Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [(3,<<Cursor Key Value Blob>>)], nextID = 4, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["NewCursor","Updates","Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,2)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [(3,<<Cursor Key Value Blob>>)], nextID = 4, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["ReadCursor","NewCursor","Updates","Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,2)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [(3,<<Cursor Key Value Blob>>)], nextID = 4, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["RetrieveBlobs","ReadCursor","NewCursor","Updates","Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,2)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2])]})
#   System under test returned: OEither (Right (OVector [OBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [1])})])) (Right [WrapBlob {unwrapBlob = Blob (SerialisedBlob [1])}])
#   but model returned:         OEither (Right (OVector [OBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [0])})])) (MEither (Right (MVector [MBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [0])})])))
#   Use --quickcheck-replay="(SMGen 6681553406898521959 16697549380326312555,88)" to reproduce.
cabal run lsm-tree-test --  -p '/propLockstep_RealImpl_RealFS_IO/' --quickcheck-replay="(SMGen 8643971879424937837 17377864027749461975,88)"

# REPRODUCED with GHC 9.8.2
#
# propLockstep_RealImpl_RealFS_IO: FAIL (11.73s)
#   *** Failed! Assertion failed (after 1 test and 158 shrinks):
#   do var1 <- action $ New (PrettyProxy @((Key,Value,Blob))) (TableConfig {confMergePolicy = MergePolicyLazyLevelling, confSizeRatio = Four, confWriteBufferAlloc = AllocNumEntries (NumEntries 2), confBloomFilterAlloc = AllocFixed 10, confFencePointerIndex = CompactIndex, confDiskCachePolicy = DiskCacheNone, confMergeSchedule = Incremental})
#      action $ Updates [(Key (KeyForIndexCompact {getKeyForIndexCompact = [121,62,225,0,0,0,0,0]}),Insert (Value (SerialisedValue [])) (Just (Blob (SerialisedBlob [0]))))] (unsafeMkGVar var1 (OpFromRight `OpComp` OpId))
#      action $ CreateSnapshot (SnapshotLabel "Key Value Blob") "snap1" (unsafeMkGVar var1 (OpFromRight `OpComp` OpId))
#      var6 <- action $ OpenSnapshot (SnapshotLabel "Key Value Blob") "snap1"
#      var10 <- action $ Duplicate (unsafeMkGVar var6 (OpFromRight `OpComp` OpId))
#      action $ Updates [(Key (KeyForIndexCompact {getKeyForIndexCompact = [0,0,0,0,0,0,0,0]}),Insert (Value (SerialisedValue [])) (Just (Blob (SerialisedBlob [1]))))] (unsafeMkGVar var10 (OpFromRight `OpComp` OpId))
#      var15 <- action $ NewCursor (Just (Key (KeyForIndexCompact {getKeyForIndexCompact = [121,62,224,96,53,92,252,220,175,239,140,57,8,65,169,5,100,247,111,202,35,151,202,139,199,203,174,214,40,157,156,189,98,28,59,226,206,234,147,114,163,92,201,7,143,220,140,176,239,130,40,54,57,93,139,13,136,97,182,85,161,139,223,24,216,231,106,90,66,136,24,105,62,156]}))) (unsafeMkGVar var10 (OpFromRight `OpComp` OpId))
#      var23 <- action $ ReadCursor 59 (unsafeMkGVar var15 (OpFromRight `OpComp` OpId))
#      action $ RetrieveBlobs (unsafeMkGVar var23 (OpQueryResults `OpComp` OpFromRight `OpComp` OpId))
#      pure ()
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 0,fromList []))], cursors = fromList [], nextID = 1, snapshots = fromList []}) (Stats {snapshotted = fromList [], numLookupsResults = (0,0,0), numUpdates = (0,0,0,0), successActions = ["New"], failActions = [], numActionsPerTable = fromList [(0,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0)], dupTableActionLog = fromList []})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 1,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 1, snapshots = fromList []}) (Stats {snapshotted = fromList [], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 1, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 2, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 3, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 3, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["Updates","Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [(3,<<Cursor Key Value Blob>>)], nextID = 4, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["NewCursor","Updates","Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,2)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [(3,<<Cursor Key Value Blob>>)], nextID = 4, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["ReadCursor","NewCursor","Updates","Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,2)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [(3,<<Cursor Key Value Blob>>)], nextID = 4, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("y>\225\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["RetrieveBlobs","ReadCursor","NewCursor","Updates","Duplicate","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0),(2,2)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2])]})
#   System under test returned: OEither (Right (OVector [OBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [1])})])) (Right [WrapBlob {unwrapBlob = Blob (SerialisedBlob [1])}])
#   but model returned:         OEither (Right (OVector [OBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [0])})])) (MEither (Right (MVector [MBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [0])})])))
#   Use --quickcheck-replay="(SMGen 6681553406898521959 16697549380326312555,88)" to reproduce.
cabal run lsm-tree-test -- -p '/propLockstep_RealImpl_RealFS_IO/' --quickcheck-replay="(SMGen 6681553406898521959 16697549380326312555,88)"

# REPRODUCED with GHC 9.8.2
#
# propLockstep_RealImpl_MockFS_IO: FAIL (41.87s)
#   *** Failed! Assertion failed (after 1 test and 280 shrinks):
#   do var1 <- action $ New (PrettyProxy @((Key,Value,Blob))) (TableConfig {confMergePolicy = MergePolicyLazyLevelling, confSizeRatio = Four, confWriteBufferAlloc = AllocNumEntries (NumEntries 2), confBloomFilterAlloc = AllocFixed 10, confFencePointerIndex = CompactIndex, confDiskCachePolicy = DiskCacheNone, confMergeSchedule = Incremental})
#      action $ Updates [(Key (KeyForIndexCompact {getKeyForIndexCompact = [6,0,0,0,0,0,0,0]}),Insert (Value (SerialisedValue [])) (Just (Blob (SerialisedBlob [0]))))] (unsafeMkGVar var1 (OpFromRight `OpComp` OpId))
#      action $ CreateSnapshot (SnapshotLabel "Key Value Blob") "snap1" (unsafeMkGVar var1 (OpFromRight `OpComp` OpId))
#      var17 <- action $ OpenSnapshot (SnapshotLabel "Key Value Blob") "snap1"
#      action $ Updates [(Key (KeyForIndexCompact {getKeyForIndexCompact = [0,0,0,0,0,0,0,0]}),Insert (Value (SerialisedValue [])) (Just (Blob (SerialisedBlob [1]))))] (unsafeMkGVar var17 (OpFromRight `OpComp` OpId))
#      var21 <- action $ Duplicate (unsafeMkGVar var17 (OpFromRight `OpComp` OpId))
#      var22 <- action $ RangeLookup (FromToExcluding (Key (KeyForIndexCompact {getKeyForIndexCompact = [5,227,86,48,98,173,194,45,23,220,116,97,188,153,32,140,117,157,159,43,45,43,242,245,16,140,191,188,124,233,23,63,85,237,202,154,113,142,157,208,240,248,83,145,199,253,69,169,190,187,24,25,88,33,108,25,14,244,56,122,151,94,143,32,204,202,158,250,245,50,80,115,134,68,28]})) (Key (KeyForIndexCompact {getKeyForIndexCompact = [174,85,235,200,159,46,48,93,200,144,34,95,64,87,157,140,51,172,50,35,76,160,1,194,109,98,172,31,8,59,34,148,207,129,98,149,32,134,185,35,132,60,207,237,165,138,29,101,62,91,129,215,181,104,17,219,145,130,143,127,115,72,8,115,106,126,242,253,61,105,138,85,18,230,76,94,231,165,227,235,163,181,174,79,74,188,81,84,34,20,222,198,143,93,228,179,247,197,158,94,151,120,215,234,36,34,3,154]}))) (unsafeMkGVar var21 (OpFromRight `OpComp` OpId))
#      action $ RetrieveBlobs (unsafeMkGVar var22 (OpQueryResults `OpComp` OpFromRight `OpComp` OpId))
#      pure ()
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 0,fromList []))], cursors = fromList [], nextID = 1, snapshots = fromList []}) (Stats {snapshotted = fromList [], numLookupsResults = (0,0,0), numUpdates = (0,0,0,0), successActions = ["New"], failActions = [], numActionsPerTable = fromList [(0,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0)], dupTableActionLog = fromList []})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 1,fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 1, snapshots = fromList []}) (Stats {snapshotted = fromList [], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 1, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 0,fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 2, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,1,0,0), successActions = ["OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1)], dupTableActionLog = fromList [(0,[0])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 2, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["Updates","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1)], dupTableActionLog = fromList [(0,[0]),(1,[1])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 0,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 3, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["Duplicate","Updates","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,1),(2,0)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[1])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 0,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 3, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["RangeLookup","Duplicate","Updates","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,1),(2,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2,1])]})
#   State: ModelState (Model {tables = fromList [(0,(UpdateCounter 2,fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(1,(UpdateCounter 1,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])),(2,(UpdateCounter 0,fromList [("\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\SOH"),("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")]))], cursors = fromList [], nextID = 3, snapshots = fromList [("snap1",Snapshot TableConfig (SnapshotLabel "Key Value Blob") fromList [("\ACK\NUL\NUL\NUL\NUL\NUL\NUL\NUL","",Just "\NUL")])]}) (Stats {snapshotted = fromList ["snap1"], numLookupsResults = (0,0,0), numUpdates = (0,2,0,0), successActions = ["RetrieveBlobs","RangeLookup","Duplicate","Updates","OpenSnapshot","CreateSnapshot","Updates","New"], failActions = [], numActionsPerTable = fromList [(0,1),(1,1),(2,1)], closedTableSizes = fromList [], parentTable = fromList [(0,0),(1,1),(2,1)], dupTableActionLog = fromList [(0,[0]),(1,[2,1])]})
#   System under test returned: OEither (Right (OVector [OBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [1])})])) (Right [WrapBlob {unwrapBlob = Blob (SerialisedBlob [1])}])
#   but model returned:         OEither (Right (OVector [OBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [0])})])) (MEither (Right (MVector [MBlob (WrapBlob {unwrapBlob = Blob (SerialisedBlob [0])})])))
#   Use --quickcheck-replay="(SMGen 7870052982708161788 8378283829282625597,88)" to reproduce.
cabal run lsm-tree-test -- -p '$0=="lsm-tree.Test.Database.LSMTree.StateMachine.propLockstep_RealImpl_MockFS_IO"' --quickcheck-replay="(SMGen 7870052982708161788 8378283829282625597,88)"

# CANNOT REPRODUCE
cabal run lsm-tree-test -- -p '/prop_example/' --quickcheck-replay="(SMGen 5733627160528804339 16188990791829584463,0)"

# CANNOT REPRODUCE (the original error is a timeout with GHC 8.10.7)
cabal run lsm-tree-test -- -p '$0=="lsm-tree.Test.Database.LSMTree.StateMachine.propLockstep_RealImpl_MockFS_IO"'

wenkokke · 2024-12-19T16:16:13Z

This is ready for review

jorisdral · 2025-01-02T15:03:20Z

A general note: it would have been easier to review the PR if the commits had been more structured and informative

I'll admit I didn't structure my commit much. However, do we have any guidelines for structuring commits and commit messages? Something like conventional commits?

We don't have any guidelines, it's rather free form.

I like the style that @dcoutts uses, for example in https://github.com/IntersectMBO/lsm-tree/pull/417/commits, because it creates kind of a story with a flow. Each commit generally changes some smallish thing and the message describes the goal, and if the changes are non-trivial, the messages describes the why and how. Subsequent commits then build on top of previous commit, which advances the story.

IMO the smaller the commit with a clearly defined goal, the easier it is to review. Contrast that with less structure, for example just one larger commit, then it's less straightforward where to start, and in which order to review different parts of the diff, and to know which details are important and which aren't. In general, it's easier to sign off on a small diff that tackles only a single thing (or a few things).

I can't speak for the review experience of my own PRs because I am not the one reviewing them, but I typically spend time rewriting the commit history to also craft a bit of a story. I don't really have rules for myself on how granular to make commits, but as a baseline I make sure that each commit can build and pass tests individually.

wenkokke · 2025-01-02T15:29:26Z

@jorisdral There's two remaining open discussions:

Do you have any strong feelings either way on whether or not these should be implemented before merging?

jorisdral

Seems like almost everything is resolved. I have two commits to include in this PR, however, if that's okay with you @wenkokke ?

test/Test/Database/LSMTree/Internal/Run.hs

src/Database/LSMTree/Internal/WriteBufferReader.hs

jorisdral

LGTM! Let's squash/rewrite the history a bit, and then we can merge

This is required for properly modelling the model's response to disk faults in `createSnapshot`. The current implementation for `createSnapshot` in the real system can invalidate blob references even if creating the snapshot failed, so the model will invalidate blob references as well. This change might become obsolete once #478 is merged.

chore: run pre-commit

This commit fixes the error in the open function for WriteBufferBlobs, but also contains a significant amount of refactoring based on feedback on the PR.

Co-authored-by: Joris Dral <[email protected]>

This is required for properly modelling the model's response to disk faults in `createSnapshot`. The current implementation for `createSnapshot` in the real system can invalidate blob references even if creating the snapshot failed, so the model will invalidate blob references as well. This change might become obsolete once #478 is merged.

wenkokke requested review from dcoutts, jorisdral, mheinzel and recursion-ninja as code owners November 22, 2024 12:07

wenkokke marked this pull request as draft November 22, 2024 12:07

jorisdral reviewed Nov 25, 2024

View reviewed changes

wenkokke force-pushed the wenkokke/issue392-2 branch 3 times, most recently from 8e1f2bf to 6a7eee5 Compare December 3, 2024 13:22

wenkokke force-pushed the wenkokke/issue392-2 branch from eb55cf0 to b472261 Compare December 6, 2024 09:21

wenkokke marked this pull request as ready for review December 9, 2024 19:04

wenkokke force-pushed the wenkokke/issue392-2 branch from f5ad722 to c7d5451 Compare December 10, 2024 12:32

wenkokke changed the title ~~WIP: serialise write buffer when creating a snapshot~~ Persist write buffer when creating a snapshot Dec 10, 2024

wenkokke force-pushed the wenkokke/issue392-2 branch 2 times, most recently from 19be2a2 to d68478f Compare December 19, 2024 15:28

wenkokke requested a review from jorisdral December 19, 2024 16:16

jorisdral reviewed Jan 2, 2025

View reviewed changes

test/Test/Database/LSMTree/Internal/Run.hs Show resolved Hide resolved

test/Test/Database/LSMTree/Internal/Run.hs Outdated Show resolved Hide resolved

src/Database/LSMTree/Internal/WriteBufferReader.hs Outdated Show resolved Hide resolved

wenkokke force-pushed the wenkokke/issue392-2 branch 2 times, most recently from 8ca3798 to 35be751 Compare January 2, 2025 17:11

jorisdral approved these changes Jan 2, 2025

View reviewed changes

wenkokke force-pushed the wenkokke/issue392-2 branch 3 times, most recently from e0398ef to 71f541a Compare January 2, 2025 17:42

wenkokke and others added 11 commits January 2, 2025 17:43

feat(issue392): add WriteBufferWriter

d8186fc

feat(issue392): write the WriteBuffer in createSnapshot

ad670e8

fix(issue392): process feedback on PR

23ac52e

fix(issue392): add SerialisedWriteBuffer

5f55c46

feat(issue392): add WriteBufferReader

7e171ad

fix(issue392): refactor tests

4408fff

chore: run pre-commit

fix(issue392): rebase and use new refcount API

89d9c18

feat(issue392): write/open write buffer in snapshot

787c194

fix(issue392): instance of Encode for SnapshotMetaData

923f2dd

fix(issue392): set blob file pointer to end

dbd0bcd

This commit fixes the error in the open function for WriteBufferBlobs, but also contains a significant amount of refactoring based on feedback on the PR.

fix(issue392): process feedback on PR

1efe0e0

Co-authored-by: Joris Dral <[email protected]>

wenkokke force-pushed the wenkokke/issue392-2 branch from 71f541a to 1efe0e0 Compare January 2, 2025 17:43

wenkokke added this pull request to the merge queue Jan 2, 2025

Merged via the queue into main with commit 840c26a Jan 2, 2025
27 checks passed

wenkokke deleted the wenkokke/issue392-2 branch January 2, 2025 18:49

wenkokke mentioned this pull request Jan 3, 2025

Change how snapshot handles flushing of write buffer #392

Closed

jorisdral linked an issue Jan 3, 2025 that may be closed by this pull request

Change how snapshot handles flushing of write buffer #392

Closed

wenkokke mentioned this pull request Jan 3, 2025

Share code between RunBuilder/WriteBufferWriter and RunReader/WriteBufferReader #516

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Persist write buffer when creating a snapshot #478

Persist write buffer when creating a snapshot #478

wenkokke commented Nov 22, 2024

jorisdral left a comment

wenkokke commented Nov 25, 2024

wenkokke commented Nov 25, 2024

jorisdral commented Nov 25, 2024

jorisdral commented Nov 25, 2024

wenkokke commented Nov 25, 2024

jorisdral commented Nov 25, 2024

wenkokke commented Nov 25, 2024

wenkokke commented Nov 29, 2024

jorisdral commented Dec 2, 2024 •

edited

Loading

wenkokke commented Dec 5, 2024

wenkokke commented Dec 9, 2024

wenkokke commented Dec 10, 2024

wenkokke commented Dec 10, 2024

wenkokke commented Dec 19, 2024

jorisdral commented Jan 2, 2025

wenkokke commented Jan 2, 2025

jorisdral left a comment

jorisdral left a comment

Persist write buffer when creating a snapshot #478

Persist write buffer when creating a snapshot #478

Conversation

wenkokke commented Nov 22, 2024

jorisdral left a comment

Choose a reason for hiding this comment

wenkokke commented Nov 25, 2024

wenkokke commented Nov 25, 2024

jorisdral commented Nov 25, 2024

jorisdral commented Nov 25, 2024

wenkokke commented Nov 25, 2024

jorisdral commented Nov 25, 2024

wenkokke commented Nov 25, 2024

wenkokke commented Nov 29, 2024

jorisdral commented Dec 2, 2024 • edited Loading

wenkokke commented Dec 5, 2024

wenkokke commented Dec 9, 2024

wenkokke commented Dec 10, 2024

wenkokke commented Dec 10, 2024

wenkokke commented Dec 19, 2024

jorisdral commented Jan 2, 2025

wenkokke commented Jan 2, 2025

jorisdral left a comment

Choose a reason for hiding this comment

jorisdral left a comment

Choose a reason for hiding this comment

jorisdral commented Dec 2, 2024 •

edited

Loading