Golden test suite for snapshot codec #499
Conversation
recursion-ninja
requested review from
dcoutts,
jorisdral,
mheinzel and
wenkokke
as code owners
|
So a big question I have is should this be a stand alone test suite (as it currently is) or should I graft it into the "monolithic" LST Tree test suite. I think the later might be more appropriate, and would appreciate a pointer as to where a good place in the |
jorisdral
changed the title
recursion-ninja
force-pushed
the
recursion-ninja/golden-test-suite
branch
3 times, most recently
from
8b0d38e to
4c00cf4
Compare
…e for serialization backwards compatibility testing.
recursion-ninja
force-pushed
the
recursion-ninja/golden-test-suite
branch
from
4c00cf4 to
00e7a07
Compare
| @@ -70,7 +70,7 @@ instance RefCounted m (MergingRun m h) where | |||
| getRefCounter = mergeRefCounter | |||
|
|
|||
| data MergePolicyForLevel = LevelTiering | LevelLevelling | |||
| deriving stock (Show, Eq) | |||
There was a problem hiding this comment.
I think you might have missed this suggestion: #499 (review)
There was a problem hiding this comment.
BTW, did we end up using these instances for the golden tests? It doesn't look like it -- if that's the case, let's remove these instances
| @@ -47,6 +49,7 @@ tests = testGroup "Test.Database.LSMTree.Internal.Snapshot.Codec" [ | |||
| testAll $ \(p :: Proxy a) -> | |||
| testGroup (show $ typeRep p) $ | |||
| prop_arbitraryAndShrinkPreserveInvariant @a deepseqInvariant | |||
| , goldenFileTests | |||
There was a problem hiding this comment.
The typical approach we use in this test suite is to include the tests at the top level, so in the Main module
| -- | | ||
| -- Compare the serialization of snapshot metadata with a known reference file. | ||
| goldenFileTests :: TestTree | ||
| goldenFileTests = handleOutputFiles . testGroup "Golden File Comparisons" $ |
There was a problem hiding this comment.
| goldenFileTests = handleOutputFiles . testGroup "Golden File Comparisons" $ | |
| goldenFileTests = handleOutputFiles . testGroup "Test.Database.LSMTree.Internal.Snapshot.Codec.Golden" $ |
| -- | | ||
| -- Compare the serialization of snapshot metadata with a known reference file. |
There was a problem hiding this comment.
Is the newline intentional? Does haddock render it correctly?
| import System.FS.IO (HandleIO, ioHasFS) | ||
| import qualified Test.Tasty as Tasty | ||
| import Test.Tasty (TestName, TestTree, testGroup) | ||
| import qualified Test.Tasty.Golden as Au |
There was a problem hiding this comment.
Au as in the chemical symbol for gold? 😄
| testCodecBuilder tName metadata = | ||
| testGroup tName $ uncurry snapshotCodecTest <$> metadata | ||
|
|
||
| type ComponentAnnotation = String |
There was a problem hiding this comment.
It was not immediately clear to me what String is used for. Maybe add some documentation?
| enumerateSnapshotLabel :: [(ComponentAnnotation, SnapshotLabel)] | ||
| enumerateSnapshotLabel = | ||
| [ ("Bs", SnapshotLabel "UserProvidedLabel") | ||
| , ("Bn", SnapshotLabel "") | ||
| ] |
There was a problem hiding this comment.
So I think I understand now that the ComponentAnnotation is used to name the specific enumerations, and this is reflected in the names of the golden files. Can you explain how you picked these labels? And the labels for the other enumerators below?
Description
Add the golden test cases to ensure that metadata serialization does not accidentally change. This will help ensure we maintain backwards compatibility. Changes in backwards compatibility should case test case failures.