-
Notifications
You must be signed in to change notification settings - Fork 92
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Raftstore v2 #389
Raftstore v2 #389
Commits on Oct 2, 2024
-
Return sequence number of writes (tikv#292)
* return sequence number of writes Signed-off-by: 5kbpers <[email protected]> * fix compile error Signed-off-by: 5kbpers <[email protected]> Signed-off-by: tabokie <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 46db718 - Browse repository at this point
Copy the full SHA 46db718View commit details -
Expose seqno for multi-batch-write (tikv#307)
* expose seqno for multi-batch-write Signed-off-by: 5kbpers <[email protected]> * format Signed-off-by: 5kbpers <[email protected]> Signed-off-by: 5kbpers <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for d2a2d6c - Browse repository at this point
Copy the full SHA d2a2d6cView commit details -
support post write callback (tikv#326)
A callback that is called after write succeeds and changes have been applied to memtable. Titan change: tikv/titan#270 Signed-off-by: tabokie <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 4ad7512 - Browse repository at this point
Copy the full SHA 4ad7512View commit details -
fix bug of using post write callback with empty batch (tikv#327)
* fix bug of using post write callback with empty batch Signed-off-by: tabokie <[email protected]> * fix nullptr Signed-off-by: tabokie <[email protected]> Signed-off-by: tabokie <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 6e06a15 - Browse repository at this point
Copy the full SHA 6e06a15View commit details -
Support instance merge (tikv#294)
Add support to merge multiple DBs that have no overlapping data (tombstone included). Memtables are frozen and then referenced by the target DB. Table files are hard linked with new file numbers into the target DB. After merge, the sequence numbers of memtables and L0 files will appear out-of-order compared to a single DB. But for any given user key, the ordering still holds because there will only be one unique source DB that contains the key and the source DB's ordering is inherited by the target DB. If source and target instances share the same block cache, target instance will be able to reuse cache. This is done by cloning the table readers of source instances to the target instance. Because the cache key is stored in table reader, reads after the merge can still retrieve source instances' blocks via old cache key. Under release build, it takes 8ms to merge a 25GB DB (500 files) into another. Signed-off-by: tabokie <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 00c70e3 - Browse repository at this point
Copy the full SHA 00c70e3View commit details -
Add largest memtable seqno and return earliest and largest seqno in O…
…nFlushBegin event (tikv#300) * add largest seqno of memtable Signed-off-by: 5kbpers <[email protected]> * add test Signed-off-by: 5kbpers <[email protected]> * address comment Signed-off-by: 5kbpers <[email protected]> * address comment Signed-off-by: 5kbpers <[email protected]> * format Signed-off-by: 5kbpers <[email protected]> * memtable info Signed-off-by: 5kbpers <[email protected]> Signed-off-by: 5kbpers <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 056d239 - Browse repository at this point
Copy the full SHA 056d239View commit details -
Schedule memtable flush across multiple instances (tikv#305)
Summary: Modify existing write buffer manager to support multiple instances. Previously, a flush is triggered before user writes if `ShouldFlush()` returns true. But in the multiple-instance context, this will cause flushing for all DBs that are undergoing writes. In this patch, column families are registered to a shared linked list inside the write buffer manager. When flush condition is triggered, the column family with highest score from this list will be chosen and flushed. The score can be either size or age. The flush condition calculation is also changed to exclude immutable memtables. This is because RocksDB schedules flush every time an immutable memtable is generated. They will eventually be evicted from memory given the flush bandwidth doesn't bottleneck. Test plan: - Unit test cases - Trigger flush of largest/oldest memtable in another DB - Resolve flush condition by destroy CF/DB - Dynamically change flush threshold - Manual test insert, update, read-write workload, [script](https://gist.github.com/tabokie/d38d27dc3843946c7813ab7bafd0f753). Signed-off-by: tabokie <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 5abc7c2 - Browse repository at this point
Copy the full SHA 5abc7c2View commit details -
Write buffer manager supports dynamic config (tikv#332)
* add toggle Signed-off-by: tabokie <[email protected]> * protect underflow Signed-off-by: tabokie <[email protected]> * fix build Signed-off-by: tabokie <[email protected]> * remove deadline and add penalty for l0 files Signed-off-by: tabokie <[email protected]> * fix build Signed-off-by: tabokie <[email protected]> * consider compaction trigger Signed-off-by: tabokie <[email protected]> --------- Signed-off-by: tabokie <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 40a482b - Browse repository at this point
Copy the full SHA 40a482bView commit details -
hook delete dir in encrypted env (tikv#334)
* hook delete dir in encrypted env Signed-off-by: tabokie <[email protected]> * add a comment Signed-off-by: tabokie <[email protected]> --------- Signed-off-by: tabokie <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 235b78e - Browse repository at this point
Copy the full SHA 235b78eView commit details -
FlushForGetLiveFiles does not wait for write stall (tikv#336)
Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for cc43393 - Browse repository at this point
Copy the full SHA cc43393View commit details -
use expected age instead of size to avoid spurious flush (tikv#337)
Also added a new options to detect whether manual compaction is disabled. In practice we use this to avoid blocking on flushing a tablet that will be destroyed shortly after. --------- Signed-off-by: tabokie <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for a8c6b85 - Browse repository at this point
Copy the full SHA a8c6b85View commit details -
fix renaming encrypted directory and check compaction paused during c…
…heckpoint (tikv#338) * fix renaming encrypted directory Signed-off-by: tabokie <[email protected]> * fix build Signed-off-by: tabokie <[email protected]> * patch test manager Signed-off-by: tabokie <[email protected]> * fix build Signed-off-by: tabokie <[email protected]> * check compaction paused during checkpoint Signed-off-by: tabokie <[email protected]> * add comment Signed-off-by: tabokie <[email protected]> --------- Signed-off-by: tabokie <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for de03a33 - Browse repository at this point
Copy the full SHA de03a33View commit details -
fix checkpoint again (tikv#342)
Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for bc7cc71 - Browse repository at this point
Copy the full SHA bc7cc71View commit details -
filter deletion in compaction filter (tikv#344)
And delay the buffer initialization of writable file to first actual write. --------- Signed-off-by: tabokie <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 891d589 - Browse repository at this point
Copy the full SHA 891d589View commit details -
enable cf uses separete write buffer manager (tikv#343)
Signed-off-by: Spade A <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for de310e6 - Browse repository at this point
Copy the full SHA de310e6View commit details -
fix deadlock between Flush and UnregisterDB (tikv#349)
Signed-off-by: SpadeA-Tang <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for a2bd301 - Browse repository at this point
Copy the full SHA a2bd301View commit details
Commits on Oct 3, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 3dccaa1 - Browse repository at this point
Copy the full SHA 3dccaa1View commit details
Commits on Oct 5, 2024
-
Configuration menu - View commit details
-
Copy full SHA for eb7cf9a - Browse repository at this point
Copy the full SHA eb7cf9aView commit details