You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
One of the key insights of the Shard Manager paper is that many services do not need dynamic range-based sharding in order to balance their load. They can simply pre-split the keyspace into a fixed (though large) number of shards, and balance load by moving shards around. Not having to deal with key ranges can make implementation/integration of consumers much simpler, and is also a more natural fit when wrapping systems which are already partitioned statically and just want balancing.
This is unlike the Slicer model, which is key-range-based like Ranger. (Though the paper claims "Slicer will support unhashed application-defined keys and implement range sharding to preserve locality among adjacent application-defined keys"; maybe that future is here already.)
I believe we can implement this as a subset of the range-based approach as follows:
Allow controller to operate in "pre-sharded" mode, where:
All ranges are created at Keyspace creation, where we currently create the genesis range.
Range start and end keys are always the same; they have a single key. (One would assume that this key would be a hash, but that's up to the consumer.)
Ranges are never split nor joined. Only placed and moved.
Provide an alternative shard-centric Rangelet interface, which is basically the same but uses RangeKey (or ShardKey? something else?) parameters instead of RangeMeta and RangeID. This would be a helpful wrapper around the existing interface, for keyspaces where the controller is in pre-sharded mode. Same underlying RPCs etc.
The text was updated successfully, but these errors were encountered:
One of the key insights of the Shard Manager paper is that many services do not need dynamic range-based sharding in order to balance their load. They can simply pre-split the keyspace into a fixed (though large) number of shards, and balance load by moving shards around. Not having to deal with key ranges can make implementation/integration of consumers much simpler, and is also a more natural fit when wrapping systems which are already partitioned statically and just want balancing.
This is unlike the Slicer model, which is key-range-based like Ranger. (Though the paper claims "Slicer will support unhashed application-defined keys and implement range sharding to preserve locality among adjacent application-defined keys"; maybe that future is here already.)
I believe we can implement this as a subset of the range-based approach as follows:
The text was updated successfully, but these errors were encountered: