test: ensure chunk count is at least half of target #1282
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added assertion in TestOperatorAssignments to verify that when using target number of chunks, the actual number of chunks is not less than half of the target value. This helps ensure the chunk size calculation produces reasonable results within expected bounds.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
The FuzzOperatorAssignments test had a TODO comment indicating that we need to verify the actual number of chunks is not less than half of the target value. This check is important to ensure the CalculateChunkLength algorithm produces reasonable chunk distributions. Without this validation, we might miss cases where the chunk count becomes too low relative to the target, which could lead to inefficient data distribution among operators.
Checks
[x] I've made sure the tests are passing. Note that there might be a few flaky tests, in that case, please comment that they are not relevant.
[x] I've checked the new test coverage and the coverage percentage didn't drop.
Testing Strategy
[x] Unit tests
[ ] Integration tests
[ ] This PR is not tested :(