-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
handle unsigned long in flush operations for star tree #16646
base: main
Are you sure you want to change the base?
handle unsigned long in flush operations for star tree #16646
Conversation
❌ Gradle check result for 80db308: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
@@ -36,14 +36,15 @@ public class DimensionFactory { | |||
public static Dimension parseAndCreateDimension( | |||
String name, | |||
String type, | |||
Boolean isUnsignedLong, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not extensible, lets go with a separate dimension for unsigned long?
That should solve isUnsignedLong
problem easily
return Long.compare(dimensions[i], dimension); | ||
if (dimension instanceof NumericDimension) { | ||
NumericDimension numericDimension = (NumericDimension) dimension; | ||
if (numericDimension.isUnsignedLong()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lets avoid if else checks, lets have a default implementation of comparator in the dimension base class, and child dimensions [such as unsignedLongDimension] can overload if needed.
* | ||
* @return true if the dimension is unsigned long, false otherwise. | ||
*/ | ||
default Boolean isUnsignedLong() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah lets not do this , mapperBuilderProperties is implemented by every builder, we can't introduce another new method specific for star tree.
@@ -24,46 +26,87 @@ | |||
* Tests for {@link StarTreeDocumentsSorter}. | |||
*/ | |||
public class StarTreeDocumentsSorterTests extends OpenSearchTestCase { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lets add tests in flush/merge UTs and also in integ tests. Also how does unsigned long work as a metric ? Can we test that as well ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-
Can we also please test the behavior of metrics for unsigned long ? And tests around the same - the results should be same as normal aggregation query on unsigned long fields.
-
We need to add tests to mapper as well. You can refer to Changes to support IP field in star tree indexing #16641
Description
This change handles the comparison and sorting of unsigned long values during flush operations. Since unsigned long values are stored as Java Long (and values greater than Long.MAX_VALUE turn negative),
Long.compareUnsigned
is used to ensure correct sorting and comparison of unsigned long dimensions. This ensures that unsigned long values are properly handled and sorted during flush operationsRelated Issues
Issue - Handle unsigned long during flush and merge operations
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.