Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle unsigned long in flush operations for star tree #16646

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Shailesh-Kumar-Singh
Copy link

Description

This change handles the comparison and sorting of unsigned long values during flush operations. Since unsigned long values are stored as Java Long (and values greater than Long.MAX_VALUE turn negative),Long.compareUnsigned is used to ensure correct sorting and comparison of unsigned long dimensions. This ensures that unsigned long values are properly handled and sorted during flush operations

Related Issues

Issue - Handle unsigned long during flush and merge operations

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

❌ Gradle check result for 80db308: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@@ -36,14 +36,15 @@ public class DimensionFactory {
public static Dimension parseAndCreateDimension(
String name,
String type,
Boolean isUnsignedLong,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not extensible, lets go with a separate dimension for unsigned long?

That should solve isUnsignedLong problem easily

return Long.compare(dimensions[i], dimension);
if (dimension instanceof NumericDimension) {
NumericDimension numericDimension = (NumericDimension) dimension;
if (numericDimension.isUnsignedLong()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets avoid if else checks, lets have a default implementation of comparator in the dimension base class, and child dimensions [such as unsignedLongDimension] can overload if needed.

*
* @return true if the dimension is unsigned long, false otherwise.
*/
default Boolean isUnsignedLong() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah lets not do this , mapperBuilderProperties is implemented by every builder, we can't introduce another new method specific for star tree.

@@ -24,46 +26,87 @@
* Tests for {@link StarTreeDocumentsSorter}.
*/
public class StarTreeDocumentsSorterTests extends OpenSearchTestCase {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets add tests in flush/merge UTs and also in integ tests. Also how does unsigned long work as a metric ? Can we test that as well ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Can we also please test the behavior of metrics for unsigned long ? And tests around the same - the results should be same as normal aggregation query on unsigned long fields.

  2. We need to add tests to mapper as well. You can refer to Changes to support IP field in star tree indexing #16641

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants