Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: handles default case for string array IN filter for pinot handlers #215

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

kotharironak
Copy link
Contributor

Description

The default value for string column is null in pinot - https://docs.pinot.apache.org/configuration-reference/schema
So, there are cases when query can only returns null group. These cases a follow up query with empty clause as shown in the below example:

Select dateTimeConvert(start_time_millis,'1:MILLISECONDS:EPOCH','1:MILLISECONDS:EPOCH','300:SECONDS'), span_name, COUNT(*) FROM spanEventView WHERE customer_id = '__default' 
AND start_time_millis >= 1701052800000 AND start_time_millis < 1701138900000 
AND ( user_roles IN ('null') ) ) ) 
GROUP BY dateTimeConvert(start_time_millis,'1:MILLISECONDS:EPOCH','1:MILLISECONDS:EPOCH','300:SECONDS'), user_roles 
limit 1000

It could possible that given time limit window, we have only one group null.

This PR, handles the above default scenario.

Testing

Added Unit test.

Checklist:

  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • Any dependent changes have been merged and published in downstream modules

Copy link

codecov bot commented Nov 28, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (8c2c883) 73.26% compared to head (0924bf9) 73.27%.

Additional details and impacted files
@@            Coverage Diff            @@
##               main     #215   +/-   ##
=========================================
  Coverage     73.26%   73.27%           
- Complexity     1115     1116    +1     
=========================================
  Files            96       96           
  Lines          4784     4785    +1     
  Branches        547      548    +1     
=========================================
+ Hits           3505     3506    +1     
  Misses         1054     1054           
  Partials        225      225           
Flag Coverage Δ
integration 73.27% <100.00%> (+<0.01%) ⬆️
unit 69.27% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

This comment has been minimized.

Copy link

Unit Test Results

  40 files  ±0    40 suites  ±0   7s ⏱️ -4s
320 tests +1  320 ✔️ +1  0 💤 ±0  0 ❌ ±0 

Results for commit 0924bf9. ± Comparison against base commit 8c2c883.

@@ -450,7 +450,9 @@ private String convertLiteralToString(LiteralConstant literal, Params.Builder pa
String ret = null;
switch (value.getValueType()) {
case STRING_ARRAY:
ret = buildArrayValue(value.getStringArrayList(), paramsBuilder::addStringParam);
List<String> values =
value.getStringArrayList().size() > 0 ? value.getStringArrayList() : List.of("null");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is gateway service converting null string to empty ? Shouldn't be fix that instead of expecting every converter to handle this ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'null' is special case on how it is stored in Pinot and needs to be converted to an appropriate value. Isn't it?

@@ -450,7 +450,9 @@ private String convertLiteralToString(LiteralConstant literal, Params.Builder pa
String ret = null;
switch (value.getValueType()) {
case STRING_ARRAY:
ret = buildArrayValue(value.getStringArrayList(), paramsBuilder::addStringParam);
List<String> values =
value.getStringArrayList().size() > 0 ? value.getStringArrayList() : List.of("null");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we handling special case only for string array? There are other array types as well. We don't need to handle this case for them?

@kotharironak
Copy link
Contributor Author

Closing this with respect : hypertrace/gateway-service#186

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants