Fix JoinDataSource to not ignore DataSource layers silently #17726
Conversation
- Rename createReport() to createModesReport() to reflect the updated functionality. - Update getAnnotation() to getModesAnnotation() to align with the new annotation type. - Add a new test method createQuidemReasonReport() to handle DecoupledTestConfig annotations. - Update getQuidemReasonAnnotations() to handle DecoupledTestConfig annotations. - Update createReport() to handle both NotYetSupported and DecoupledTestConfig annotations. This commit introduces changes to the test methods and annotations to improve clarity and consistency in the test suite.
This reverts commit 173a057.
This reverts commit 293a483.
(cherry picked from commit e875e92)
This reverts commit c236af7.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| @@ -107,13 +109,23 @@ | |||
| return getQuerySegmentSpecForLookUp(this).lookup(this, walker); | |||
| } | |||
|
|
|||
| public DataSourceAnalysis getDataSourceAnalysis() | |||
Check notice
Code scanning / CodeQL
Missing Override annotation
| throw new ISE("Unable to apply specific segments to query with dataSource[%s]", query.getDataSource()); | ||
| if (!analysis.getEffectiveQuerySegmentSpec().equals(new MultipleSpecificSegmentSpec(descriptors))) { | ||
| // If you see the error message below, it's a bug in either this function or in DataSourceAnalysis. | ||
| throw new ISE("Unable to apply specific segments to query with dataSource[%s]", query.getDataSource()); |
There was a problem hiding this comment.
Could change this to a DruidException.defensive
| @@ -161,21 +161,18 @@ public static <T> Query<T> withSpecificSegments(final Query<T> query, final List | |||
| retVal = query.withDataSource(new QueryDataSource(withSpecificSegments(subQuery, descriptors))); | |||
| } else { | |||
| retVal = query.withQuerySegmentSpec(new MultipleSpecificSegmentSpec(descriptors)); | |||
| } | |||
There was a problem hiding this comment.
Was the else block being closed here earlier a bug?
There was a problem hiding this comment.
Earlier it was able to traverse even QueryDataSource-es ; so it was available more widely.
The change here is that the getEffectiveQuery means The applicable {@link QuerySegmentSpec} for this vertex. ; so it only considers queries belonging to the current vertex.
The other interpretation also has a valuable meaning ; but I've not seen any need for it.
| ) | ||
| ); | ||
| final Function<SegmentReference, SegmentReference> baseMapFn = joinAnalysis.getBaseDataSource().createSegmentMapFunction(query); | ||
| return baseSegment -> newHashJoinSegment( |
There was a problem hiding this comment.
nit: maybe this should be createHashJoinSegment, so that it doesn't look like a constructor call?
JoinDataSourceusesDataSourceAnalysisas a model - but during construction it ignores some other typedDataSource-screateSegmentMapFunctionon those sourcesDataSourceAnalysisalso acts as a vertex-boundary like object; which makes this a bit more complicated.This change is limited to fix the bug: I think in a followup will be needed to create a separate object which could be used inside the join - and a separate one to work with the boundary - the
DataSource#getAnalysiswas being used at a lot of places so the bugfix was separated for now.