Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor: Move some classes from sql to processing & server for reusability #17542

Merged
merged 12 commits into from
Dec 6, 2024

Conversation

abhishekrb19
Copy link
Contributor

This PR contains non-functional / refactoring changes of the following classes in the sql module:

  1. Move ExplainPlan and ExplainAttributes fromsql/src/main/java/org/apache/druid/sql/http to processing/src/main/java/org/apache/druid/query/explain
  2. Move sql/src/main/java/org/apache/druid/sql/SqlTaskStatus.java -> processing/src/main/java/org/apache/druid/query/http/SqlTaskStatus.java
  3. Add a new class processing/src/main/java/org/apache/druid/query/http/ClientSqlQuery.java that is effectively a thin POJO version of SqlQuery in the sql module but without any of the Calcite functionality and business logic.
  4. Move BrokerClient, BrokerClientImpl and Broker classes from sql/src/main/java/org/apache/druid/sql/client to server/src/main/java/org/apache/druid/client/broker.
  5. Remove BrokerServiceModule that provided the BrokerClient. The functionality is now contained in ServiceClientModule in the server package itself which provides all the clients as well.

This is done so that we can reuse the said classes in #17353 without brining in Calcite and other dependencies to the Overlord.

This PR has:

  • been self-reviewed.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • been tested in a test Druid cluster.

@github-actions github-actions bot added Area - Batch Ingestion Area - Querying Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Dec 6, 2024
@abhishekrb19 abhishekrb19 changed the title Refactor: Move some classes in sql to processing & server for reusability Refactor: Move some classes from sql to processing & server for reusability Dec 6, 2024
Copy link
Contributor

@kfaraz kfaraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, minor suggestion for tests.

public class ClientSqlQueryTest
{
@Test
public void testSerde() throws JsonProcessingException
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a test in SqlQueryTest to test inter-conversion between SqlQuery and ClientSqlQuery.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Also, fixed a related bug with parameters caught in the unit test.

@abhishekrb19 abhishekrb19 force-pushed the refactor_broker_classes branch from ea7a58d to 0616081 Compare December 6, 2024 06:18
Copy link
Contributor

@kfaraz kfaraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@abhishekrb19
Copy link
Contributor Author

Thanks for the review, @kfaraz. Merging the PR as the failing ITs are unrelated (#17543 should address the flakiness to an extent).

@abhishekrb19 abhishekrb19 merged commit 3a2220c into apache:master Dec 6, 2024
79 of 82 checks passed
@abhishekrb19 abhishekrb19 deleted the refactor_broker_classes branch December 6, 2024 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area - Batch Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 Area - Querying
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants