Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[question] reuse subtree for follow-up query of same user #14937

Open
hpvd opened this issue Jan 29, 2025 · 1 comment
Open

[question] reuse subtree for follow-up query of same user #14937

hpvd opened this issue Jan 29, 2025 · 1 comment

Comments

@hpvd
Copy link

hpvd commented Jan 29, 2025

Reuse common expressions in a query #14196 introduces reuse of subtrees within same query.

When using Pinot for user facing analytics where

  • users slice and dice their data and
  • often a user starts relatively broad (e.g. dashboard standard views) and narrow the results further with multiple steps:
Example:
Simple analyse of website usage
(e.g. number of visitors
or time on page
or number of subpages visited...)
1. start by week,
2. narrow to days,
3. narrow origin country
4. narrow to mobile
  • with freshness requirements of minutes (for this part of pinot usage..)

=> Questions:
A) what are the current possibilities to reuse the result subtree of 1. in 2., of 2. in 3. ... instead of always do an possibly expensive full query?
B) or would this kind of load reduction not look like to be worth it (opinion)?

inspired by
@gortiz in Reuse common expressions in a query #14196

and
Splitting Queries Across Data Warehouses
https://cotera.co/blog/how-era-brings-last-mile-analytics-to-any-data-warehouse-via-duckdb

@hpvd hpvd changed the title [question] reuse subtree for follow up query [question] reuse subtree for follow up query (same user) Jan 29, 2025
@hpvd hpvd changed the title [question] reuse subtree for follow up query (same user) [question] reuse subtree for follow up query of same user Jan 29, 2025
@hpvd hpvd changed the title [question] reuse subtree for follow up query of same user [question] reuse subtree for follow-up query of same user Jan 29, 2025
@hpvd
Copy link
Author

hpvd commented Jan 29, 2025

C) Is this /can this be maybe some how related to pagination support as described here by @siddharthteotia
#5246 (comment)

where the first query may fetch all results and "deliver" only the first part, but the follow-up queries p2, p3.. reuse the full results and deliver the requested part?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant