Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(hogql): Speed up hogql queries #27531

Merged
merged 2 commits into from
Jan 15, 2025
Merged

Conversation

timgl
Copy link
Collaborator

@timgl timgl commented Jan 15, 2025

Problem

(rerun of #27451 which created issues with the caching, so removed that for this PR)

create_hogql_database is slow (somewhere between 1-1.5 seconds per run for team 2), and we run it multiple times for a single query.

This should make every query we run 1-2 seconds faster.

Changes

make sure we only run create_hogql_database once

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Does this work well for both Cloud and self-hosted?

How did you test this code?

@timgl timgl requested a review from mariusandra January 15, 2025 04:26
Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

Here's my review of the performance optimization changes for HogQL queries:

Optimizes HogQL query execution by preventing redundant database creation operations that were causing 1-1.5 second overhead per query.

Key changes:

  • Moves create_hogql_database call into initial context creation in query.py rather than calling it multiple times
  • Adds null check for context.database in printer.py to avoid recreating database if it already exists
  • Maintains database creation within HogQLContext to ensure proper team_id validation
  • Consolidates database initialization timing measurement under "create_hogql_database" block

The changes look well-structured and should provide significant performance gains by eliminating duplicate database creation overhead. The approach maintains proper security checks while improving efficiency.

💡 (4/5) You can add custom instructions or style guidelines for the bot here!

2 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile

@mariusandra mariusandra merged commit 076458f into master Jan 15, 2025
95 checks passed
@mariusandra mariusandra deleted the speed-up-hogql-queries branch January 15, 2025 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants