Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(search): update README to add some references for KQIR #2581

Merged
merged 1 commit into from
Oct 5, 2024
Merged

chore(search): update README to add some references for KQIR #2581

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions src/search/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,27 +1,34 @@
## KQIR: Kvrocks Query Intermediate Representation
## KQIR (a.k.a. Kvrocks Search): Kvrocks Query Intermediate Representation

Here, *KQIR* refers to both
- the multiple-level *query intermediate representation* for Apache Kvrocks, and
- the *architecture and toolset* for the query optimization and execution.

Note that when interacting with users, we also refer to KQIR as *Kvrocks Search* to prevent overwhelming with excessive technical information.

### Architecture

![Architecture of KQIR](../../assets/KQIR.png)

### Components

- User Interface: both SQL and Redis Query syntax is supported to be the frontend language of KQIR
- User Interface: both SQL and RediSearch Query syntax is supported to be the frontend language of KQIR
- SQL Parser: A parser that accepts an extended subset of MySQL syntax
- Redis Query Parser: A parser that accepts [Redis query syntax](https://redis.io/docs/latest/develop/interact/search-and-query/advanced-concepts/query_syntax/) (only DIALECT 2 or greater is planned to be supported)
- Redis Query Parser: A parser that accepts [RediSearch query syntax](https://redis.io/docs/latest/develop/interact/search-and-query/advanced-concepts/query_syntax/) (only DIALECT 2 or greater is planned to be supported)
- KQIR: a multiple level query intermediate representation, currently including two levels (syntactical IR and planning IR)
- Syntactical IR: A high level IR that syntactically represents the query language
- Planning IR: A low level IR that represents plan operators for query execution
- KQIR passes: analysis and transformation procedures on KQIR
- Semantic Checker: to check if there is any semantic errors in the IR
- Expression Passes: passes for query expressions, especially for logical expressions
- Numeric Passes: passes for numeric & arithmetic properties
- Plan Passes: passes on the plan operators
- Expression Passes: passes for query expressions, especially for logical/boolean expressions
- Numeric Passes: passes for numeric & arithmetic expressions utilizing math properties
- Planning Passes: passes on the plan operators for scheduling a better execution plan
- Pass Manager: to manage the pass execution sequence and order
- Cost Model: to analyze the cost for the current plan, used by some plan passes
- Plan Executor: a component for query execution via iterator model
- Cost Model: to analyze the cost for the current plan, used by some planning passes
- Plan Executor: a component for query execution via the iterator model
- Indexer: to perform the indexing for various types of fields during data changes

### References

- [KQIR: a query engine for Apache Kvrocks that supports both SQL and RediSearch queries](https://kvrocks.apache.org/blog/kqir-query-engine)
- [Index encoding format for Kvrocks Search](https://kvrocks.apache.org/community/kvrocks-search-index-encoding)