Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: resolve relations with lateral joins #4509

Merged
merged 40 commits into from
Dec 5, 2023
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
e1f6818
wip: joins aggregations
Weakky Oct 13, 2023
5f2ca56
wip: basic impl with lateral joins + serialization support
Weakky Oct 26, 2023
0474565
fix: iterate over record fields in serializer
Weakky Oct 26, 2023
d1f0831
wip: support to-one relation and fix serializer
Weakky Oct 26, 2023
4fb012f
add basic m2m support
Weakky Oct 26, 2023
90e6617
fix m2m support and make basic ordering + pagination work
Weakky Oct 31, 2023
0ed76a7
add SelectedField::Relation and remove RelatedQuery
Weakky Nov 27, 2023
c7a753f
add RelationLoadStrategy and feature gate joins
Weakky Nov 27, 2023
fb4ada0
update json coercion to use RelationSelection + reverse in memory whe…
Weakky Nov 27, 2023
d3e9322
split get_many_records based on relation load strategy
Weakky Nov 27, 2023
3d5ca92
update join query builder to better support filters, ordering and pag…
Weakky Nov 27, 2023
84141bd
fix: relation & relevance ordering
Weakky Nov 27, 2023
e80fb7a
fix: bring query parameter exceeded error back
Weakky Nov 27, 2023
f16fb3d
fix: avoid joins with nested aggregation selection
Weakky Nov 27, 2023
4b691a6
fix aggregated order bys
Weakky Nov 27, 2023
bd4f551
add comment trace
Weakky Nov 27, 2023
6e43413
small cleanup
Weakky Nov 27, 2023
ff68957
clippy fixes
Weakky Nov 27, 2023
02b451e
temporarily exclude batching tests
Weakky Nov 28, 2023
4ce384b
rename preview feature to "joins"
Weakky Nov 28, 2023
1598984
fix generator error test
Weakky Nov 28, 2023
fd25b20
clippy fixes
Weakky Nov 28, 2023
86ee2f5
remove mssql support
Weakky Nov 28, 2023
d5c9986
fix ordering on CRDB & refactor sql builder
Weakky Nov 30, 2023
8988c8c
fix crdb tests
Weakky Nov 30, 2023
9710c41
cleanup
Weakky Nov 30, 2023
defe57a
add scalar coercion tests + handle coercion errors
Weakky Nov 30, 2023
6068b98
cleanup error message
Weakky Dec 1, 2023
bc60a31
fix driver adapter enum selection
Weakky Dec 1, 2023
a84fe43
rename preview feature from joins to relationJoins
Weakky Dec 1, 2023
410139e
fix unit test
Weakky Dec 1, 2023
e96f014
Merge remote-tracking branch 'origin/main' into integration/join-support
aqrln Dec 4, 2023
e04e5f6
exclude tests + reframe error message
Weakky Dec 4, 2023
28d5206
undo some test changes
Weakky Dec 4, 2023
c4f05a6
rename "data" to "__prisma_data__"
Weakky Dec 4, 2023
cede9be
add findUnique support
Weakky Dec 4, 2023
91d579b
Fix wasm build script
aqrln Dec 4, 2023
6e804a0
fix tests and remove dbg
Weakky Dec 4, 2023
8109c3f
hack: filter null values from joined JSON arrays
Weakky Dec 4, 2023
d6d5262
Merge branch 'main' into integration/join-support
Weakky Dec 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .envrc
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ export SIMPLE_TEST_MODE="yes" # Reduces the amount of generated `relation_link_t
### QE specific logging vars ###
export QE_LOG_LEVEL=debug # Set it to "trace" to enable query-graph debugging logs
# export PRISMA_RENDER_DOT_FILE=1 # Uncomment to enable rendering a dot file of the Query Graph from an executed query.
# export FMT_SQL=1 # Uncomment it to enable logging formatted SQL queries
export FMT_SQL=1 # Uncomment it to enable logging formatted SQL queries

### Uncomment to run driver adapters tests. See query-engine-driver-adapters.yml workflow for how tests run in CI.
# export EXTERNAL_TEST_EXECUTOR="napi"
Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions libs/prisma-value/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,14 @@ impl PrismaValue {
_ => None,
}
}

pub fn as_json(&self) -> Option<&String> {
if let Self::Json(v) = self {
Some(v)
} else {
None
}
}
}

impl fmt::Display for PrismaValue {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,8 @@ const CAPABILITIES: ConnectorCapabilities = enumflags2::make_bitflags!(Connector
FilteredInlineChildNestedToOneDisconnect |
InsertReturning |
UpdateReturning |
RowIn
RowIn |
LateralJoin
});

const SCALAR_TYPE_DEFAULTS: &[(ScalarType, CockroachType)] = &[
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,8 @@ const CAPABILITIES: ConnectorCapabilities = enumflags2::make_bitflags!(Connector
NativeUpsert |
InsertReturning |
UpdateReturning |
RowIn
RowIn |
LateralJoin
});

pub struct PostgresDatamodelConnector;
Expand Down
2 changes: 2 additions & 0 deletions psl/psl-core/src/common/preview_features.rs
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ features!(
TransactionApi,
UncheckedScalarInputs,
Views,
RelationJoins
);

/// Generator preview features
Expand All @@ -90,6 +91,7 @@ pub const ALL_PREVIEW_FEATURES: FeatureMap = FeatureMap {
| PostgresqlExtensions
| Tracing
| Views
| RelationJoins
}),
deprecated: enumflags2::make_bitflags!(PreviewFeature::{
AtomicNumberOperations
Expand Down
1 change: 1 addition & 0 deletions psl/psl-core/src/datamodel_connector/capabilities.rs
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ capabilities!(
InsertReturning,
UpdateReturning,
RowIn, // Connector supports (a, b) IN (c, d) expression.
LateralJoin,
);

/// Contains all capabilities that the connector is able to serve.
Expand Down
2 changes: 1 addition & 1 deletion psl/psl/tests/config/generators.rs
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,7 @@ fn nice_error_for_unknown_generator_preview_feature() {
.unwrap_err();

let expectation = expect![[r#"
error: The preview feature "foo" is not known. Expected one of: deno, driverAdapters, fullTextIndex, fullTextSearch, metrics, multiSchema, postgresqlExtensions, tracing, views
error: The preview feature "foo" is not known. Expected one of: deno, driverAdapters, fullTextIndex, fullTextSearch, metrics, multiSchema, postgresqlExtensions, tracing, views, relationJoins [0m
--> schema.prisma:3
 | 
 2 |  provider = "prisma-client-js"
Expand Down
10 changes: 9 additions & 1 deletion quaint/src/ast/function.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ mod average;
mod coalesce;
mod concat;
mod count;
mod json_array_agg;
mod json_build_obj;
#[cfg(any(feature = "postgresql", feature = "mysql"))]
mod json_extract;
#[cfg(any(feature = "postgresql", feature = "mysql"))]
Expand All @@ -28,6 +30,8 @@ pub use average::*;
pub use coalesce::*;
pub use concat::*;
pub use count::*;
pub use json_array_agg::*;
pub use json_build_obj::*;
#[cfg(any(feature = "postgresql", feature = "mysql"))]
pub use json_extract::*;
#[cfg(any(feature = "postgresql", feature = "mysql"))]
Expand Down Expand Up @@ -98,6 +102,8 @@ pub(crate) enum FunctionType<'a> {
JsonExtractFirstArrayElem(JsonExtractFirstArrayElem<'a>),
#[cfg(any(feature = "postgresql", feature = "mysql"))]
JsonUnquote(JsonUnquote<'a>),
JsonArrayAgg(JsonArrayAgg<'a>),
JsonBuildObject(JsonBuildObject<'a>),
#[cfg(any(feature = "postgresql", feature = "mysql"))]
TextSearch(TextSearch<'a>),
#[cfg(any(feature = "postgresql", feature = "mysql"))]
Expand Down Expand Up @@ -154,5 +160,7 @@ function!(
Minimum,
Maximum,
Coalesce,
Concat
Concat,
JsonArrayAgg,
JsonBuildObject
);
18 changes: 18 additions & 0 deletions quaint/src/ast/function/json_array_agg.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
use crate::prelude::*;

#[derive(Debug, Clone, PartialEq)]
pub struct JsonArrayAgg<'a> {
pub(crate) expr: Box<Expression<'a>>,
}

/// Builds a JSON array out of a list of values.
pub fn json_array_agg<'a, E>(expr: E) -> Function<'a>
where
E: Into<Expression<'a>>,
{
let fun = JsonArrayAgg {
expr: Box::new(expr.into()),
};

fun.into()
}
15 changes: 15 additions & 0 deletions quaint/src/ast/function/json_build_obj.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
use std::borrow::Cow;

use crate::prelude::*;

#[derive(Debug, Clone, PartialEq)]
pub struct JsonBuildObject<'a> {
pub(crate) exprs: Vec<(Cow<'a, str>, Expression<'a>)>,
}

/// Builds a JSON object out of a list of key-value pairs.
pub fn json_build_object<'a>(exprs: Vec<(Cow<'a, str>, Expression<'a>)>) -> Function<'a> {
let fun = JsonBuildObject { exprs };

fun.into()
}
9 changes: 9 additions & 0 deletions quaint/src/ast/join.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ use crate::ast::{ConditionTree, Table};
pub struct JoinData<'a> {
pub(crate) table: Table<'a>,
pub(crate) conditions: ConditionTree<'a>,
pub(crate) lateral: bool,
}

impl<'a> JoinData<'a> {
Expand All @@ -13,8 +14,14 @@ impl<'a> JoinData<'a> {
Self {
table: table.into(),
conditions: ConditionTree::NoCondition,
lateral: false,
}
}

pub fn lateral(mut self) -> Self {
self.lateral = true;
self
}
}

impl<'a, T> From<T> for JoinData<'a>
Expand Down Expand Up @@ -73,6 +80,7 @@ where
JoinData {
table: self.into(),
conditions: conditions.into(),
lateral: false,
}
}
}
Expand All @@ -90,6 +98,7 @@ impl<'a> Joinable<'a> for JoinData<'a> {
JoinData {
table: self.table,
conditions,
lateral: false,
}
}
}
9 changes: 9 additions & 0 deletions quaint/src/ast/select.rs
Original file line number Diff line number Diff line change
Expand Up @@ -391,6 +391,15 @@ impl<'a> Select<'a> {
self
}

pub fn left_join_lateral<J>(self, join: J) -> Self
where
J: Into<JoinData<'a>>,
{
let join_data: JoinData = join.into();

self.left_join(join_data.lateral())
}

/// Adds `RIGHT JOIN` clause to the query.
///
/// ```rust
Expand Down
9 changes: 9 additions & 0 deletions quaint/src/ast/table.rs
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,15 @@ impl<'a> Table<'a> {
self
}

pub fn left_join_lateral<J>(self, join: J) -> Self
where
J: Into<JoinData<'a>>,
{
let join_data: JoinData = join.into();

self.left_join(join_data.lateral())
}

/// Adds an `INNER JOIN` clause to the query, specifically for that table.
/// Useful to positionally add a JOIN clause in case you are selecting from multiple tables.
///
Expand Down
41 changes: 41 additions & 0 deletions quaint/src/visitor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -188,18 +188,38 @@ pub trait Visitor<'a> {
match j {
Join::Inner(data) => {
self.write(" INNER JOIN ")?;

if data.lateral {
self.write("LATERAL ")?;
}

self.visit_join_data(data)?;
}
Join::Left(data) => {
self.write(" LEFT JOIN ")?;

if data.lateral {
self.write("LATERAL ")?;
}

self.visit_join_data(data)?;
}
Join::Right(data) => {
self.write(" RIGHT JOIN ")?;

if data.lateral {
self.write("LATERAL ")?;
}

self.visit_join_data(data)?;
}
Join::Full(data) => {
self.write(" FULL JOIN ")?;

if data.lateral {
self.write("LATERAL ")?;
}

self.visit_join_data(data)?;
}
}
Expand Down Expand Up @@ -1106,6 +1126,27 @@ pub trait Visitor<'a> {
FunctionType::Concat(concat) => {
self.visit_concat(concat)?;
}
FunctionType::JsonArrayAgg(array_agg) => {
self.write("JSON_AGG")?;
self.surround_with("(", ")", |s| s.visit_expression(*array_agg.expr))?;
}
FunctionType::JsonBuildObject(build_obj) => {
let len = build_obj.exprs.len();

self.write("JSON_BUILD_OBJECT")?;
self.surround_with("(", ")", |s| {
for (i, (name, expr)) in build_obj.exprs.into_iter().enumerate() {
s.visit_raw_value(Value::text(name))?;
s.write(", ")?;
s.visit_expression(expr)?;
if i < (len - 1) {
s.write(", ")?;
}
}

Ok(())
})?;
}
};

if let Some(alias) = fun.alias {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -434,7 +434,7 @@ mod multi_schema {
insta::assert_snapshot!(
run_query!(&runner, r#"
query {
findManyCategoriesOnPosts(where: {postId: {gt: 0}}) {
findManyCategoriesOnPosts(orderBy: [{ postId: asc }, { categoryId: asc }], where: {postId: {gt: 0}}) {
category {
name
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@ mod not_in_batching {
assert_error!(
runner,
"query { findManyTestModel(where: { id: { notIn: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] } }) { id }}",
2029,
"Parameter limits for this database provider require this query to be split into multiple queries, but the negation filters used prevent the query from being split. Please reduce the used values in the query."
2029 // QueryParameterLimitExceeded
);

Ok(())
Expand All @@ -30,8 +29,7 @@ mod not_in_batching_cockroachdb {
assert_error!(
runner,
"query { findManyTestModel(where: { id: { notIn: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] } }) { id }}",
2029,
"Parameter limits for this database provider require this query to be split into multiple queries, but the negation filters used prevent the query from being split. Please reduce the used values in the query."
2029 // QueryParameterLimitExceeded
);

Ok(())
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,9 @@ mod isb {
}

// "batching of IN queries" should "work when having more than the specified amount of items"
#[connector_test]
// TODO(joins): Excluded because we have no support for batched queries with joins. In practice, it should happen under much less circumstances
// TODO(joins): than with the query-based strategy, because we don't issue `WHERE IN (parent_ids)` queries anymore to resolve relations.
#[connector_test(exclude_features("relationJoins"))]
async fn in_more_items(runner: Runner) -> TestResult<()> {
create_test_data(&runner).await?;

Expand All @@ -51,7 +53,9 @@ mod isb {
}

// "ascending ordering of batched IN queries" should "work when having more than the specified amount of items"
#[connector_test]
// TODO(joins): Excluded because we have no support for batched queries with joins. In practice, it should happen under much less circumstances
// TODO(joins): than with the query-based strategy, because we don't issue `WHERE IN (parent_ids)` queries anymore to resolve relations.
#[connector_test(exclude_features("relationJoins"))]
async fn asc_in_ordering(runner: Runner) -> TestResult<()> {
create_test_data(&runner).await?;

Expand All @@ -67,7 +71,9 @@ mod isb {
}

// "ascending ordering of batched IN queries" should "work when having more than the specified amount of items"
#[connector_test]
// TODO(joins): Excluded because we have no support for batched queries with joins. In practice, it should happen under much less circumstances
// TODO(joins): than with the query-based strategy, because we don't issue `WHERE IN (parent_ids)` queries anymore to resolve relations.
#[connector_test(exclude_features("relationJoins"))]
async fn desc_in_ordering(runner: Runner) -> TestResult<()> {
create_test_data(&runner).await?;

Expand All @@ -91,8 +97,7 @@ mod isb {
r#"query {
findManyA(where: {id: { in: [5,4,3,2,1,1,1,2,3,4,5,6,7,6,5,4,3,2,1,2,3,4,5,6] }}, orderBy: { b: { as: { _count: asc } } }) { id }
}"#,
2029,
"Your query cannot be split into multiple queries because of the order by aggregation or relevance."
2029 // QueryParameterLimitExceeded
);

Ok(())
Expand All @@ -107,8 +112,7 @@ mod isb {
r#"query {
findManyA(where: {id: { in: [5,4,3,2,1,1,1,2,3,4,5,6,7,6,5,4,3,2,1,2,3,4,5,6] }}, orderBy: { _relevance: { fields: text, search: "something", sort: asc } }) { id }
}"#,
2029,
"Your query cannot be split into multiple queries because of the order by aggregation or relevance."
2029 // QueryParameterLimitExceeded
);

Ok(())
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@ mod float;
mod int;
mod json;
mod string;
mod through_relation;
Loading