Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process expressions inside param values when build a query #806

Merged
merged 34 commits into from
Apr 12, 2024
Merged
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
e1f142f
Build expressions if they inside `Expression::$params`
Tigrov Feb 5, 2024
c60f3a1
Improve
Tigrov Feb 5, 2024
f15c515
Fix errors
Tigrov Feb 5, 2024
17eb937
Fix errors
Tigrov Feb 6, 2024
b325bab
Update tests
Tigrov Feb 6, 2024
54dedfb
Apply fixes from StyleCI
StyleCIBot Feb 6, 2024
02708bb
Add tests
Tigrov Feb 11, 2024
5984d34
Resolve BC, make `$queryBuilder` optional in the constructor.
Tigrov Feb 11, 2024
7803afb
Apply fixes from StyleCI
StyleCIBot Feb 11, 2024
0845c13
Merge branch 'master' into fix-update-with-expressions
Tigrov Feb 11, 2024
8f76d49
Fix psalm issue
Tigrov Feb 11, 2024
b922159
Improve
Tigrov Feb 13, 2024
a5bff4a
Add test for indexed params
Tigrov Feb 13, 2024
be292a7
Merge branch 'master' into fix-update-with-expressions
vjik Mar 13, 2024
766fc54
Merge branch 'master' into fix-update-with-expressions
Tigrov Mar 25, 2024
e9660dc
Fix using regex when string value containing a placeholder name
Tigrov Mar 28, 2024
5daa211
Add SqlParser to fix case when string value containing a placeholder …
Tigrov Mar 30, 2024
27ad79f
Apply fixes from StyleCI
StyleCIBot Mar 30, 2024
79395aa
Fix psalm issue
Tigrov Mar 30, 2024
6f2cbb5
Cover tests
Tigrov Mar 30, 2024
04696b9
Cover tests
Tigrov Mar 30, 2024
026a51d
Cover tests
Tigrov Mar 30, 2024
011e68c
Optimization
Tigrov Apr 1, 2024
6d1de52
Make `SqlParser` abstract. Add test for `Param` and fix
Tigrov Apr 1, 2024
f8f226a
Fix array merge for php8.0
Tigrov Apr 1, 2024
42c263e
Update comments and remove `null` for `$queryBuilder`
Tigrov Apr 2, 2024
4c22f58
Add lines to CHANGELOG.md and UPGRADE.md [skip ci]
Tigrov Apr 6, 2024
76bac2c
Update CHANGELOG.md
Tigrov Apr 6, 2024
2731b3a
Apply Rector changes (CI)
Tigrov Apr 6, 2024
3c3f944
Apply fixes from StyleCI
StyleCIBot Apr 6, 2024
6dd4791
Apply suggestions from code review [skip ci]
Tigrov Apr 9, 2024
03fe6dc
Merge branch 'master' into fix-update-with-expressions
Tigrov Apr 9, 2024
5885a6f
Improve UPGRADE.md
Tigrov Apr 9, 2024
fe50046
Add test with UTF-8 multibyte symbols
Tigrov Apr 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# Yii Database Change Log

## 1.3.1 under development
## 2.0.0 under development

- no changes in this release.
- Enh #806: Non-unique placeholder names inside `Expression::$params` will be replaced with unique names (@Tigrov)
- Enh #806: Build `Expression` instances inside `Expression::$params` when build a query using `QueryBuilder` (@Tigrov)

## 1.3.0 March 21, 2024

Expand Down
5 changes: 5 additions & 0 deletions UPGRADE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,10 @@ All the "Yes, it is" cool stuff, and Yii soul is still in place.

Changes summary:

## 1.x to 2.0
* `Expression::$params` can contain non-unique placeholder names, they will be replaced with unique names.
* `Expression::$params` can contain `Expression` instances, they will be built when building a query using `QueryBuilder`.

## Yii2 to 1.0
Tigrov marked this conversation as resolved.
Show resolved Hide resolved
* `Yiisoft\Db\Connection::$charset` has been removed. All supported PDO classes allow you to specify the connection
charset in the DSN.
242 changes: 242 additions & 0 deletions src/Expression/AbstractExpressionBuilder.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,242 @@
<?php

declare(strict_types=1);

namespace Yiisoft\Db\Expression;

use Yiisoft\Db\Command\Param;
use Yiisoft\Db\Connection\ConnectionInterface;
use Yiisoft\Db\QueryBuilder\QueryBuilderInterface;
use Yiisoft\Db\Syntax\AbstractSqlParser;

use function array_merge;
use function count;
use function strlen;
use function substr;
use function substr_replace;

/**
* It's used to build expressions for use in database queries.
*
* It provides a methods {@see build()} for creating various types of expressions, such as conditions, joins, and
Tigrov marked this conversation as resolved.
Show resolved Hide resolved
* ordering clauses.
*
* These expressions can be used with the query builder to build complex and customizable database queries
* {@see Expression} class.
*
* @psalm-import-type ParamsType from ConnectionInterface
*/
abstract class AbstractExpressionBuilder implements ExpressionBuilderInterface
{
public function __construct(private QueryBuilderInterface $queryBuilder)
{
}

/**
* Builds an SQL expression from the given expression object.
*
* This method is called by the query builder to build SQL expressions from {@see ExpressionInterface} objects.
*
* @param Expression $expression The expression to build.
* @param array $params The parameters to be bound to the query.
*
* @psalm-param ParamsType $params
*
* @return string SQL expression.
*/
public function build(ExpressionInterface $expression, array &$params = []): string
{
$sql = $expression->__toString();
$expressionParams = $expression->getParams();

if (empty($expressionParams)) {
return $sql;
}

if (isset($expressionParams[0])) {
$params = array_merge($params, $expressionParams);
return $sql;
}

$nonUniqueReplacements = $this->appendParams($expressionParams, $params);
$expressionReplacements = $this->buildParamExpressions($expressionParams, $params);

$replacements = $this->mergeReplacements($nonUniqueReplacements, $expressionReplacements);

if (empty($replacements)) {
return $sql;
}

return $this->replacePlaceholders($sql, $replacements);
}

/**
* Appends parameters to the list of query parameters replacing non-unique parameters with unique ones.
*
* @param array $expressionParams Parameters to be appended.
* @param array $params Parameters to be bound to the query.
*
* @psalm-param ParamsType $expressionParams
* @psalm-param ParamsType $params
*
* @return string[] Replacements for non-unique parameters.
*/
private function appendParams(array &$expressionParams, array &$params): array
{
$nonUniqueParams = [];

/** @var non-empty-string $name */
foreach ($expressionParams as $name => $value) {
$paramName = $name[0] === ':' ? substr($name, 1) : $name;
Tigrov marked this conversation as resolved.
Show resolved Hide resolved

if (!isset($params[$paramName]) && !isset($params[":$paramName"])) {
$params[$name] = $value;
continue;
}

$nonUniqueParams[$name] = $value;
}

$replacements = [];

/** @var non-empty-string $name */
foreach ($nonUniqueParams as $name => $value) {
$paramName = $name[0] === ':' ? substr($name, 1) : $name;
$uniqueName = $this->getUniqueName($paramName, $params);

$replacements[":$paramName"] = ":$uniqueName";

if ($name[0] === ':') {
$uniqueName = ":$uniqueName";
}

$params[$uniqueName] = $value;
$expressionParams[$uniqueName] = $value;
unset($expressionParams[$name]);
}

return $replacements;
}

/**
* Build expression values of parameters.
*
* @param array $expressionParams Parameters from the expression.
* @param array $params Parameters to be bound to the query.
*
* @psalm-param ParamsType $expressionParams
* @psalm-param ParamsType $params
*
* @return string[] Replacements for parameters.
*/
private function buildParamExpressions(array $expressionParams, array &$params): array
{
$replacements = [];

/** @var non-empty-string $name */
foreach ($expressionParams as $name => $value) {
if (!$value instanceof ExpressionInterface || $value instanceof Param) {
continue;
}

$placeholder = $name[0] !== ':' ? ":$name" : $name;
$replacements[$placeholder] = $this->queryBuilder->buildExpression($value, $params);

/** @psalm-var ParamsType $params */
unset($params[$name]);
}

return $replacements;
}

/**
* Merges replacements for non-unique parameters with replacements for expression parameters.
*
* @param string[] $replacements Replacements for non-unique parameters.
* @param string[] $expressionReplacements Replacements for expression parameters.
*
* @return string[] Merged replacements.
*/
private function mergeReplacements(array $replacements, array $expressionReplacements): array
{
if (empty($replacements)) {
return $expressionReplacements;
}

if (empty($expressionReplacements)) {
return $replacements;
}

/** @var non-empty-string $value */
foreach ($replacements as $name => $value) {
if (isset($expressionReplacements[$value])) {
$replacements[$name] = $expressionReplacements[$value];
unset($expressionReplacements[$value]);
}
}

return $replacements + $expressionReplacements;
}

/**
* Returns a unique name for the parameter without colon at the beginning.
*
* @param string $name Name of the parameter without colon at the beginning.
* @param array $params Parameters to be bound to the query.
*
* @psalm-param ParamsType $params
*
* @return string Unique name of the parameter with colon at the beginning.
*
* @psalm-return non-empty-string
*/
private function getUniqueName(string $name, array $params): string
{
$uniqueName = $name . '_0';

for ($i = 1; isset($params[$uniqueName]) || isset($params[":$uniqueName"]); ++$i) {
$uniqueName = $name . '_' . $i;
}

return $uniqueName;
}

/**
* Replaces placeholders with replacements in a SQL expression.
*
* @param string $sql SQL expression where the placeholder should be replaced.
* @param string[] $replacements Replacements for placeholders.
*
* @return string SQL expression with replaced placeholders.
*/
private function replacePlaceholders(string $sql, array $replacements): string
{
$parser = $this->createSqlParser($sql);
$offset = 0;

while (null !== $placeholder = $parser->getNextPlaceholder($position)) {
if (isset($replacements[$placeholder])) {
/** @var int $position */
$sql = substr_replace($sql, $replacements[$placeholder], $position + $offset, strlen($placeholder));

if (count($replacements) === 1) {
break;
}

$offset += strlen($replacements[$placeholder]) - strlen($placeholder);
unset($replacements[$placeholder]);
}
}

return $sql;
}

/**
* Creates an instance of {@see AbstractSqlParser} for the given SQL expression.
*
* @param string $sql SQL expression to be parsed.
*
* @return AbstractSqlParser SQL parser instance.
*/
abstract protected function createSqlParser(string $sql): AbstractSqlParser;
}
25 changes: 0 additions & 25 deletions src/Expression/ExpressionBuilder.php

This file was deleted.

2 changes: 0 additions & 2 deletions src/Query/Query.php
Original file line number Diff line number Diff line change
Expand Up @@ -680,8 +680,6 @@ public function withQueries(array $withQueries): static
*
* Restores the value of select to make this query reusable.
*
* @param ExpressionInterface|string $selectExpression
*
* @throws Exception
* @throws InvalidArgumentException
* @throws InvalidConfigException
Expand Down
49 changes: 13 additions & 36 deletions src/QueryBuilder/AbstractDQLQueryBuilder.php
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
use Yiisoft\Db\Exception\InvalidConfigException;
use Yiisoft\Db\Exception\NotSupportedException;
use Yiisoft\Db\Expression\Expression;
use Yiisoft\Db\Expression\ExpressionBuilder;
use Yiisoft\Db\Expression\ExpressionBuilderInterface;
use Yiisoft\Db\Expression\ExpressionInterface;
use Yiisoft\Db\QueryBuilder\Condition\HashCondition;
Expand Down Expand Up @@ -105,25 +104,7 @@ public function build(QueryInterface $query, array $params = []): array
$this->buildHaving($query->getHaving(), $params),
];
$sql = implode($this->separator, array_filter($clauses));
$sql = $this->buildOrderByAndLimit($sql, $query->getOrderBy(), $query->getLimit(), $query->getOffset());

if (!empty($query->getOrderBy())) {
/** @psalm-var array<string, ExpressionInterface|string> */
foreach ($query->getOrderBy() as $expression) {
if ($expression instanceof ExpressionInterface) {
$this->buildExpression($expression, $params);
}
}
}

if (!empty($query->getGroupBy())) {
/** @psalm-var array<string, ExpressionInterface|string> */
foreach ($query->getGroupBy() as $expression) {
if ($expression instanceof ExpressionInterface) {
$this->buildExpression($expression, $params);
}
}
}
$sql = $this->buildOrderByAndLimit($sql, $query->getOrderBy(), $query->getLimit(), $query->getOffset(), $params);

$union = $this->buildUnion($query->getUnions(), $params);

Expand Down Expand Up @@ -165,19 +146,22 @@ public function buildColumns(array|string $columns): string

public function buildCondition(array|string|ExpressionInterface|null $condition, array &$params = []): string
{
if (is_array($condition)) {
if (empty($condition)) {
return '';
if (empty($condition)) {
if ($condition === '0') {
return '0';
}

$condition = $this->createConditionFromArray($condition);
return '';
}

if ($condition instanceof ExpressionInterface) {
return $this->buildExpression($condition, $params);
if (is_array($condition)) {
$condition = $this->createConditionFromArray($condition);
} elseif (is_string($condition)) {
$condition = new Expression($condition, $params);
$params = [];
}

return $condition ?? '';
return $this->buildExpression($condition, $params);
}

public function buildExpression(ExpressionInterface $expression, array &$params = []): string
Expand Down Expand Up @@ -208,10 +192,7 @@ public function buildGroupBy(array $columns, array &$params = []): string
/** @psalm-var array<string, ExpressionInterface|string> $columns */
foreach ($columns as $i => $column) {
if ($column instanceof ExpressionInterface) {
$columns[$i] = $this->buildExpression($column);
if ($column instanceof Expression || $column instanceof QueryInterface) {
$params = array_merge($params, $column->getParams());
}
$columns[$i] = $this->buildExpression($column, $params);
} elseif (!str_contains($column, '(')) {
$columns[$i] = $this->quoter->quoteColumnName($column);
}
Expand Down Expand Up @@ -299,10 +280,7 @@ public function buildOrderBy(array $columns, array &$params = []): string
/** @psalm-var array<string, ExpressionInterface|int|string> $columns */
foreach ($columns as $name => $direction) {
if ($direction instanceof ExpressionInterface) {
$orders[] = $this->buildExpression($direction);
if ($direction instanceof Expression || $direction instanceof QueryInterface) {
$params = array_merge($params, $direction->getParams());
}
$orders[] = $this->buildExpression($direction, $params);
} else {
$orders[] = $this->quoter->quoteColumnName($name) . ($direction === SORT_DESC ? ' DESC' : '');
}
Expand Down Expand Up @@ -524,7 +502,6 @@ protected function defaultExpressionBuilders(): array
return [
Query::class => QueryExpressionBuilder::class,
Param::class => ParamBuilder::class,
Expression::class => ExpressionBuilder::class,
Condition\AbstractConjunctionCondition::class => Condition\Builder\ConjunctionConditionBuilder::class,
Condition\NotCondition::class => Condition\Builder\NotConditionBuilder::class,
Condition\AndCondition::class => Condition\Builder\ConjunctionConditionBuilder::class,
Expand Down
Loading
Loading