Skip to content

Commit

Permalink
Remove the test of Local transactions for ClickHouse integration modu…
Browse files Browse the repository at this point in the history 
…le in nativeTest (#34530)
  • Loading branch information
@linghengqian
linghengqian authored Jan 31, 2025
1 parent ba0d5f7 commit ec5380b
Show file tree
Hide file tree
Showing 22 changed files with 158 additions and 591 deletions.
4 changes: 2 additions & 2 deletions ...ument/content/user-manual/shardingsphere-jdbc/graalvm-native-image/_index.cn.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -389,11 +389,11 @@ Reachability Metadata 位于 `shardingsphere-infra-reachability-metadata` 子模
```bash
git clone [email protected]:apache/shardingsphere.git
cd ./shardingsphere/
./mvnw -PgenerateMetadata -DskipNativeTests -e -T 1C clean test native:metadata-copy
./mvnw -PgenerateMetadata -e -T 1C clean test native:metadata-copy
```

受 https://github.com/apache/shardingsphere/issues/33206 影响，
贡献者执行 `./mvnw -PgenerateMetadata -DskipNativeTests -T 1C -e clean test native:metadata-copy` 后，
贡献者执行 `./mvnw -PgenerateMetadata -T 1C -e clean test native:metadata-copy` 后，
`infra/reachability-metadata/src/main/resources/META-INF/native-image/org.apache.shardingsphere/generated-reachability-metadata/resource-config.json` 会生成不必要的包含绝对路径的 JSON 条目，
类似如下，

Expand Down
4 changes: 2 additions & 2 deletions ...ument/content/user-manual/shardingsphere-jdbc/graalvm-native-image/_index.en.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -405,11 +405,11 @@ contributors should place it on the classpath of the `shardingsphere-test-native
```bash
git clone [email protected]:apache/shardingsphere.git
cd ./shardingsphere/
./mvnw -PgenerateMetadata -DskipNativeTests -e -T 1C clean test native:metadata-copy
./mvnw -PgenerateMetadata -e -T 1C clean test native:metadata-copy
```

Affected by https://github.com/apache/shardingsphere/issues/33206 ,
After the contributor executes `./mvnw -PgenerateMetadata -DskipNativeTests -T 1C -e clean test native:metadata-copy`,
After the contributor executes `./mvnw -PgenerateMetadata -T 1C -e clean test native:metadata-copy`,
`infra/reachability-metadata/src/main/resources/META-INF/native-image/org.apache.shardingsphere/generated-reachability-metadata/resource-config.json` will generate unnecessary JSON entries containing absolute paths,
similar to the following.

Expand Down
178 changes: 4 additions & 174 deletions ...ontent/user-manual/shardingsphere-jdbc/optional-plugins/clickhouse/_index.cn.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -213,182 +213,12 @@ public class ExampleTest {

### 事务限制

ClickHouse 不支持 ShardingSphere 集成级别的 XA 事务，
因为 https://github.com/ClickHouse/clickhouse-java 未实现 `javax.sql.XADataSource` 的相关 Java 接口。

ClickHouse 不支持 ShardingSphere 集成级别的 Seata AT 模式事务，
因为 https://github.com/apache/incubator-seata 未实现 ClickHouse 的 SQL 方言解析。

ClickHouse 支持 ShardingSphere 集成级别的本地事务，但需要对 ClickHouse 进行额外配置，
ClickHouse 不支持 ShardingSphere 集成级别的本地事务，XA 事务或 Seata 的 AT 模式事务，
更多讨论位于 https://github.com/ClickHouse/clickhouse-docs/issues/2300 。

引入讨论，编写 Docker Compose 文件来启动 ClickHouse 和 ClickHouse Keeper。

```yaml
services:
clickhouse-keeper-01:
image: clickhouse/clickhouse-keeper:24.11.1.2557
volumes:
- ./keeper_config.xml:/etc/clickhouse-keeper/keeper_config.xml
clickhouse-server:
image: clickhouse/clickhouse-server:24.11.1.2557
depends_on:
- clickhouse-keeper-01
ports:
- "8123:8123"
volumes:
- ./transactions.xml:/etc/clickhouse-server/config.d/transactions.xml
```
`./keeper_config.xml` 的内容如下，

```xml
<clickhouse replace="true">
<listen_host>0.0.0.0</listen_host>
<keeper_server>
<tcp_port>9181</tcp_port>
<server_id>1</server_id>
<snapshot_storage_path>/var/lib/clickhouse/coordination/snapshots</snapshot_storage_path>
<raft_configuration>
<server>
<id>1</id>
<hostname>clickhouse-keeper-01</hostname>
<port>9234</port>
</server>
</raft_configuration>
</keeper_server>
</clickhouse>
```

`./transactions.xml` 的内容如下，

```xml
<clickhouse>
<allow_experimental_transactions>1</allow_experimental_transactions>
<zookeeper>
<node index="1">
<host>clickhouse-keeper-01</host>
<port>9181</port>
</node>
</zookeeper>
</clickhouse>
```

在 DBeaver Community 内，使用 `jdbc:ch://localhost:8123/default` 的 `jdbcUrl`，`default` 的`username` 连接至 ClickHouse，
`password` 留空。
执行如下 SQL，

```sql
-- noinspection SqlNoDataSourceInspectionForFile
CREATE DATABASE demo_ds_0;
CREATE DATABASE demo_ds_1;
CREATE DATABASE demo_ds_2;
```

分别使用 `jdbc:ch://localhost:8123/demo_ds_0` ，
`jdbc:ch://localhost:8123/demo_ds_1` 和 `jdbc:ch://localhost:8123/demo_ds_2` 的 `jdbcUrl` 连接至 ClickHouse 来执行如下 SQL，

```sql
-- noinspection SqlNoDataSourceInspectionForFile
create table IF NOT EXISTS t_order (
order_id Int64 NOT NULL,
order_type Int32,
user_id Int32 NOT NULL,
address_id Int64 NOT NULL,
status VARCHAR(50)
) engine = MergeTree
primary key (order_id)
order by (order_id);
TRUNCATE TABLE t_order;
```

在业务项目引入`前提条件`涉及的依赖后，在业务项目的 classpath 上编写 ShardingSphere 数据源的配置文件`demo.yaml`，

```yaml
dataSources:
ds_0:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.clickhouse.jdbc.ClickHouseDriver
jdbcUrl: jdbc:ch://localhost:8123/demo_ds_0?transactionSupport=true
username: default
password:
ds_1:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.clickhouse.jdbc.ClickHouseDriver
jdbcUrl: jdbc:ch://localhost:8123/demo_ds_1?transactionSupport=true
username: default
password:
ds_2:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.clickhouse.jdbc.ClickHouseDriver
jdbcUrl: jdbc:ch://localhost:8123/demo_ds_2?transactionSupport=true
username: default
password:
rules:
- !SHARDING
tables:
t_order:
actualDataNodes:
keyGenerateStrategy:
column: order_id
keyGeneratorName: snowflake
defaultDatabaseStrategy:
standard:
shardingColumn: user_id
shardingAlgorithmName: inline
shardingAlgorithms:
inline:
type: INLINE
props:
algorithm-expression: ds_${user_id % 2}
keyGenerators:
snowflake:
type: SNOWFLAKE
```

创建 ShardingSphere 的数据源后可正常使用本地事务，

```java
import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;
import java.sql.Connection;
import java.sql.SQLException;
@SuppressWarnings({"SqlNoDataSourceInspection", "AssertWithSideEffects"})
public class ExampleUtils {
void test() throws SQLException {
HikariConfig config = new HikariConfig();
config.setJdbcUrl("jdbc:shardingsphere:classpath:demo.yaml");
config.setDriverClassName("org.apache.shardingsphere.driver.ShardingSphereDriver");
try (HikariDataSource dataSource = new HikariDataSource(config); Connection connection = dataSource.getConnection()) {
try {
connection.setAutoCommit(false);
connection.createStatement().executeUpdate("INSERT INTO t_order (user_id, order_type, address_id, status) VALUES (2024, 0, 2024, 'INSERT_TEST')");
connection.createStatement().executeUpdate("INSERT INTO t_order_does_not_exist (test_id_does_not_exist) VALUES (2024)");
connection.commit();
} catch (final SQLException ignored) {
connection.rollback();
} finally {
connection.setAutoCommit(true);
}
try (Connection conn = dataSource.getConnection()) {
assert !conn.createStatement().executeQuery("SELECT * FROM t_order WHERE user_id = 2024").next();
}
}
}
}
```

一旦在 ShardingSphere 的配置文件为 ClickHouse JDBC Driver 的 jdbcUrl 设置 `transactionSupport=true`，
用户在执行 `alter table` 语句前应确保没有尚未完成执行的 `insert` 语句，以避免如下 Error 的发生，

```shell
java.sql.BatchUpdateException: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_6.txt' with part 'all_1_1_0' reason: 'Serialization error: part all_1_1_0 is locked by transaction 5672402456378293316'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED) (version 24.10.2.80 (official build))
at com.clickhouse.jdbc.SqlExceptionUtils.batchUpdateError(SqlExceptionUtils.java:107)
at com.clickhouse.jdbc.internal.SqlBasedPreparedStatement.executeAny(SqlBasedPreparedStatement.java:223)
at com.clickhouse.jdbc.internal.SqlBasedPreparedStatement.executeLargeUpdate(SqlBasedPreparedStatement.java:302)
at com.clickhouse.jdbc.internal.AbstractPreparedStatement.executeUpdate(AbstractPreparedStatement.java:135)
```
这与 https://clickhouse.com/docs/en/guides/developer/transactional 为 ClickHouse 提供的 `Transactions, Commit, and Rollback` 功能无关，
仅与 `com.clickhouse.jdbc.ConnectionImpl` 未实现 `java.sql.Connection#rollback()` 有关。
参考 https://github.com/ClickHouse/clickhouse-java/issues/2023 。

### 嵌入式 ClickHouse 限制

Expand Down
Loading

0 comments on commit ec5380b

Please sign in to comment.