Skip to content

Commit

Permalink
Add shardingsphere-infra-url module (#30148)
Browse files Browse the repository at this point in the history
* Add shardingsphere-infra-url module

* Add shardingsphere-infra-url module
  • Loading branch information
terrymanu authored Feb 17, 2024
1 parent 0f4f997 commit 79c92f8
Show file tree
Hide file tree
Showing 31 changed files with 461 additions and 39 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ chapter = true
## 背景信息

对于 `org.apache.shardingsphere.driver.ShardingSphereDriver` 的驱动类,
通过实现 `org.apache.shardingsphere.driver.jdbc.core.driver.url.ShardingSphereURLLoader` 的 SPI,
通过实现 `org.apache.shardingsphere.infra.url.ShardingSphereURLLoader` 的 SPI,
可允许从多种来源和 File System 获取并解析为 ShardingSphere 的 YAML 配置文件。
如无特定声明,以下实现均采用 YAML 1.1 作为 YAML 的编写规范,
这并不阻止 `org.apache.shardingsphere.driver.jdbc.core.driver.url.ShardingSphereURLLoader` 的自定义实现从 XML 或 JSON 等文件手动转化为 YAML。
这并不阻止 `org.apache.shardingsphere.infra.url.ShardingSphereURLLoader` 的自定义实现从 XML 或 JSON 等文件手动转化为 YAML。

在解析并加载 YAML 文件为 ShardingSphere 的元数据后,
会再次通过[模式配置](../../../java-api/mode)的相关配置决定下一步行为。讨论两种情况,
Expand Down Expand Up @@ -48,8 +48,8 @@ chapter = true

## JDBC URL 参数

对于 `org.apache.shardingsphere.driver.jdbc.core.driver.url.ShardingSphereURLLoader` 的实现,并非所有的 JDBC URL 参数都必须被解析,
这涉及到如何实现 `org.apache.shardingsphere.driver.jdbc.core.driver.url.ShardingSphereURLLoader.getContent()`
对于 `org.apache.shardingsphere.infra.url.ShardingSphereURLLoader` 的实现,并非所有的 JDBC URL 参数都必须被解析,
这涉及到如何实现 `org.apache.shardingsphere.infra.url.ShardingSphereURLLoader.load()`

### placeholder-type

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ chapter = true
## Background Information

For the driver class of `org.apache.shardingsphere.driver.ShardingSphereDriver`,
by implementing the SPI of `org.apache.shardingsphere.driver.jdbc.core.driver.url.ShardingSphereURLLoader`,
by implementing the SPI of `org.apache.shardingsphere.infra.url.ShardingSphereURLLoader`,
allows YAML configuration files to be fetched from multiple sources and File Systems and parsed into ShardingSphere.
If there is no specific statement, the following implementations all use YAML 1.1 as the YAML writing specification.
This does not prevent custom implementations of `org.apache.shardingsphere.driver.jdbc.core.driver.url.ShardingSphereURLLoader` from being manually converted to YAML from files such as XML or JSON.
This does not prevent custom implementations of `org.apache.shardingsphere.infra.url.ShardingSphereURLLoader` from being manually converted to YAML from files such as XML or JSON.

After parsing and loading the YAML file into ShardingSphere's metadata,
the next behavior will be determined again through the relevant configuration of [Mode Configuration](../../../java-api/mode).
Expand Down Expand Up @@ -56,9 +56,9 @@ Example:

## JDBC URL parameters

For implementations of `org.apache.shardingsphere.driver.jdbc.core.driver.url.ShardingSphereURLLoader`,
For implementations of `org.apache.shardingsphere.infra.url.ShardingSphereURLLoader`,
not all JDBC URL parameters must be parsed,
this involves how to implement `org.apache.shardingsphere.driver.jdbc.core.driver.url.ShardingSphereURLLoader.getContent()`.
this involves how to implement `org.apache.shardingsphere.infra.url.ShardingSphereURLLoader.load()`.

### placeholder-type

Expand Down
1 change: 1 addition & 0 deletions infra/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
<module>data-source-pool</module>
<module>common</module>
<module>context</module>
<module>url</module>
<module>distsql-handler</module>
<module>parser</module>
<module>binder</module>
Expand Down
41 changes: 41 additions & 0 deletions infra/url/core/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-infra-url</artifactId>
<version>5.4.2-SNAPSHOT</version>
</parent>
<artifactId>shardingsphere-infra-url-core</artifactId>
<name>${project.artifactId}</name>

<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-infra-url-spi</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-infra-exception-core</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,15 @@
* limitations under the License.
*/

package org.apache.shardingsphere.driver.jdbc.core.driver.url;
package org.apache.shardingsphere.infra.url;

import com.google.common.base.Preconditions;
import com.google.common.base.Strings;
import lombok.AccessLevel;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
import org.apache.shardingsphere.driver.jdbc.exception.syntax.URLProviderNotFoundException;
import org.apache.shardingsphere.infra.exception.core.ShardingSpherePreconditions;
import org.apache.shardingsphere.infra.url.exception.URLProviderNotFoundException;

import java.util.Collections;
import java.util.HashMap;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@
* limitations under the License.
*/

package org.apache.shardingsphere.driver.jdbc.core.driver.url;
package org.apache.shardingsphere.infra.url;

import org.apache.shardingsphere.driver.jdbc.core.driver.url.arg.URLArgumentLineRender;
import org.apache.shardingsphere.driver.jdbc.core.driver.url.arg.URLArgumentPlaceholderTypeFactory;
import org.apache.shardingsphere.infra.url.arg.URLArgumentLineRender;
import org.apache.shardingsphere.infra.url.arg.URLArgumentPlaceholderTypeFactory;
import org.apache.shardingsphere.infra.spi.type.typed.TypedSPILoader;

import java.util.Arrays;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
* limitations under the License.
*/

package org.apache.shardingsphere.driver.jdbc.core.driver.url.arg;
package org.apache.shardingsphere.infra.url.arg;

import com.google.common.base.Strings;
import lombok.AccessLevel;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
* limitations under the License.
*/

package org.apache.shardingsphere.driver.jdbc.core.driver.url.arg;
package org.apache.shardingsphere.infra.url.arg;

import lombok.AccessLevel;
import lombok.NoArgsConstructor;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
* limitations under the License.
*/

package org.apache.shardingsphere.driver.jdbc.core.driver.url.arg;
package org.apache.shardingsphere.infra.url.arg;

/**
* URL argument placeholder type.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
* limitations under the License.
*/

package org.apache.shardingsphere.driver.jdbc.core.driver.url.arg;
package org.apache.shardingsphere.infra.url.arg;

import lombok.AccessLevel;
import lombok.NoArgsConstructor;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
* limitations under the License.
*/

package org.apache.shardingsphere.driver.jdbc.exception.syntax;
package org.apache.shardingsphere.infra.url.exception;

import org.apache.shardingsphere.infra.exception.core.external.sql.type.kernel.category.SyntaxSQLException;
import org.apache.shardingsphere.infra.exception.core.external.sql.sqlstate.XOpenSQLState;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
* limitations under the License.
*/

package org.apache.shardingsphere.driver.jdbc.core.driver.url;
package org.apache.shardingsphere.infra.url;

import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.params.ParameterizedTest;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
* limitations under the License.
*/

package org.apache.shardingsphere.driver.jdbc.core.driver.url.arg;
package org.apache.shardingsphere.infra.url.arg;

import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
Expand Down Expand Up @@ -52,15 +52,15 @@ static void afterAll() {

@Test
void assertReadWithNonePlaceholder() throws IOException, URISyntaxException {
byte[] actual = readContent("config/driver/foo-driver-to-be-replaced-fixture.yaml", URLArgumentPlaceholderType.NONE);
byte[] expected = readContent("config/driver/foo-driver-to-be-replaced-fixture.yaml", URLArgumentPlaceholderType.NONE);
byte[] actual = readContent("config/to-be-replaced-fixture.yaml", URLArgumentPlaceholderType.NONE);
byte[] expected = readContent("config/to-be-replaced-fixture.yaml", URLArgumentPlaceholderType.NONE);
assertThat(new String(actual), is(new String(expected)));
}

@Test
void assertReadWithSystemPropertiesPlaceholder() throws IOException, URISyntaxException {
byte[] actual = readContent("config/driver/foo-driver-to-be-replaced-fixture.yaml", URLArgumentPlaceholderType.SYSTEM_PROPS);
byte[] expected = readContent("config/driver/foo-driver-fixture.yaml", URLArgumentPlaceholderType.SYSTEM_PROPS);
byte[] actual = readContent("config/to-be-replaced-fixture.yaml", URLArgumentPlaceholderType.SYSTEM_PROPS);
byte[] expected = readContent("config/replaced-fixture.yaml", URLArgumentPlaceholderType.SYSTEM_PROPS);
assertThat(new String(actual), is(new String(expected)));
}

Expand Down
57 changes: 57 additions & 0 deletions infra/url/core/src/test/resources/config/replaced-fixture.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

databaseName: foo_driver_fixture_db

dataSources:
ds_0:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: org.h2.Driver
jdbcUrl: jdbc:h2:mem:foo_ds_0;DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=false;MODE=MySQL
username: sa
password:
ds_1:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: org.h2.Driver
jdbcUrl: jdbc:h2:mem:foo_ds_1;DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=false;MODE=MySQL
username: sa
password:

rules:
- !SHARDING
autoTables:
t_order:
actualDataSources: ds_0,ds_1
shardingStrategy:
standard:
shardingColumn: order_id
shardingAlgorithmName: auto_mod
keyGenerateStrategy:
column: user_id
keyGeneratorName: snowflake
shardingAlgorithms:
auto_mod:
type: HASH_MOD
props:
sharding-count: 2

keyGenerators:
snowflake:
type: SNOWFLAKE

props:
sql-show: true
35 changes: 35 additions & 0 deletions infra/url/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-infra</artifactId>
<version>5.4.2-SNAPSHOT</version>
</parent>
<artifactId>shardingsphere-infra-url</artifactId>
<packaging>pom</packaging>
<name>${project.artifactId}</name>

<modules>
<module>spi</module>
<module>core</module>
<module>type</module>
</modules>
</project>
36 changes: 36 additions & 0 deletions infra/url/spi/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-infra-url</artifactId>
<version>5.4.2-SNAPSHOT</version>
</parent>
<artifactId>shardingsphere-infra-url-spi</artifactId>
<name>${project.artifactId}</name>

<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-infra-spi</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
* limitations under the License.
*/

package org.apache.shardingsphere.driver.jdbc.core.driver.url;
package org.apache.shardingsphere.infra.url;

import org.apache.shardingsphere.infra.spi.annotation.SingletonSPI;
import org.apache.shardingsphere.infra.spi.type.typed.TypedSPI;
Expand Down
36 changes: 36 additions & 0 deletions infra/url/type/absolutepath/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-infra-url-type</artifactId>
<version>5.4.2-SNAPSHOT</version>
</parent>
<artifactId>shardingsphere-infra-url-absolutepath</artifactId>
<name>${project.artifactId}</name>

<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-infra-url-spi</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</project>
Loading

0 comments on commit 79c92f8

Please sign in to comment.