Skip to content

Commit

Permalink
Removed CSP limitation
Browse files Browse the repository at this point in the history
  • Loading branch information
kapustor authored Oct 1, 2024
1 parent 8172eac commit f1502df
Showing 1 changed file with 8 additions and 10 deletions.
18 changes: 8 additions & 10 deletions docs/en/cloud/reference/compute-compute-separation.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,9 +107,7 @@ Once compute-compute is enabled for a service, the `clusterAllReplicas()` functi

Because this compute-compute separation is currently in private preview, there are some limitations to using this feature. Most of these limitations will be removed once the feature is released to GA (general availability):

1. **Only AWS and GCP services are supported (limitation will be removed in GA).** Azure services will support compute-compute separation in GA. If you need compute-compute separation for Azure services, please contact support.

2. **Services are created manually by the ClickHouse team (limitation will be removed in GA).** Once you are ready to create a service that will access the same data as an existing service, please notify the support team. The ClickHouse team will create such a service and you will see the new service in the cloud console.
1. **Services are created manually by the ClickHouse team (limitation will be removed in GA).** Once you are ready to create a service that will access the same data as an existing service, please notify the support team. The ClickHouse team will create such a service and you will see the new service in the cloud console.

Once the feature is released in GA, you will be able to create such services with using the cloud console. During the private preview, cloud console support will be limited, but secondary services will be marked as sub-services:

Expand All @@ -124,26 +122,26 @@ Once the feature is released in GA, you will be able to create such services wit
<br />


3. **The original service should be recently created or migrated.** Unfortunately, not all existing services can share their storage with other services. During the last year, we released a few features that the service will need to support (like the Shared Merge Tree engine), so unupdated services will mostly not be able to share their data with other services. This does not depend on ClickHouse version.
2. **The original service should be recently created or migrated.** Unfortunately, not all existing services can share their storage with other services. During the last year, we released a few features that the service will need to support (like the Shared Merge Tree engine), so unupdated services will mostly not be able to share their data with other services. This does not depend on ClickHouse version.

The good news is that we can migrate the old service to the new engine so it can support creating additional services. Reach out to support and we will let you know if your desired service needs to be migrated.

4. **The original service should always be up and should not be idled (limitation will be removed some time after GA).** During the private preview and some time after GA, the first service (usually the existing service that you want to extend by adding other services) needs to be always up and should have the idling setting disabled. If the first service becomes idled or stopped, there is a risk that some changes from the second service will not be represented after the service wakes up (especially `CREATE`/`DROP` databases). By participating in the private preview program, you agree not to idle or stop the original service during the private preview.
3. **The original service should always be up and should not be idled (limitation will be removed some time after GA).** During the private preview and some time after GA, the first service (usually the existing service that you want to extend by adding other services) needs to be always up and should have the idling setting disabled. If the first service becomes idled or stopped, there is a risk that some changes from the second service will not be represented after the service wakes up (especially `CREATE`/`DROP` databases). By participating in the private preview program, you agree not to idle or stop the original service during the private preview.

5. **Sometimes workloads cannot be isolated.** Though the goal is to give you an option to isolate database workloads from each other, there can be corner cases where one workload in one service will affect another service sharing the same data. These are quite rare situations that are mostly connected to OLTP-like workloads.
4. **Sometimes workloads cannot be isolated.** Though the goal is to give you an option to isolate database workloads from each other, there can be corner cases where one workload in one service will affect another service sharing the same data. These are quite rare situations that are mostly connected to OLTP-like workloads.

6. **All read-write services are doing background merge operations.** When inserting data to ClickHouse, the database at first inserts the data to some staging partitions, and then performs merges in the background. These merges can consume memory and CPU resources. When two read-write services share the same storage, they both are performing background operations. That means that there can be a situation where there is an `INSERT` query in Service 1, but the merge operation is completed by Service 2. Note that read-only services will not complete background merges.
5. **All read-write services are doing background merge operations.** When inserting data to ClickHouse, the database at first inserts the data to some staging partitions, and then performs merges in the background. These merges can consume memory and CPU resources. When two read-write services share the same storage, they both are performing background operations. That means that there can be a situation where there is an `INSERT` query in Service 1, but the merge operation is completed by Service 2. Note that read-only services will not complete background merges.

7. **Inserts in one read-write service can prevent another read-write service from idling if idling is enabled.** Because of the previous point, a second service perform background merge operations for the first service. These background operations can prevent the second service from going to sleep when idling. Once the background operations are finished, the service will be idled. Read-only services are not affected and will be idled without delay.
6. **Inserts in one read-write service can prevent another read-write service from idling if idling is enabled.** Because of the previous point, a second service perform background merge operations for the first service. These background operations can prevent the second service from going to sleep when idling. Once the background operations are finished, the service will be idled. Read-only services are not affected and will be idled without delay.

8. **CREATE/RENAME/DROP DATABASE queries could be blocked by idled/stopped services by default (limitation will be removed in GA).** These queries can hang. To bypass this, you can run database management queries with `settings distributed_ddl_task_timeout=0` at the session or per query level. For example:
7. **CREATE/RENAME/DROP DATABASE queries could be blocked by idled/stopped services by default (limitation will be removed in GA).** These queries can hang. To bypass this, you can run database management queries with `settings distributed_ddl_task_timeout=0` at the session or per query level. For example:

```sql
create database db_test_ddl_single_query_setting
settings distributed_ddl_task_timeout=0
```

9. **When performing a DELETE or UPDATE query, all services should be running (not stopped and not idled) (limitation will be removed before GA).** This is because currently [mutations](https://clickhouse.com/docs/en/guides/developer/mutations) are only possible when all relicas are running. Otherwise the database will return an error:
8. **When performing a DELETE or UPDATE query, all services should be running (not stopped and not idled) (limitation will be removed before GA).** This is because currently [mutations](https://clickhouse.com/docs/en/guides/developer/mutations) are only possible when all relicas are running. Otherwise the database will return an error:

```
Code: 341. DB::Exception: Mutation is not finished because some replicas are inactive right now
Expand Down

0 comments on commit f1502df

Please sign in to comment.