Skip to content

Commit

Permalink
[Doc] Update Data Migration Tool
Browse files Browse the repository at this point in the history
Signed-off-by: EsoragotoSpirit <[email protected]>
  • Loading branch information
EsoragotoSpirit committed Feb 27, 2025
1 parent 9ad0484 commit 0ca6571
Show file tree
Hide file tree
Showing 2 changed files with 0 additions and 88 deletions.
44 changes: 0 additions & 44 deletions docs/en/administration/data_migration_tool.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,50 +47,6 @@ After the data migration is completed, you need to remove the configuration `ena
ADMIN SET FRONTEND CONFIG("enable_legacy_compatibility_for_replication"="false");
```

### Disable Compaction

If the target cluster for data migration is a shared-data cluster, you need to manually disable Compaction before starting the data migration and re-enable it after the data migration is completed.

1. You can check whether Compaction is enabled by using the following statement:

```SQL
ADMIN SHOW FRONTEND CONFIG LIKE 'lake_compaction_max_tasks';
```

If `0` is returned, it indicates that Compaction is disabled.

2. Dynamically disable Compaction:

```SQL
ADMIN SET FRONTEND CONFIG("lake_compaction_max_tasks"="0");
```

3. To prevent Compaction from automatically enabling during the data migration process in case of cluster restart, you also need to add the following configuration item in the FE configuration file **fe.conf**:

```Properties
lake_compaction_max_tasks = 0
```

After the data migration is completed, you need to remove the configuration `lake_compaction_max_tasks = 0` from the configuration file, and dynamically enable Compaction using the following statement:

```SQL
ADMIN SET FRONTEND CONFIG("lake_compaction_max_tasks"="-1");
```

### Disable column filtering

The optimization for unused column filtering at the Scan stage may cause a crash during queries against the migrated data. You need to disable this optimization before data migration:

```SQL
SET GLOBAL enable_filter_unused_columns_in_scan_stage=false;
```

#### enable_filter_unused_columns_in_scan_stage

* **Description**: Whether to filter unused column at the Scan stage.
* **Default**: true
* **Introduced in**: v3.1

### Configure Data Migration (Optional)

You can configure data migration operations using the following FE and BE parameters. In most cases, the default configuration can meet your needs. If you wish to use the default configuration, you can skip this step.
Expand Down
44 changes: 0 additions & 44 deletions docs/zh/administration/data_migration_tool.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,50 +47,6 @@ StarRocks 跨集群数据迁移工具是社区提供的 StarRocks 数据迁移
ADMIN SET FRONTEND CONFIG("enable_legacy_compatibility_for_replication"="false");
```

### 关闭 Compaction

如果数据迁移的目标集群为存算分离集群,在数据迁移之前,您需要手动关闭 Compaction,并在数据迁移完成后重新开启。

1. 您可以通过以下语句查看当前集群是否开启 Compaction:

```SQL
ADMIN SHOW FRONTEND CONFIG LIKE 'lake_compaction_max_tasks';
```

如果返回值为 `0` 则表示 Compaction 关闭。

2. 动态关闭 Compaction:

```SQL
ADMIN SET FRONTEND CONFIG("lake_compaction_max_tasks"="0");
```

3. 为防止数据迁移过程中集群重启后 Compaction 自动开启,您还需要在 FE 配置文件 **fe.conf** 中添加以下配置项:

```Properties
lake_compaction_max_tasks = 0
```

数据迁移完成后,您需要删除配置文件中的 `lake_compaction_max_tasks = 0`,并通过以下语句动态开启 Compaction:

```SQL
ADMIN SET FRONTEND CONFIG("lake_compaction_max_tasks"="-1");
```

### 禁用列过滤

在 SCAN 阶段过滤未使用列的优化可能会导致查询被迁移的数据时发生崩溃,因此您需要在数据迁移前禁用此优化:

```SQL
SET GLOBAL enable_filter_unused_columns_in_scan_stage=false;
```

#### enable_filter_unused_columns_in_scan_stage

- 描述:是否在 SCAN 阶段过滤未使用的列。
- 默认值:true
- 引入版本:v3.1

### 配置数据迁移(可选)

您可以通过以下 FE 和 BE 参数配置数据迁移操作。通常情况下,默认配置即可满足需求。如果您想保留默认配置,可以选择跳过该步骤。
Expand Down

0 comments on commit 0ca6571

Please sign in to comment.