Skip to content

Commit

Permalink
[FLINK-35871][doc] Add missed "snapshot" mode to mysql connector star…
Browse files Browse the repository at this point in the history
…tup options

This closes #3484.
  • Loading branch information
lvyanquan authored Jul 22, 2024
1 parent 26ff6d2 commit fcb4cd8
Show file tree
Hide file tree
Showing 4 changed files with 13 additions and 4 deletions.
4 changes: 3 additions & 1 deletion docs/content.zh/docs/connectors/flink-sources/mysql-cdc.md
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ Flink SQL> SELECT * FROM orders;
<td style="word-wrap: break-word;">initial</td>
<td>String</td>
<td> MySQL CDC 消费者可选的启动模式,
合法的模式为 "initial","earliest-offset","latest-offset","specific-offset" 和 "timestamp"。
合法的模式为 "initial","earliest-offset","latest-offset","specific-offset","timestamp" 和 "snapshot"。
请查阅 <a href="#a-name-id-002-a">启动模式</a> 章节了解更多详细信息。</td>
</tr>
<tr>
Expand Down Expand Up @@ -629,6 +629,7 @@ MySQLSource.builder()
.startupOptions(StartupOptions.specificOffset("mysql-bin.000003", 4L) // 从指定 binlog 文件名和位置启动
.startupOptions(StartupOptions.specificOffset("24DA167-0C0C-11E8-8442-00059A3C7B00:1-19")) // 从 GTID 集合启动
.startupOptions(StartupOptions.timestamp(1667232000000L) // 从时间戳启动
.startupOptions(StartupOptions.snapshot()) // 仅读取快照
...
.build()
```
Expand All @@ -642,6 +643,7 @@ CREATE TABLE mysql_source (...) WITH (
'scan.startup.mode' = 'latest-offset', -- 从最晚位点启动
'scan.startup.mode' = 'specific-offset', -- 从特定位点启动
'scan.startup.mode' = 'timestamp', -- 从特定位点启动
'scan.startup.mode' = 'snapshot', -- 仅读取快照
'scan.startup.specific-offset.file' = 'mysql-bin.000003', -- 在特定位点启动模式下指定 binlog 文件名
'scan.startup.specific-offset.pos' = '4', -- 在特定位点启动模式下指定 binlog 位置
'scan.startup.specific-offset.gtid-set' = '24DA167-0C0C-11E8-8442-00059A3C7B00:1-19', -- 在特定位点启动模式下指定 GTID 集合
Expand Down
4 changes: 3 additions & 1 deletion docs/content.zh/docs/connectors/pipeline-connectors/mysql.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ pipeline:
<td style="word-wrap: break-word;">initial</td>
<td>String</td>
<td> MySQL CDC 消费者可选的启动模式,
合法的模式为 "initial","earliest-offset","latest-offset","specific-offset" 和 "timestamp"。</td>
合法的模式为 "initial","earliest-offset","latest-offset","specific-offset","timestamp" 和 ""snapshot"。</td>
</tr>
<tr>
<td>scan.startup.specific-offset.file</td>
Expand Down Expand Up @@ -280,6 +280,7 @@ pipeline:
- `latest-offset`:首次启动时,从不对受监视的数据库表执行快照, 连接器仅从 binlog 的结尾处开始读取,这意味着连接器只能读取在连接器启动之后的数据更改。
- `specific-offset`:跳过快照阶段,从指定的 binlog 位点开始读取。位点可通过 binlog 文件名和位置指定,或者在 GTID 在集群上启用时通过 GTID 集合指定。
- `timestamp`:跳过快照阶段,从指定的时间戳开始读取 binlog 事件。
- `snapshot`: 只进行快照阶段,跳过增量阶段,快照阶段读取结束后退出。

例如,可以在 YAML 配置文件中这样指定启动模式:

Expand All @@ -290,6 +291,7 @@ source:
scan.startup.mode: latest-offset # Start from latest offset
scan.startup.mode: specific-offset # Start from specific offset
scan.startup.mode: timestamp # Start from timestamp
scan.startup.mode: snapshot # Read snapshot only
scan.startup.specific-offset.file: 'mysql-bin.000003' # Binlog filename under specific offset startup mode
scan.startup.specific-offset.pos: 4 # Binlog position under specific offset mode
scan.startup.specific-offset.gtid-set: 24DA167-... # GTID set under specific offset startup mode
Expand Down
5 changes: 4 additions & 1 deletion docs/content/docs/connectors/flink-sources/mysql-cdc.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ Connector Options
<td>optional</td>
<td style="word-wrap: break-word;">initial</td>
<td>String</td>
<td>Optional startup mode for MySQL CDC consumer, valid enumerations are "initial", "earliest-offset", "latest-offset", "specific-offset" and "timestamp".
<td>Optional startup mode for MySQL CDC consumer, valid enumerations are "initial", "earliest-offset", "latest-offset", "specific-offset", "timestamp" and "snapshot".
Please see <a href="#startup-reading-position">Startup Reading Position</a> section for more detailed information.</td>
</tr>
<tr>
Expand Down Expand Up @@ -653,6 +653,7 @@ the end of the binlog which means only have the changes since the connector was
- `specific-offset`: Skip snapshot phase and start reading binlog events from a specific offset. The offset could be
specified with binlog filename and position, or a GTID set if GTID is enabled on server.
- `timestamp`: Skip snapshot phase and start reading binlog events from a specific timestamp.
- `snapshot`: Only the snapshot phase is performed and exits after the snapshot phase reading is completed.

For example in DataStream API:
```java
Expand All @@ -662,6 +663,7 @@ MySQLSource.builder()
.startupOptions(StartupOptions.specificOffset("mysql-bin.000003", 4L) // Start from binlog file and offset
.startupOptions(StartupOptions.specificOffset("24DA167-0C0C-11E8-8442-00059A3C7B00:1-19")) // Start from GTID set
.startupOptions(StartupOptions.timestamp(1667232000000L) // Start from timestamp
.startupOptions(StartupOptions.snapshot()) // Read snapshot only
...
.build()
```
Expand All @@ -675,6 +677,7 @@ CREATE TABLE mysql_source (...) WITH (
'scan.startup.mode' = 'latest-offset', -- Start from latest offset
'scan.startup.mode' = 'specific-offset', -- Start from specific offset
'scan.startup.mode' = 'timestamp', -- Start from timestamp
'scan.startup.mode' = 'snapshot', -- Read snapshot only
'scan.startup.specific-offset.file' = 'mysql-bin.000003', -- Binlog filename under specific offset startup mode
'scan.startup.specific-offset.pos' = '4', -- Binlog position under specific offset mode
'scan.startup.specific-offset.gtid-set' = '24DA167-0C0C-11E8-8442-00059A3C7B00:1-19', -- GTID set under specific offset startup mode
Expand Down
4 changes: 3 additions & 1 deletion docs/content/docs/connectors/pipeline-connectors/mysql.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ pipeline:
<td>optional</td>
<td style="word-wrap: break-word;">initial</td>
<td>String</td>
<td>Optional startup mode for MySQL CDC consumer, valid enumerations are "initial", "earliest-offset", "latest-offset", "specific-offset" and "timestamp".</td>
<td>Optional startup mode for MySQL CDC consumer, valid enumerations are "initial", "earliest-offset", "latest-offset", "specific-offset", "timestamp" and "snapshot".</td>
</tr>
<tr>
<td>scan.startup.specific-offset.file</td>
Expand Down Expand Up @@ -290,6 +290,7 @@ The config option `scan.startup.mode` specifies the startup mode for MySQL CDC c
- `specific-offset`: Skip snapshot phase and start reading binlog events from a specific offset. The offset could be
specified with binlog filename and position, or a GTID set if GTID is enabled on server.
- `timestamp`: Skip snapshot phase and start reading binlog events from a specific timestamp.
- `snapshot`: Only the snapshot phase is performed and exits after the snapshot phase reading is completed.

For example in YAML definition:

Expand All @@ -300,6 +301,7 @@ source:
scan.startup.mode: latest-offset # Start from latest offset
scan.startup.mode: specific-offset # Start from specific offset
scan.startup.mode: timestamp # Start from timestamp
scan.startup.mode: snapshot # Read snapshot only
scan.startup.specific-offset.file: 'mysql-bin.000003' # Binlog filename under specific offset startup mode
scan.startup.specific-offset.pos: 4 # Binlog position under specific offset mode
scan.startup.specific-offset.gtid-set: 24DA167-... # GTID set under specific offset startup mode
Expand Down

0 comments on commit fcb4cd8

Please sign in to comment.