Skip to content

Commit

Permalink
[FLINK-34180] Migrate doc website from ververica to flink (apache#3028)
Browse files Browse the repository at this point in the history
  • Loading branch information
GOODBOY008 authored and wuzhenhua01 committed Aug 4, 2024
1 parent 0e51c67 commit b55a2be
Show file tree
Hide file tree
Showing 214 changed files with 15,577 additions and 5,194 deletions.
86 changes: 63 additions & 23 deletions .github/workflows/build_docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,33 +13,73 @@
# See the License for the specific language governing permissions and
# limitations under the License.

name: build_docs

# execute this docs build workflow automatically when new push happens in any branch
name: "Build documentation"

on:
push:
paths:
- 'docs/**'

branches:
- master
- release-*
schedule:
- cron: '0 0 * * *' # Deploy every day

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:

build_docs_job:
build-documentation:
if: github.repository == 'apache/flink-cdc'
runs-on: ubuntu-latest
container: debian:buster-slim

strategy:
max-parallel: 1
matrix:
branch:
- master
- release-3.0

steps:

- name: Prereqs
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
apt-get update
apt-get install -y git
git clone "https://token:${GITHUB_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" .
shell: bash

- name: Execute script to build our documentation and update pages
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: "docs/build_docs.sh"
shell: bash
- uses: actions/checkout@v3
with:
ref: ${{ matrix.branch }}

- name: Set branch environment variable
run: |
currentBranch=$(git branch --show-current)
echo "flink_branch=${currentBranch}" >> ${GITHUB_ENV}
if [ "${currentBranch}" = "master" ]; then
echo "flink_alias=release-3.1" >> ${GITHUB_ENV}
elif [ "${currentBranch}" = "release-3.0" ]; then
echo "flink_alias=stable" >> ${GITHUB_ENV}
fi
- name: Build documentation
run: |
docker run --rm --volume "$PWD:/root/flink-cdc" chesnay/flink-ci:java_8_11_17_21_maven_386 bash -c "cd /root/flink-cdc && chmod +x ./.github/workflows/docs.sh && ./.github/workflows/docs.sh"
- name: Upload documentation
uses: burnett01/[email protected]
with:
switches: --archive --compress
path: docs/target/
remote_path: ${{ secrets.NIGHTLIES_RSYNC_PATH }}/flink/flink-cdc-docs-${{ env.flink_branch }}/
remote_host: ${{ secrets.NIGHTLIES_RSYNC_HOST }}
remote_port: ${{ secrets.NIGHTLIES_RSYNC_PORT }}
remote_user: ${{ secrets.NIGHTLIES_RSYNC_USER }}
remote_key: ${{ secrets.NIGHTLIES_RSYNC_KEY }}

- name: Upload documentation alias
if: env.flink_alias != ''
uses: burnett01/[email protected]
with:
switches: --archive --compress
path: docs/target/
remote_path: ${{ secrets.NIGHTLIES_RSYNC_PATH }}/flink/flink-cdc-docs-${{ env.flink_alias }}/
remote_host: ${{ secrets.NIGHTLIES_RSYNC_HOST }}
remote_port: ${{ secrets.NIGHTLIES_RSYNC_PORT }}
remote_user: ${{ secrets.NIGHTLIES_RSYNC_USER }}
remote_key: ${{ secrets.NIGHTLIES_RSYNC_KEY }}
59 changes: 59 additions & 0 deletions .github/workflows/docs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
#!/usr/bin/env bash
################################################################################
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################
set -e

mvn --version
java -version
javadoc -J-version

# workaround for a git security patch
git config --global --add safe.directory /root/flink-cdc
git submodule update --init --recursive

HUGO_REPO=https://github.com/gohugoio/hugo/releases/download/v0.80.0/hugo_extended_0.80.0_Linux-64bit.tar.gz
HUGO_ARTIFACT=hugo_extended_0.80.0_Linux-64bit.tar.gz
if ! curl --fail -OL $HUGO_REPO ; then
echo "Failed to download Hugo binary"
exit 1
fi
tar -zxvf $HUGO_ARTIFACT

# Build the docs
hugo --source docs

# generate docs into docs/target
hugo -v --source docs --destination target
if [ $? -ne 0 ]; then
echo "Error building the docs"
exit 1
fi

# build Flink; required for Javadoc step
mvn clean install -B -DskipTests -Dfast

# build java/scala docs
mkdir -p docs/target/api
mvn javadoc:aggregate -B \
-DadditionalJOption="-Xdoclint:none --allow-script-in-comments" \
-Dmaven.javadoc.failOnError=false \
-Dcheckstyle.skip=true \
-Dspotless.check.skip=true \
-Denforcer.skip=true \
-Dheader="<a href=\"http://flink.apache.org/\" target=\"_top\"><h1>Back to Flink Website</h1></a> <script>var _paq=window._paq=window._paq||[];_paq.push([\"disableCookies\"]),_paq.push([\"setDomains\",[\"*.flink.apache.org\",\"*.nightlies.apache.org/flink\"]]),_paq.push([\"trackPageView\"]),_paq.push([\"enableLinkTracking\"]),function(){var u=\"//matomo.privacy.apache.org/\";_paq.push([\"setTrackerUrl\",u+\"matomo.php\"]),_paq.push([\"setSiteId\",\"1\"]);var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s)}();</script>"
mv target/site/apidocs docs/target/api/java
6 changes: 6 additions & 0 deletions .github/workflows/flink_cdc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,16 @@ on:
branches:
- master
- release-*
paths-ignore:
- 'docs/**'
- 'README.md'
pull_request:
branches:
- master
- release-*
paths-ignore:
- 'docs/**'
- 'README.md'

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "docs/themes/book"]
path = docs/themes/book
url = https://github.com/alex-shpak/hugo-book
1 change: 1 addition & 0 deletions .idea/vcs.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

26 changes: 13 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,17 @@ This README is meant as a brief walkthrough on the core features of CDC Connecto

## Supported (Tested) Databases

| Connector | Database | Driver |
|------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------|
| [mongodb-cdc](docs/content/connectors/mongodb-cdc.md) | <li> [MongoDB](https://www.mongodb.com): 3.6, 4.x, 5.0, 6.0 | MongoDB Driver: 4.3.4 |
| [mysql-cdc](docs/content/connectors/mysql-cdc.md) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.28 |
| [oceanbase-cdc](/docs/content/connectors/oceanbase-cdc.md) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x, 4.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase): 2.x, 3.x, 4.x | OceanBase Driver: 2.4.x |
| [oracle-cdc](docs/content/connectors/oracle-cdc.md) | <li> [Oracle](https://www.oracle.com/index.html): 11, 12, 19, 21 | Oracle Driver: 19.3.0.0 |
| [postgres-cdc](docs/content/connectors/postgres-cdc.md) | <li> [PostgreSQL](https://www.postgresql.org): 9.6, 10, 11, 12, 13, 14 | JDBC Driver: 42.5.1 |
| [sqlserver-cdc](docs/content/connectors/sqlserver-cdc.md) | <li> [Sqlserver](https://www.microsoft.com/sql-server): 2012, 2014, 2016, 2017, 2019 | JDBC Driver: 9.4.1.jre8 |
| [tidb-cdc](docs/content/connectors/tidb-cdc.md) | <li> [TiDB](https://www.pingcap.com): 5.1.x, 5.2.x, 5.3.x, 5.4.x, 6.0.0 | JDBC Driver: 8.0.27 |
| [Db2-cdc](docs/content/connectors/db2-cdc.md) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
| [Vitess-cdc](docs/content/connectors/vitess-cdc.md) | <li> [Vitess](https://vitess.io/): 8.0.x, 9.0.x | MySql JDBC Driver: 8.0.26 |
| Connector | Database | Driver |
|-------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------|
| [mongodb-cdc](docs/content/docs/connectors/cdc-connectors/mongodb-cdc.md) | <li> [MongoDB](https://www.mongodb.com): 3.6, 4.x, 5.0, 6.0 | MongoDB Driver: 4.3.4 |
| [mysql-cdc](docs/content/docs/connectors/cdc-connectors/mysql-cdc.md) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.28 |
| [oceanbase-cdc](docs/content/docs/connectors/cdc-connectors/oceanbase-cdc.md) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x, 4.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase): 2.x, 3.x, 4.x | OceanBase Driver: 2.4.x |
| [oracle-cdc](docs/content/docs/connectors/cdc-connectors/oracle-cdc.md) | <li> [Oracle](https://www.oracle.com/index.html): 11, 12, 19, 21 | Oracle Driver: 19.3.0.0 |
| [postgres-cdc](docs/content/docs/connectors/cdc-connectors/postgres-cdc.md) | <li> [PostgreSQL](https://www.postgresql.org): 9.6, 10, 11, 12, 13, 14 | JDBC Driver: 42.5.1 |
| [sqlserver-cdc](docs/content/docs/connectors/cdc-connectors/sqlserver-cdc.md) | <li> [Sqlserver](https://www.microsoft.com/sql-server): 2012, 2014, 2016, 2017, 2019 | JDBC Driver: 9.4.1.jre8 |
| [tidb-cdc](docs/content/docs/connectors/cdc-connectors/tidb-cdc.md) | <li> [TiDB](https://www.pingcap.com): 5.1.x, 5.2.x, 5.3.x, 5.4.x, 6.0.0 | JDBC Driver: 8.0.27 |
| [Db2-cdc](docs/content/docs/connectors/cdc-connectors/db2-cdc.md) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
| [Vitess-cdc](docs/content/docs/connectors/cdc-connectors/vitess-cdc.md) | <li> [Vitess](https://vitess.io/): 8.0.x, 9.0.x | MySql JDBC Driver: 8.0.26 |

## Features

Expand Down Expand Up @@ -106,10 +106,10 @@ Include following Maven dependency (available through Maven Central):

```
<dependency>
<groupId>com.ververica</groupId>
<groupId>org.apache.flink</groupId>
<!-- add the dependency matching your database -->
<artifactId>flink-connector-mysql-cdc</artifactId>
<!-- The dependency is available only for stable releases, SNAPSHOT dependencies need to be built based on master or release- branches by yourself. -->
<!-- The dependency is available only for stable releases, SNAPSHOT dependencies need to be built based on master or release branches by yourself. -->
<version>2.5-SNAPSHOT</version>
</dependency>
```
Expand Down
10 changes: 10 additions & 0 deletions docs/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.bundle/
.jekyll-metadata
.jekyll-cache/
.rubydeps/
ruby2/.bundle/
ruby2/.rubydeps/
public/
resources/
.hugo_build.lock
.DS_Store
6 changes: 0 additions & 6 deletions docs/Dockerfile

This file was deleted.

19 changes: 0 additions & 19 deletions docs/Makefile

This file was deleted.

Loading

0 comments on commit b55a2be

Please sign in to comment.