github-forks
/
flink-cdc-connectors
mirror of https://github.com/ververica/flink-cdc-connectors.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[FLINK-34724][docs] Fix broken web link in document of legacy Flink CDC Sources (#3167)

Browse Source
pull/3169/head
Hongshun Wang 11 months ago committed by GitHub
parent 32c8e0bc2a
commit 8098a9ca3e
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
21 changed files with 112 additions and 75 deletions
Show Stats Download Patch File Download Diff File

2
docs/config.toml
Unescape Escape View File

@ -58,7 +58,7 @@ pygmentsUseClasses = true
]
PreviousDocs = [
["3.0", "https://nightlies.apache.org/flink-cdc/flink-cdc-docs-release-3.0"],
["3.0", "https://nightlies.apache.org/flink/flink-cdc-docs-release-3.0"],
]
[markup]

7
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/db2-cdc.md
Unescape Escape View File

@ -34,7 +34,7 @@ describes how to setup the db2 CDC connector to run SQL queries against Db2 data
| Connector | Database | Driver |
|-----------------------|----------------------------------------------------|----------------------|
| [Db2-cdc](db2-cdc.md) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
| [Db2-cdc](../db2-cdc) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
Dependencies
------------
@ -53,9 +53,8 @@ using a build automation tool (such as Maven or SBT) and SQL Client with SQL JAR
Download flink-sql-connector-db2-cdc-3.0-SNAPSHOT.jar and
put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-db2-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users
need to download the source code and compile the corresponding jar. Users should use the released version, such as
[flink-sql-connector-db2-cdc-2.3.0.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-connector-db2-cdc),
**Note:** flink-sql-connector-db2-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users should use the released version, such as
[flink-sql-connector-db2-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-connector-db2-cdc),
the released version will be available in the Maven central warehouse.
Setup Db2 server

4
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/mongodb-cdc.md
Unescape Escape View File

@ -43,7 +43,7 @@ In order to setup the MongoDB CDC connector, the following table provides depend
Download [flink-sql-connector-mongodb-cdc-3.0-SNAPSHOT.jar](https://repo1.maven.org/maven2/org/apache/flink/flink-sql-connector-mongodb-cdc/3.0-SNAPSHOT/flink-sql-connector-mongodb-cdc-3.0-SNAPSHOT.jar) and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-mongodb-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-mongodb-cdc-2.2.1.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-mongodb-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-mongodb-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-mongodb-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-mongodb-cdc), the released version will be available in the Maven central warehouse.
Setup MongoDB
----------------
@ -52,7 +52,7 @@ Setup MongoDB
- MongoDB version
MongoDB version >= 3.6 <br>
We use [change streams](https://docs.mongodb.com/manual/changeStreams/) feature (new in version 3.6) to capture change data.
We use [change streams](https://docs.mongodb.com/manual/changeStreams/) feature (new in version 3.6) to capture change data.
- Cluster Deployment

8
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/mysql-cdc.md
Unescape Escape View File

@ -31,9 +31,9 @@ The MySQL CDC connector allows for reading snapshot data and incremental data fr
## Supported Databases
| Connector | Database | Driver |
|-----------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|
| [mysql-cdc](mysql-cdc.md) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.27 |
| Connector | Database | Driver |
|---------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|
| [mysql-cdc](../mysql-cdc) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.27 |
Dependencies
------------
@ -50,7 +50,7 @@ In order to setup the MySQL CDC connector, the following table provides dependen
Download flink-sql-connector-mysql-cdc-3.0-SNAPSHOT.jar and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-mysql-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-mysql-cdc-2.3.0.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-connector-mysql-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-mysql-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users should use the released version, such as [flink-sql-connector-mysql-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-connector-mysql-cdc), the released version will be available in the Maven central warehouse.
Setup MySQL server
----------------

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/oceanbase-cdc.md
Unescape Escape View File

@ -51,7 +51,7 @@ If you want to use OceanBase JDBC driver to connect to the enterprise edition da
Download [flink-sql-connector-oceanbase-cdc-3.0-SNAPSHOT.jar](https://repo1.maven.org/maven2/org/apache/flink/flink-sql-connector-oceanbase-cdc/3.0-SNAPSHOT/flink-sql-connector-oceanbase-cdc-3.0-SNAPSHOT.jar) and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-oceanbase-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-oceanbase-cdc-2.2.1.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-oceanbase-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-oceanbase-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-oceanbase-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-oceanbase-cdc), the released version will be available in the Maven central warehouse.
For JDBC driver, the cdc jar above already contains MySQL JDBC driver 5.1.47, which is our recommended version. Due to the license issue, we can not include the OceanBase JDBC driver in the cdc jar. If you need to use it, you can download it from [here](https://repo1.maven.org/maven2/com/oceanbase/oceanbase-client/2.4.2/oceanbase-client-2.4.2.jar) and put it under `<FLINK_HOME>/lib/`, you also need to set the start option `jdbc.driver` to `com.oceanbase.jdbc.Driver`.

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/oracle-cdc.md
Unescape Escape View File

@ -43,7 +43,7 @@ In order to setup the Oracle CDC connector, the following table provides depende
Download [flink-sql-connector-oracle-cdc-3.0-SNAPSHOT.jar](https://repo1.maven.org/maven2/org/apache/flink/flink-sql-connector-oracle-cdc/3.0-SNAPSHOT/flink-sql-connector-oracle-cdc-3.0-SNAPSHOT.jar) and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-oracle-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-oracle-cdc-2.3.0.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-oracle-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-oracle-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-oracle-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-oracle-cdc), the released version will be available in the Maven central warehouse.
Setup Oracle
----------------

64
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/overview.md
Unescape Escape View File

@ -24,10 +24,11 @@ specific language governing permissions and limitations
under the License.
-->
# CDC Connectors for Apache Flink
# Legacy CDC sources for Apache Flink
Flink CDC sources is a set of source connectors for <a href="https://flink.apache.org/">Apache Flink<sup>®</sup></a>, ingesting changes from different databases using change data capture (CDC).
Some CDC sources integrate Debezium as the engine to capture data changes. So it can fully leverage the ability of Debezium. See more about what is [Debezium](https://github.com/debezium/debezium).
CDC Connectors for Apache Flink<sup>®</sup> is a set of source connectors for <a href="https://flink.apache.org/">Apache Flink<sup>®</sup></a>, ingesting changes from different databases using change data capture (CDC).
The CDC Connectors for Apache Flink<sup>®</sup> integrate Debezium as the engine to capture data changes. So it can fully leverage the ability of Debezium. See more about what is [Debezium](https://github.com/debezium/debezium).
{{< img src="/fig/cdc-flow.png" width="600px" alt="Flink CDC" >}}
@ -35,15 +36,15 @@ The CDC Connectors for Apache Flink<sup>®</sup> integrate Debezium as the engin
| Connector | Database | Driver |
|-----------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------|
| [mongodb-cdc](mongodb-cdc.md) | <li> [MongoDB](https://www.mongodb.com): 3.6, 4.x, 5.0 | MongoDB Driver: 4.3.4 |
| [mysql-cdc](mysql-cdc.md) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.28 |
| [oceanbase-cdc](oceanbase-cdc.md) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x, 4.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase): 2.x, 3.x, 4.x | OceanBase Driver: 2.4.x |
| [oracle-cdc](oracle-cdc.md) | <li> [Oracle](https://www.oracle.com/index.html): 11, 12, 19, 21 | Oracle Driver: 19.3.0.0 |
| [postgres-cdc](postgres-cdc.md) | <li> [PostgreSQL](https://www.postgresql.org): 9.6, 10, 11, 12, 13, 14 | JDBC Driver: 42.5.1 |
| [sqlserver-cdc](sqlserver-cdc.md) | <li> [Sqlserver](https://www.microsoft.com/sql-server): 2012, 2014, 2016, 2017, 2019 | JDBC Driver: 9.4.1.jre8 |
| [tidb-cdc](tidb-cdc.md) | <li> [TiDB](https://www.pingcap.com/): 5.1.x, 5.2.x, 5.3.x, 5.4.x, 6.0.0 | JDBC Driver: 8.0.27 |
| [db2-cdc](db2-cdc.md) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
| [vitess-cdc](vitess-cdc.md) | <li> [Vitess](https://vitess.io/): 8.0.x, 9.0.x | MySql JDBC Driver: 8.0.26 |
| [mongodb-cdc](../mongodb-cdc) | <li> [MongoDB](https://www.mongodb.com): 3.6, 4.x, 5.0 | MongoDB Driver: 4.3.4 |
| [mysql-cdc](../mysql-cdc) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.28 |
| [oceanbase-cdc](../oceanbase-cdc) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x, 4.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase): 2.x, 3.x, 4.x | OceanBase Driver: 2.4.x |
| [oracle-cdc](../oracle-cdc) | <li> [Oracle](https://www.oracle.com/index.html): 11, 12, 19, 21 | Oracle Driver: 19.3.0.0 |
| [postgres-cdc](../postgres-cdc) | <li> [PostgreSQL](https://www.postgresql.org): 9.6, 10, 11, 12, 13, 14 | JDBC Driver: 42.5.1 |
| [sqlserver-cdc](../sqlserver-cdc) | <li> [Sqlserver](https://www.microsoft.com/sql-server): 2012, 2014, 2016, 2017, 2019 | JDBC Driver: 9.4.1.jre8 |
| [tidb-cdc](../tidb-cdc) | <li> [TiDB](https://www.pingcap.com/): 5.1.x, 5.2.x, 5.3.x, 5.4.x, 6.0.0 | JDBC Driver: 8.0.27 |
| [db2-cdc](../db2-cdc) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
| [vitess-cdc](../vitess-cdc) | <li> [Vitess](https://vitess.io/): 8.0.x, 9.0.x | MySql JDBC Driver: 8.0.26 |
## Supported Flink Versions
The following table shows the version mapping between Flink<sup>®</sup> CDC Connectors and Flink<sup>®</sup>:
@ -70,24 +71,24 @@ The following table shows the version mapping between Flink<sup>®</sup> CDC Con
The following table shows the current features of the connector:
| Connector | No-lock Read | Parallel Read | Exactly-once Read | Incremental Snapshot Read |
|-----------------------------------|--------------|---------------|-------------------|---------------------------|
| [mongodb-cdc](mongodb-cdc.md) | ✅ | ✅ | ✅ | ✅ |
| [mysql-cdc](mysql-cdc.md) | ✅ | ✅ | ✅ | ✅ |
| [oracle-cdc](oracle-cdc.md) | ✅ | ✅ | ✅ | ✅ |
| [postgres-cdc](postgres-cdc.md) | ✅ | ✅ | ✅ | ✅ |
| [sqlserver-cdc](sqlserver-cdc.md) | ✅ | ✅ | ✅ | ✅ |
| [oceanbase-cdc](oceanbase-cdc.md) | ❌ | ❌ | ❌ | ❌ |
| [tidb-cdc](tidb-cdc.md) | ✅ | ❌ | ✅ | ❌ |
| [db2-cdc](db2-cdc.md) | ❌ | ❌ | ✅ | ❌ |
| [vitess-cdc](vitess-cdc.md) | ✅ | ❌ | ✅ | ❌ |
| Connector | No-lock Read | Parallel Read | Exactly-once Read | Incremental Snapshot Read |
|--------------------------------|--------------|---------------|-------------------|---------------------------|
| [mongodb-cdc](../mongodb-cdc) | ✅ | ✅ | ✅ | ✅ |
| [mysql-cdc](../mysql-cdc) | ✅ | ✅ | ✅ | ✅ |
| [oracle-cdc](../oracle-cdc) | ✅ | ✅ | ✅ | ✅ |
| [postgres-cdc](../postgres-cdc) | ✅ | ✅ | ✅ | ✅ |
| [sqlserver-cdc](../sqlserver-cdc) | ✅ | ✅ | ✅ | ✅ |
| [oceanbase-cdc](../oceanbase-cdc) | ❌ | ❌ | ❌ | ❌ |
| [tidb-cdc](../tidb-cdc) | ✅ | ❌ | ✅ | ❌ |
| [db2-cdc](../db2-cdc) | ❌ | ❌ | ✅ | ❌ |
| [vitess-cdc](../vitess-cdc) | ✅ | ❌ | ✅ | ❌ |
## Usage for Table/SQL API
We need several steps to setup a Flink cluster with the provided connector.
1. Setup a Flink cluster with version 1.12+ and Java 8+ installed.
2. Download the connector SQL jars from the [Downloads](../downloads.md) page (or [build yourself](#building-from-source)).
2. Download the connector SQL jars from the [Downloads](https://github.com/apache/flink-cdc/releases) page (or [build yourself](#building-from-source)).
3. Put the downloaded jars under `FLINK_HOME/lib/`.
4. Restart the Flink cluster.
@ -304,4 +305,19 @@ you can construct `JsonDebeziumDeserializationSchema` as following:
new JsonDebeziumDeserializationSchema(true, customConverterConfigs);
```
## Building from source
Prerequisites:
- git
- Maven
- At least Java 8
```
git clone https://github.com/apache/flink-cdc.git
cd flink-cdc
mvn clean install -DskipTests
```
The dependencies are now available in your local `.m2` repository.
{{< top >}}

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/postgres-cdc.md
Unescape Escape View File

@ -43,7 +43,7 @@ In order to setup the Postgres CDC connector, the following table provides depen
Download flink-sql-connector-postgres-cdc-3.0-SNAPSHOT.jar and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-postgres-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-postgres-cdc-2.3.0.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-postgres-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-postgres-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users should use the released version, such as [flink-sql-connector-postgres-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-postgres-cdc), the released version will be available in the Maven central warehouse.
How to create a Postgres CDC table
----------------

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/sqlserver-cdc.md
Unescape Escape View File

@ -43,7 +43,7 @@ In order to setup the SQLServer CDC connector, the following table provides depe
Download [flink-sql-connector-sqlserver-cdc-3.0-SNAPSHOT.jar](https://repo1.maven.org/maven2/org/apache/flink/flink-sql-connector-sqlserver-cdc/3.0-SNAPSHOT/flink-sql-connector-sqlserver-cdc-3.0-SNAPSHOT.jar) and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-sqlserver-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-sqlserver-cdc-2.2.1.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-sqlserver-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-sqlserver-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users should use the released version, such as [flink-sql-connector-sqlserver-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-sqlserver-cdc), the released version will be available in the Maven central warehouse.
Setup SQLServer Database
----------------

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/tidb-cdc.md
Unescape Escape View File

@ -43,7 +43,7 @@ In order to setup the TiDB CDC connector, the following table provides dependenc
Download [flink-sql-connector-tidb-cdc-3.0-SNAPSHOT.jar](https://repo1.maven.org/maven2/org/apache/flink/flink-sql-connector-tidb-cdc/3.0-SNAPSHOT/flink-sql-connector-tidb-cdc-3.0-SNAPSHOT.jar) and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-tidb-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-tidb-cdc-2.2.1.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-tidb-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-tidb-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-tidb-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-tidb-cdc), the released version will be available in the Maven central warehouse.
How to create a TiDB CDC table
----------------

4
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/vitess-cdc.md
Unescape Escape View File

@ -42,6 +42,10 @@ In order to setup the Vitess CDC connector, the following table provides depende
Download [flink-sql-connector-vitess-cdc-3.0-SNAPSHOT.jar](https://repo1.maven.org/maven2/org/apache/flink/flink-sql-connector-vitess-cdc/3.0-SNAPSHOT/flink-sql-connector-vitess-cdc-3.0-SNAPSHOT.jar) and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-vitess-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users should use the released version, such as
[flink-sql-connector-vitess-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-connector-vitess-cdc),
the released version will be available in the Maven central warehouse.
Setup Vitess server
----------------

7
docs/content/docs/connectors/legacy-flink-cdc-sources/db2-cdc.md
Unescape Escape View File

@ -34,7 +34,7 @@ describes how to setup the db2 CDC connector to run SQL queries against Db2 data
| Connector | Database | Driver |
|-----------------------|----------------------------------------------------|----------------------|
| [Db2-cdc](db2-cdc.md) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
| [Db2-cdc](../db2-cdc) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
Dependencies
------------
@ -53,9 +53,8 @@ using a build automation tool (such as Maven or SBT) and SQL Client with SQL JAR
Download flink-sql-connector-db2-cdc-3.0-SNAPSHOT.jar and
put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-db2-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users
need to download the source code and compile the corresponding jar. Users should use the released version, such as
[flink-sql-connector-db2-cdc-2.3.0.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-connector-db2-cdc),
**Note:** flink-sql-connector-db2-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users should use the released version, such as
[flink-sql-connector-db2-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-connector-db2-cdc),
the released version will be available in the Maven central warehouse.
Setup Db2 server

2
docs/content/docs/connectors/legacy-flink-cdc-sources/mongodb-cdc.md
Unescape Escape View File

@ -43,7 +43,7 @@ In order to setup the MongoDB CDC connector, the following table provides depend
Download [flink-sql-connector-mongodb-cdc-3.0-SNAPSHOT.jar](https://repo1.maven.org/maven2/org/apache/flink/flink-sql-connector-mongodb-cdc/3.0-SNAPSHOT/flink-sql-connector-mongodb-cdc-3.0-SNAPSHOT.jar) and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-mongodb-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-mongodb-cdc-2.2.1.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-mongodb-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-mongodb-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-mongodb-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-mongodb-cdc), the released version will be available in the Maven central warehouse.
Setup MongoDB
----------------

8
docs/content/docs/connectors/legacy-flink-cdc-sources/mysql-cdc.md
Unescape Escape View File

@ -31,9 +31,9 @@ The MySQL CDC connector allows for reading snapshot data and incremental data fr
## Supported Databases
| Connector | Database | Driver |
|-----------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|
| [mysql-cdc](mysql-cdc.md) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.27 |
| Connector | Database | Driver |
|---------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|
| [mysql-cdc](../mysql-cdc) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.27 |
Dependencies
------------
@ -50,7 +50,7 @@ In order to setup the MySQL CDC connector, the following table provides dependen
Download flink-sql-connector-mysql-cdc-3.0-SNAPSHOT.jar and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-mysql-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-mysql-cdc-2.3.0.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-connector-mysql-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-mysql-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users should use the released version, such as [flink-sql-connector-mysql-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-connector-mysql-cdc), the released version will be available in the Maven central warehouse.
Setup MySQL server
----------------

2
docs/content/docs/connectors/legacy-flink-cdc-sources/oceanbase-cdc.md
Unescape Escape View File

@ -51,7 +51,7 @@ If you want to use OceanBase JDBC driver to connect to the enterprise edition da
Download [flink-sql-connector-oceanbase-cdc-3.0-SNAPSHOT.jar](https://repo1.maven.org/maven2/org/apache/flink/flink-sql-connector-oceanbase-cdc/3.0-SNAPSHOT/flink-sql-connector-oceanbase-cdc-3.0-SNAPSHOT.jar) and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-oceanbase-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-oceanbase-cdc-2.2.1.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-oceanbase-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-oceanbase-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-oceanbase-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-oceanbase-cdc), the released version will be available in the Maven central warehouse.
For JDBC driver, the cdc jar above already contains MySQL JDBC driver 5.1.47, which is our recommended version. Due to the license issue, we can not include the OceanBase JDBC driver in the cdc jar. If you need to use it, you can download it from [here](https://repo1.maven.org/maven2/com/oceanbase/oceanbase-client/2.4.2/oceanbase-client-2.4.2.jar) and put it under `<FLINK_HOME>/lib/`, you also need to set the start option `jdbc.driver` to `com.oceanbase.jdbc.Driver`.

2
docs/content/docs/connectors/legacy-flink-cdc-sources/oracle-cdc.md
Unescape Escape View File

@ -43,7 +43,7 @@ In order to setup the Oracle CDC connector, the following table provides depende
Download [flink-sql-connector-oracle-cdc-3.0-SNAPSHOT.jar](https://repo1.maven.org/maven2/org/apache/flink/flink-sql-connector-oracle-cdc/3.0-SNAPSHOT/flink-sql-connector-oracle-cdc-3.0-SNAPSHOT.jar) and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-oracle-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-oracle-cdc-2.3.0.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-oracle-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-oracle-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-oracle-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-oracle-cdc), the released version will be available in the Maven central warehouse.
Setup Oracle
----------------

57
docs/content/docs/connectors/legacy-flink-cdc-sources/overview.md
Unescape Escape View File

@ -37,15 +37,15 @@ You can also read [tutorials]({{< ref "docs/connectors/legacy-flink-cdc-sources/
| Connector | Database | Driver |
|-----------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------|
| [mongodb-cdc](mongodb-cdc.md) | <li> [MongoDB](https://www.mongodb.com): 3.6, 4.x, 5.0 | MongoDB Driver: 4.3.4 |
| [mysql-cdc](mysql-cdc.md) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.28 |
| [oceanbase-cdc](oceanbase-cdc.md) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x, 4.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase): 2.x, 3.x, 4.x | OceanBase Driver: 2.4.x |
| [oracle-cdc](oracle-cdc.md) | <li> [Oracle](https://www.oracle.com/index.html): 11, 12, 19, 21 | Oracle Driver: 19.3.0.0 |
| [postgres-cdc](postgres-cdc.md) | <li> [PostgreSQL](https://www.postgresql.org): 9.6, 10, 11, 12, 13, 14 | JDBC Driver: 42.5.1 |
| [sqlserver-cdc](sqlserver-cdc.md) | <li> [Sqlserver](https://www.microsoft.com/sql-server): 2012, 2014, 2016, 2017, 2019 | JDBC Driver: 9.4.1.jre8 |
| [tidb-cdc](tidb-cdc.md) | <li> [TiDB](https://www.pingcap.com/): 5.1.x, 5.2.x, 5.3.x, 5.4.x, 6.0.0 | JDBC Driver: 8.0.27 |
| [db2-cdc](db2-cdc.md) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
| [vitess-cdc](vitess-cdc.md) | <li> [Vitess](https://vitess.io/): 8.0.x, 9.0.x | MySql JDBC Driver: 8.0.26 |
| [mongodb-cdc](../mongodb-cdc) | <li> [MongoDB](https://www.mongodb.com): 3.6, 4.x, 5.0 | MongoDB Driver: 4.3.4 |
| [mysql-cdc](../mysql-cdc) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.28 |
| [oceanbase-cdc](../oceanbase-cdc) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x, 4.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase): 2.x, 3.x, 4.x | OceanBase Driver: 2.4.x |
| [oracle-cdc](../oracle-cdc) | <li> [Oracle](https://www.oracle.com/index.html): 11, 12, 19, 21 | Oracle Driver: 19.3.0.0 |
| [postgres-cdc](../postgres-cdc) | <li> [PostgreSQL](https://www.postgresql.org): 9.6, 10, 11, 12, 13, 14 | JDBC Driver: 42.5.1 |
| [sqlserver-cdc](../sqlserver-cdc) | <li> [Sqlserver](https://www.microsoft.com/sql-server): 2012, 2014, 2016, 2017, 2019 | JDBC Driver: 9.4.1.jre8 |
| [tidb-cdc](../tidb-cdc) | <li> [TiDB](https://www.pingcap.com/): 5.1.x, 5.2.x, 5.3.x, 5.4.x, 6.0.0 | JDBC Driver: 8.0.27 |
| [db2-cdc](../db2-cdc) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
| [vitess-cdc](../vitess-cdc) | <li> [Vitess](https://vitess.io/): 8.0.x, 9.0.x | MySql JDBC Driver: 8.0.26 |
## Supported Flink Versions
The following table shows the version mapping between Flink<sup>®</sup> CDC Connectors and Flink<sup>®</sup>:
@ -72,24 +72,24 @@ The following table shows the version mapping between Flink<sup>®</sup> CDC Con
The following table shows the current features of the connector:
| Connector | No-lock Read | Parallel Read | Exactly-once Read | Incremental Snapshot Read |
|-----------------------------------|--------------|---------------|-------------------|---------------------------|
| [mongodb-cdc](mongodb-cdc.md) | ✅ | ✅ | ✅ | ✅ |
| [mysql-cdc](mysql-cdc.md) | ✅ | ✅ | ✅ | ✅ |
| [oracle-cdc](oracle-cdc.md) | ✅ | ✅ | ✅ | ✅ |
| [postgres-cdc](postgres-cdc.md) | ✅ | ✅ | ✅ | ✅ |
| [sqlserver-cdc](sqlserver-cdc.md) | ✅ | ✅ | ✅ | ✅ |
| [oceanbase-cdc](oceanbase-cdc.md) | ❌ | ❌ | ❌ | ❌ |
| [tidb-cdc](tidb-cdc.md) | ✅ | ❌ | ✅ | ❌ |
| [db2-cdc](db2-cdc.md) | ❌ | ❌ | ✅ | ❌ |
| [vitess-cdc](vitess-cdc.md) | ✅ | ❌ | ✅ | ❌ |
| Connector | No-lock Read | Parallel Read | Exactly-once Read | Incremental Snapshot Read |
|--------------------------------|--------------|---------------|-------------------|---------------------------|
| [mongodb-cdc](../mongodb-cdc) | ✅ | ✅ | ✅ | ✅ |
| [mysql-cdc](../mysql-cdc) | ✅ | ✅ | ✅ | ✅ |
| [oracle-cdc](../oracle-cdc) | ✅ | ✅ | ✅ | ✅ |
| [postgres-cdc](../postgres-cdc) | ✅ | ✅ | ✅ | ✅ |
| [sqlserver-cdc](../sqlserver-cdc) | ✅ | ✅ | ✅ | ✅ |
| [oceanbase-cdc](../oceanbase-cdc) | ❌ | ❌ | ❌ | ❌ |
| [tidb-cdc](../tidb-cdc) | ✅ | ❌ | ✅ | ❌ |
| [db2-cdc](../db2-cdc) | ❌ | ❌ | ✅ | ❌ |
| [vitess-cdc](../vitess-cdc) | ✅ | ❌ | ✅ | ❌ |
## Usage for Table/SQL API
We need several steps to setup a Flink cluster with the provided connector.
1. Setup a Flink cluster with version 1.12+ and Java 8+ installed.
2. Download the connector SQL jars from the [Downloads](../downloads.md) page (or [build yourself](#building-from-source)).
2. Download the connector SQL jars from the [Downloads](https://github.com/apache/flink-cdc/releases) page (or [build yourself](#building-from-source)).
3. Put the downloaded jars under `FLINK_HOME/lib/`.
4. Restart the Flink cluster.
@ -306,4 +306,19 @@ you can construct `JsonDebeziumDeserializationSchema` as following:
new JsonDebeziumDeserializationSchema(true, customConverterConfigs);
```
## Building from source
Prerequisites:
- git
- Maven
- At least Java 8
```
git clone https://github.com/apache/flink-cdc.git
cd flink-cdc
mvn clean install -DskipTests
```
The dependencies are now available in your local `.m2` repository.
{{< top >}}

2
docs/content/docs/connectors/legacy-flink-cdc-sources/postgres-cdc.md
Unescape Escape View File

@ -43,7 +43,7 @@ In order to setup the Postgres CDC connector, the following table provides depen
Download flink-sql-connector-postgres-cdc-3.0-SNAPSHOT.jar and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-postgres-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-postgres-cdc-2.3.0.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-postgres-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-postgres-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users should use the released version, such as [flink-sql-connector-postgres-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-postgres-cdc), the released version will be available in the Maven central warehouse.
How to create a Postgres CDC table
----------------

2
docs/content/docs/connectors/legacy-flink-cdc-sources/sqlserver-cdc.md
Unescape Escape View File

@ -43,7 +43,7 @@ In order to setup the SQLServer CDC connector, the following table provides depe
Download [flink-sql-connector-sqlserver-cdc-3.0-SNAPSHOT.jar](https://repo1.maven.org/maven2/org/apache/flink/flink-sql-connector-sqlserver-cdc/3.0-SNAPSHOT/flink-sql-connector-sqlserver-cdc-3.0-SNAPSHOT.jar) and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-sqlserver-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-sqlserver-cdc-2.2.1.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-sqlserver-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-sqlserver-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users should use the released version, such as [flink-sql-connector-sqlserver-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-sqlserver-cdc), the released version will be available in the Maven central warehouse.
Setup SQLServer Database
----------------

2
docs/content/docs/connectors/legacy-flink-cdc-sources/tidb-cdc.md
Unescape Escape View File

@ -43,7 +43,7 @@ In order to setup the TiDB CDC connector, the following table provides dependenc
Download [flink-sql-connector-tidb-cdc-3.0-SNAPSHOT.jar](https://repo1.maven.org/maven2/org/apache/flink/flink-sql-connector-tidb-cdc/3.0-SNAPSHOT/flink-sql-connector-tidb-cdc-3.0-SNAPSHOT.jar) and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-tidb-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-tidb-cdc-2.2.1.jar](https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-tidb-cdc), the released version will be available in the Maven central warehouse.
**Note:** flink-sql-connector-tidb-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-tidb-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-tidb-cdc), the released version will be available in the Maven central warehouse.
How to create a TiDB CDC table
----------------

4
docs/content/docs/connectors/legacy-flink-cdc-sources/vitess-cdc.md
Unescape Escape View File

@ -42,6 +42,10 @@ In order to setup the Vitess CDC connector, the following table provides depende
Download [flink-sql-connector-vitess-cdc-3.0-SNAPSHOT.jar](https://repo1.maven.org/maven2/org/apache/flink/flink-sql-connector-vitess-cdc/3.0-SNAPSHOT/flink-sql-connector-vitess-cdc-3.0-SNAPSHOT.jar) and put it under `<FLINK_HOME>/lib/`.
**Note:** flink-sql-connector-vitess-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users should use the released version, such as
[flink-sql-connector-vitess-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-connector-vitess-cdc),
the released version will be available in the Maven central warehouse.
Setup Vitess server
----------------

Write Preview
Loading…
Cancel
Save