github-forks
/
flink-cdc-connectors
mirror of https://github.com/ververica/flink-cdc-connectors.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[minor][docs] Rectify names of CDC sources for Flink and improve the content directory

Browse Source 
This closes #3316.
pull/3353/head
yux 8 months ago committed by GitHub
parent 4c0cafe9f4
commit 71414d8f7d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
66 changed files with 243 additions and 203 deletions
Show Stats Download Patch File Download Diff File

2
README.md
Unescape Escape View File

@ -63,7 +63,7 @@ full database synchronization, sharding table synchronization, schema evolution
5. View job execution status through Flink WebUI or downstream database.
Try it out yourself with our more detailed [tutorial](docs/content/docs/get-started/quickstart/mysql-to-doris.md).
You can also see [connector overview](docs/content/docs/connectors/overview.md) to view a comprehensive catalog of the
You can also see [connector overview](docs/content/docs/connectors/pipeline-connectors/overview.md) to view a comprehensive catalog of the
connectors currently provided and understand more detailed configurations.

4
docs/content/docs/connectors/legacy-flink-cdc-sources/_index.md → docs/content.zh/docs/connectors/flink-sources/_index.md
Unescape Escape View File

@ -1,7 +1,7 @@
---
title: Legacy Flink CDC Sources
title: Flink Source 连接器
bookCollapseSection: true
weight: 5
weight: 2
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/datastream-api-package-guidance.md → docs/content.zh/docs/connectors/flink-sources/datastream-api-package-guidance.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "DataStream API 打包指南"
weight: 998
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/datastream-api-package-guidance
- /connectors/flink-sources/datastream-api-package-guidance
---
<!--

4
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/db2-cdc.md → docs/content.zh/docs/connectors/flink-sources/db2-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "Db2"
weight: 9
weight: 7
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/db2-cdc
- /connectors/flink-sources/db2-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

4
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/mongodb-cdc.md → docs/content.zh/docs/connectors/flink-sources/mongodb-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "MongoDB"
weight: 2
weight: 6
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/mongodb-cdc
- /connectors/flink-sources/mongodb-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

4
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/mysql-cdc.md → docs/content.zh/docs/connectors/flink-sources/mysql-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "MySQL"
weight: 7
weight: 2
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/mysql-cdc
- /connectors/flink-sources/mysql-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

6
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/oceanbase-cdc.md → docs/content.zh/docs/connectors/flink-sources/oceanbase-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "OceanBase"
weight: 4
weight: 9
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/oceanbase-cdc
- /connectors/flink-sources/oceanbase-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
@ -234,7 +234,7 @@ Flink SQL> CREATE TABLE orders (
);
```
您也可以访问 Flink CDC 官网文档，快速体验将数据从 OceanBase 导入到 Elasticsearch。更多信息，参考 [Flink CDC 官网文档](https://nightlies.apache.org/flink/flink-cdc-docs-release-3.0/docs/connectors/legacy-flink-cdc-sources/tutorials/oceanbase-tutorial/)。
您也可以访问 Flink CDC 官网文档，快速体验将数据从 OceanBase 导入到 Elasticsearch。更多信息，参考 [Flink CDC 官网文档]({{< ref "docs/connectors/flink-sources/tutorials/oceanbase-tutorial" >}})。
OceanBase CDC 连接器选项
----------------

4
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/oracle-cdc.md → docs/content.zh/docs/connectors/flink-sources/oracle-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "Oracle"
weight: 5
weight: 3
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/oracle-cdc
- /connectors/flink-sources/oracle-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

54
docs/content/docs/connectors/legacy-flink-cdc-sources/overview.md → docs/content.zh/docs/connectors/flink-sources/overview.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "Overview"
title: "概览"
weight: 1
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/
- /connectors/flink-sources/
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
@ -24,28 +24,28 @@ specific language governing permissions and limitations
under the License.
-->
# Legacy CDC sources
# Flink Sources 连接器
Legacy CDC sources is a set of source connectors for <a href="https://flink.apache.org/">Apache Flink<sup>®</sup></a>, ingesting changes from different databases using change data capture (CDC).
Flink CDC sources is a set of source connectors for <a href="https://flink.apache.org/">Apache Flink<sup>®</sup></a>, ingesting changes from different databases using change data capture (CDC).
Some CDC sources integrate Debezium as the engine to capture data changes. So it can fully leverage the ability of Debezium. See more about what is [Debezium](https://github.com/debezium/debezium).
You can also read [tutorials]({{< ref "docs/connectors/legacy-flink-cdc-sources/tutorials/build-streaming-etl-tutorial" >}}) about how to use these sources.
You can also read [tutorials]({{< ref "docs/connectors/flink-sources/tutorials/build-streaming-etl-tutorial" >}}) about how to use these sources.
{{< img src="/fig/cdc-flow.png" width="600px" alt="Flink CDC" >}}
## Supported Connectors
| Connector | Database | Driver |
|---------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------|
| [mongodb-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/mongodb-cdc" >}}) | <li> [MongoDB](https://www.mongodb.com): 3.6, 4.x, 5.0 | MongoDB Driver: 4.3.4 |
| [mysql-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/mysql-cdc" >}}) | <li> [MySQL](https://dev.mysql.com/doc" >}}): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.28 |
| [oceanbase-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/oceanbase-cdc" >}}) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x, 4.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase): 2.x, 3.x, 4.x | OceanBase Driver: 2.4.x |
| [oracle-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/oracle-cdc" >}}) | <li> [Oracle](https://www.oracle.com/index.html): 11, 12, 19, 21 | Oracle Driver: 19.3.0.0 |
| [postgres-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/postgres-cdc" >}}) | <li> [PostgreSQL](https://www.postgresql.org): 9.6, 10, 11, 12, 13, 14 | JDBC Driver: 42.5.1 |
| [sqlserver-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/sqlserver-cdc" >}}) | <li> [Sqlserver](https://www.microsoft.com/sql-server): 2012, 2014, 2016, 2017, 2019 | JDBC Driver: 9.4.1.jre8 |
| [tidb-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/tidb-cdc" >}}) | <li> [TiDB](https://www.pingcap.com/): 5.1.x, 5.2.x, 5.3.x, 5.4.x, 6.0.0 | JDBC Driver: 8.0.27 |
| [db2-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/db2-cdc" >}}) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
| [vitess-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/vitess-cdc" >}}) | <li> [Vitess](https://vitess.io/): 8.0.x, 9.0.x | MySql JDBC Driver: 8.0.26 |
| Connector | Database | Driver |
|----------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------|
| [mongodb-cdc]({{< ref "docs/connectors/flink-sources/mongodb-cdc" >}}) | <li> [MongoDB](https://www.mongodb.com): 3.6, 4.x, 5.0 | MongoDB Driver: 4.3.4 |
| [mysql-cdc]({{< ref "docs/connectors/flink-sources/mysql-cdc" >}}) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.28 |
| [oceanbase-cdc]({{< ref "docs/connectors/flink-sources/oceanbase-cdc" >}}) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x, 4.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase): 2.x, 3.x, 4.x | OceanBase Driver: 2.4.x |
| [oracle-cdc]({{< ref "docs/connectors/flink-sources/oracle-cdc" >}}) | <li> [Oracle](https://www.oracle.com/index.html): 11, 12, 19, 21 | Oracle Driver: 19.3.0.0 |
| [postgres-cdc]({{< ref "docs/connectors/flink-sources/postgres-cdc" >}}) | <li> [PostgreSQL](https://www.postgresql.org): 9.6, 10, 11, 12, 13, 14 | JDBC Driver: 42.5.1 |
| [sqlserver-cdc]({{< ref "docs/connectors/flink-sources/sqlserver-cdc" >}}) | <li> [Sqlserver](https://www.microsoft.com/sql-server): 2012, 2014, 2016, 2017, 2019 | JDBC Driver: 9.4.1.jre8 |
| [tidb-cdc]({{< ref "docs/connectors/flink-sources/tidb-cdc" >}}) | <li> [TiDB](https://www.pingcap.com/): 5.1.x, 5.2.x, 5.3.x, 5.4.x, 6.0.0 | JDBC Driver: 8.0.27 |
| [db2-cdc]({{< ref "docs/connectors/flink-sources/db2-cdc" >}}) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
| [vitess-cdc]({{< ref "docs/connectors/flink-sources/vitess-cdc" >}}) | <li> [Vitess](https://vitess.io/): 8.0.x, 9.0.x | MySql JDBC Driver: 8.0.26 |
## Supported Flink Versions
The following table shows the version mapping between Flink<sup>®</sup> CDC Connectors and Flink<sup>®</sup>:
@ -72,17 +72,17 @@ The following table shows the version mapping between Flink<sup>®</sup> CDC Con
The following table shows the current features of the connector:
| Connector | No-lock Read | Parallel Read | Exactly-once Read | Incremental Snapshot Read |
|---------------------------------------------------------------------------------------|--------------|---------------|-------------------|---------------------------|
| [mongodb-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/mongodb-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [mysql-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/mysql-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [oracle-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/oracle-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [postgres-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/postgres-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [sqlserver-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/sqlserver-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [oceanbase-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/oceanbase-cdc" >}}) | ❌ | ❌ | ❌ | ❌ |
| [tidb-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/tidb-cdc" >}}) | ✅ | ❌ | ✅ | ❌ |
| [db2-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/db2-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [vitess-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/vitess-cdc" >}}) | ✅ | ❌ | ✅ | ❌ |
| Connector | No-lock Read | Parallel Read | Exactly-once Read | Incremental Snapshot Read |
|----------------------------------------------------------------------------|--------------|---------------|-------------------|---------------------------|
| [mongodb-cdc]({{< ref "docs/connectors/flink-sources/mongodb-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [mysql-cdc]({{< ref "docs/connectors/flink-sources/mysql-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [oracle-cdc]({{< ref "docs/connectors/flink-sources/oracle-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [postgres-cdc]({{< ref "docs/connectors/flink-sources/postgres-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [sqlserver-cdc]({{< ref "docs/connectors/flink-sources/sqlserver-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [oceanbase-cdc]({{< ref "docs/connectors/flink-sources/oceanbase-cdc" >}}) | ❌ | ❌ | ❌ | ❌ |
| [tidb-cdc]({{< ref "docs/connectors/flink-sources/tidb-cdc" >}}) | ✅ | ❌ | ✅ | ❌ |
| [db2-cdc]({{< ref "docs/connectors/flink-sources/db2-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [vitess-cdc]({{< ref "docs/connectors/flink-sources/vitess-cdc" >}}) | ✅ | ❌ | ✅ | ❌ |
## Usage for Table/SQL API

4
docs/content/docs/connectors/legacy-flink-cdc-sources/postgres-cdc.md → docs/content.zh/docs/connectors/flink-sources/postgres-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "Postgres"
weight: 6
weight: 5
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/postgres-cdc
- /connectors/flink-sources/postgres-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

4
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/sqlserver-cdc.md → docs/content.zh/docs/connectors/flink-sources/sqlserver-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "SQL Server"
weight: 7
weight: 4
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/sqlserver-cdc
- /connectors/flink-sources/sqlserver-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/tidb-cdc.md → docs/content.zh/docs/connectors/flink-sources/tidb-cdc.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "TiDB"
weight: 8
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tidb-cdc
- /connectors/flink-sources/tidb-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/tutorials/_index.md → docs/content.zh/docs/connectors/flink-sources/tutorials/_index.md
Unescape Escape View File

@ -1,5 +1,5 @@
---
title: "Legacy CDC Sources 教程"
title: "Flink CDC Sources 教程"
bookCollapseSection: true
weight: 999
---

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/tutorials/build-real-time-data-lake-tutorial.md → docs/content.zh/docs/connectors/flink-sources/tutorials/build-real-time-data-lake-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "使用 Flink CDC 构建实时数据湖"
weight: 998
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/build-real-time-data-lake-tutorial.html
- /connectors/flink-sources/tutorials/build-real-time-data-lake-tutorial.html
---

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/tutorials/build-streaming-etl-tutorial.md → docs/content.zh/docs/connectors/flink-sources/tutorials/build-streaming-etl-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "使用 Flink CDC 构建 Streaming ETL"
weight: 999
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/mysql-postgres-tutorial.html
- /connectors/flink-sources/tutorials/mysql-postgres-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/tutorials/db2-tutorial.md → docs/content.zh/docs/connectors/flink-sources/tutorials/db2-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "Db2 教程"
weight: 2
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/db2-tutorial.html
- /connectors/flink-sources/tutorials/db2-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/tutorials/mongodb-tutorial.md → docs/content.zh/docs/connectors/flink-sources/tutorials/mongodb-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "MongoDB 教程"
weight: 1
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/mongodb-tutorial.html
- /connectors/flink-sources/tutorials/mongodb-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/tutorials/oceanbase-tutorial.md → docs/content.zh/docs/connectors/flink-sources/tutorials/oceanbase-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "OceanBase 教程"
weight: 3
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/oceanbase-tutorial.html
- /connectors/flink-sources/tutorials/oceanbase-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/tutorials/oracle-tutorial.md → docs/content.zh/docs/connectors/flink-sources/tutorials/oracle-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "Oracle 教程"
weight: 4
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/oracle-tutorial.html
- /connectors/flink-sources/tutorials/oracle-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/tutorials/polardbx-tutorial.md → docs/content.zh/docs/connectors/flink-sources/tutorials/polardbx-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "PolarDB-X 教程"
weight: 5
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/mongodb-tutorial.html
- /connectors/flink-sources/tutorials/mongodb-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/tutorials/sqlserver-tutorial.md → docs/content.zh/docs/connectors/flink-sources/tutorials/sqlserver-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "SqlServer 教程"
weight: 6
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/sqlserver-tutorial.html
- /connectors/flink-sources/tutorials/sqlserver-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/tutorials/tidb-tutorial.md → docs/content.zh/docs/connectors/flink-sources/tutorials/tidb-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "TiDB 教程"
weight: 7
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/tidb-tutorial.html
- /connectors/flink-sources/tutorials/tidb-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/vitess-cdc.md → docs/content.zh/docs/connectors/flink-sources/vitess-cdc.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "Vitess"
weight: 10
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/vitess-cdc
- /connectors/flink-sources/vitess-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

46
docs/content.zh/docs/connectors/overview.md
Unescape Escape View File

@ -1,46 +0,0 @@
---
title: "概览"
weight: 1
type: docs
aliases:
- /connectors/overview
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Connectors
Flink CDC 提供了多个Source和Sink连接器来与外部系统交互。您可以直接使用这些连接器，只需将 JAR 文件添加到您的 Flink CDC 环境中，并在您的 YAML Pipeline定义中指定所需的连接器。
## Supported Connectors
| Connector | Supported Type | External System |
|------------------------------------------------------|----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Apache Doris]({{< ref "docs/connectors/doris" >}}) | Sink | <li> [Apache Doris](https://doris.apache.org/): 1.2.x, 2.x.x |
| [MySQL]({{< ref "docs/connectors/mysql" >}}) | Source | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 |
| [StarRocks]({{< ref "docs/connectors/starrocks" >}}) | Sink | <li> [StarRocks](https://www.starrocks.io/): 2.x, 3.x |
## Develop Your Own Connector
如果现有的连接器无法满足您的需求，您可以自行开发自己的连接器，以将您的外部系统集成到 Flink CDC 数据管道中。查阅 [Flink CDC APIs]({{< ref "docs/developer-guide/understand-flink-cdc-api" >}}) 了解如何开发您自己的连接器。
## Legacy Flink CDC Sources
在 3.0 版本之前引入的 Flink CDC Source 仍然可以作为普通的 Flink 连接器使用。您可以在 [Legacy CDC Sources/概览]({{< ref "docs/connectors/legacy-flink-cdc-sources/overview" >}}) 中找到更多详细信息。
{{< top >}}

4
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/_index.md → docs/content.zh/docs/connectors/pipeline-connectors/_index.md
Unescape Escape View File

@ -1,7 +1,7 @@
---
title: Legacy CDC Sources
title: Pipeline 连接器
bookCollapseSection: true
weight: 5
weight: 1
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

4
docs/content.zh/docs/connectors/doris.md → docs/content.zh/docs/connectors/pipeline-connectors/doris.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "Doris"
weight: 3
weight: 5
type: docs
aliases:
- /connectors/doris
- /connectors/pipeline-connectors/doris
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

4
docs/content.zh/docs/connectors/kafka.md → docs/content.zh/docs/connectors/pipeline-connectors/kafka.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "Kafka"
weight: 5
weight: 4
type: docs
aliases:
- /connectors/kafka
- /connectors/pipeline-connectors/kafka
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content.zh/docs/connectors/mysql.md → docs/content.zh/docs/connectors/pipeline-connectors/mysql.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "MySQL"
weight: 2
type: docs
aliases:
- /connectors/mysql
- /connectors/pipeline-connectors/mysql
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

45
docs/content.zh/docs/connectors/pipeline-connectors/overview.md
Unescape Escape View File

@ -0,0 +1,45 @@
---
title: "概览"
weight: 1
type: docs
aliases:
- /connectors/pipeline-connectors/overview
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Pipeline Connectors
Flink CDC 提供了可用于 YAML 作业的 Pipeline Source 和 Sink 连接器来与外部系统交互。您可以直接使用这些连接器，只需将 JAR 文件添加到您的 Flink CDC 环境中，并在您的 YAML Pipeline 定义中指定所需的连接器。
## Supported Connectors
| 连接器 | 类型 | 支持的外部系统 |
|--------------------------------------------------------------------------|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Apache Doris]({{< ref "docs/connectors/pipeline-connectors/doris" >}}) | Sink | <li> [Apache Doris](https://doris.apache.org/): 1.2.x, 2.x.x |
| [Kafka]({{< ref "docs/connectors/pipeline-connectors/kafka" >}}) | Sink | <li> [Kafka](https://kafka.apache.org/) |
| [MySQL]({{< ref "docs/connectors/pipeline-connectors/mysql" >}}) | Source | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 |
| [Paimon]({{< ref "docs/connectors/pipeline-connectors/paimon" >}}) | Sink | <li> [Paimon](https://paimon.apache.org/): 0.6, 0.7, 0.8 |
| [StarRocks]({{< ref "docs/connectors/pipeline-connectors/starrocks" >}}) | Sink | <li> [StarRocks](https://www.starrocks.io/): 2.x, 3.x |
## Develop Your Own Connector
如果现有的连接器无法满足您的需求，您可以自行开发自己的连接器，以将您的外部系统集成到 Flink CDC 数据管道中。查阅 [Flink CDC APIs]({{< ref "docs/developer-guide/understand-flink-cdc-api" >}}) 了解如何开发您自己的连接器。
{{< top >}}

4
docs/content.zh/docs/connectors/paimon.md → docs/content.zh/docs/connectors/pipeline-connectors/paimon.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "Paimon"
weight: 6
weight: 3
type: docs
aliases:
- /connectors/paimon
- /connectors/pipeline-connectors/paimon
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

4
docs/content.zh/docs/connectors/starrocks.md → docs/content.zh/docs/connectors/pipeline-connectors/starrocks.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "StarRocks"
weight: 4
weight: 6
type: docs
aliases:
- /connectors/starrocks
- /connectors/pipeline-connectors/starrocks
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

6
docs/content.zh/docs/deployment/kubernetes.md
Unescape Escape View File

@ -48,7 +48,7 @@ If you have problems setting up a Kubernetes cluster, please take a look at [how
## Session Mode
Flink runs on all UNIX-like environments, i.e. Linux, Mac OS X, and Cygwin (for Windows).
You can refer [overview]({{< ref "docs/connectors/overview" >}}) to check supported versions and download [the binary release](https://flink.apache.org/downloads/) of Flink,
You can refer [overview]({{< ref "docs/connectors/pipeline-connectors/overview" >}}) to check supported versions and download [the binary release](https://flink.apache.org/downloads/) of Flink,
then extract the archive:
```bash
@ -134,8 +134,8 @@ pipeline:
```
You need to modify the configuration file according to your needs, refer to connectors more information.
- [MySQL pipeline connector]({{< ref "docs/connectors/mysql.md" >}})
- [Apache Doris pipeline connector]({{< ref "docs/connectors/doris.md" >}})
- [MySQL pipeline connector]({{< ref "docs/connectors/pipeline-connectors/mysql.md" >}})
- [Apache Doris pipeline connector]({{< ref "docs/connectors/pipeline-connectors/doris.md" >}})
Finally, submit job to Flink Standalone cluster using Cli.

6
docs/content.zh/docs/deployment/standalone.md
Unescape Escape View File

@ -32,7 +32,7 @@ You will also run an example Flink CDC job and view it in the web UI.
## Preparation
Flink runs on all UNIX-like environments, i.e. Linux, Mac OS X, and Cygwin (for Windows).
You can refer [overview]({{< ref "docs/connectors/overview" >}}) to check supported versions and download [the binary release](https://flink.apache.org/downloads/) of Flink,
You can refer [overview]({{< ref "docs/connectors/pipeline-connectors/overview" >}}) to check supported versions and download [the binary release](https://flink.apache.org/downloads/) of Flink,
then extract the archive:
```bash
@ -111,8 +111,8 @@ pipeline:
```
You need to modify the configuration file according to your needs, refer to connectors more information.
- [MySQL pipeline connector]({{< ref "docs/connectors/mysql.md" >}})
- [Apache Doris pipeline connector]({{< ref "docs/connectors/doris.md" >}})
- [MySQL pipeline connector]({{< ref "docs/connectors/pipeline-connectors/mysql.md" >}})
- [Apache Doris pipeline connector]({{< ref "docs/connectors/pipeline-connectors/doris.md" >}})
Finally, submit job to Flink Standalone cluster using Cli.

2
docs/content.zh/docs/deployment/yarn.md
Unescape Escape View File

@ -46,7 +46,7 @@ export HADOOP_CLASSPATH=`hadoop classpath`
## Session Mode
Flink runs on all UNIX-like environments, i.e. Linux, Mac OS X, and Cygwin (for Windows).
You can refer [overview]({{< ref "docs/connectors/overview" >}}) to check supported versions and download [the binary release](https://flink.apache.org/downloads/) of Flink,
You can refer [overview]({{< ref "docs/connectors/pipeline-connectors/overview" >}}) to check supported versions and download [the binary release](https://flink.apache.org/downloads/) of Flink,
then extract the archive:
```bash

23
docs/content/docs/connectors/flink-sources/_index.md
Unescape Escape View File

@ -0,0 +1,23 @@
---
title: Flink Sources
bookCollapseSection: true
weight: 2
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

2
docs/content/docs/connectors/legacy-flink-cdc-sources/datastream-api-package-guidance.md → docs/content/docs/connectors/flink-sources/datastream-api-package-guidance.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "DataStream API Package Guidance"
weight: 998
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/datastream-api-package-guidance
- /connectors/flink-sources/datastream-api-package-guidance
---
<!--

4
docs/content/docs/connectors/legacy-flink-cdc-sources/db2-cdc.md → docs/content/docs/connectors/flink-sources/db2-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "Db2"
weight: 9
weight: 7
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/db2-cdc
- /connectors/flink-sources/db2-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

4
docs/content/docs/connectors/legacy-flink-cdc-sources/mongodb-cdc.md → docs/content/docs/connectors/flink-sources/mongodb-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "MongoDB"
weight: 2
weight: 6
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/mongodb-cdc
- /connectors/flink-sources/mongodb-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

4
docs/content/docs/connectors/legacy-flink-cdc-sources/mysql-cdc.md → docs/content/docs/connectors/flink-sources/mysql-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "MySQL"
weight: 7
weight: 2
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/mysql-cdc
- /connectors/flink-sources/mysql-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

6
docs/content/docs/connectors/legacy-flink-cdc-sources/oceanbase-cdc.md → docs/content/docs/connectors/flink-sources/oceanbase-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "OceanBase"
weight: 4
weight: 9
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/oceanbase-cdc
- /connectors/flink-sources/oceanbase-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
@ -236,7 +236,7 @@ Flink SQL> CREATE TABLE orders (
);
```
You can also try the quickstart tutorial that sync data from OceanBase to Elasticsearch, please refer [Flink CDC Tutorial]({{< ref "docs/connectors/legacy-flink-cdc-sources/tutorials/oceanbase-tutorial" >}}) for more information.
You can also try the quickstart tutorial that sync data from OceanBase to Elasticsearch, please refer [Flink CDC Tutorial]({{< ref "docs/connectors/flink-sources/tutorials/oceanbase-tutorial" >}}) for more information.
Connector Options
----------------

4
docs/content/docs/connectors/legacy-flink-cdc-sources/oracle-cdc.md → docs/content/docs/connectors/flink-sources/oracle-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "Oracle"
weight: 5
weight: 3
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/oracle-cdc
- /connectors/flink-sources/oracle-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

50
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/overview.md → docs/content/docs/connectors/flink-sources/overview.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "概览"
title: "Overview"
weight: 1
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/
- /connectors/flink-sources/overview
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
@ -24,28 +24,28 @@ specific language governing permissions and limitations
under the License.
-->
# Legacy CDC sources
# Flink CDC sources
Legacy CDC sources is a set of source connectors for <a href="https://flink.apache.org/">Apache Flink<sup>®</sup></a>, ingesting changes from different databases using change data capture (CDC).
Flink CDC sources is a set of source connectors for <a href="https://flink.apache.org/">Apache Flink<sup>®</sup></a>, ingesting changes from different databases using change data capture (CDC).
Some CDC sources integrate Debezium as the engine to capture data changes. So it can fully leverage the ability of Debezium. See more about what is [Debezium](https://github.com/debezium/debezium).
You can also read [tutorials]({{< ref "docs/connectors/legacy-flink-cdc-sources/tutorials/build-streaming-etl-tutorial" >}}) about how to use these sources.
You can also read [tutorials]({{< ref "docs/connectors/flink-sources/tutorials/build-streaming-etl-tutorial" >}}) about how to use these sources.
{{< img src="/fig/cdc-flow.png" width="600px" alt="Flink CDC" >}}
## Supported Connectors
| Connector | Database | Driver |
|---------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------|
| [mongodb-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/mongodb-cdc" >}}) | <li> [MongoDB](https://www.mongodb.com): 3.6, 4.x, 5.0 | MongoDB Driver: 4.3.4 |
| [mysql-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/mysql-cdc" >}}) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.28 |
| [oceanbase-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/oceanbase-cdc" >}}) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x, 4.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase): 2.x, 3.x, 4.x | OceanBase Driver: 2.4.x |
| [oracle-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/oracle-cdc" >}}) | <li> [Oracle](https://www.oracle.com/index.html): 11, 12, 19, 21 | Oracle Driver: 19.3.0.0 |
| [postgres-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/postgres-cdc" >}}) | <li> [PostgreSQL](https://www.postgresql.org): 9.6, 10, 11, 12, 13, 14 | JDBC Driver: 42.5.1 |
| [sqlserver-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/sqlserver-cdc" >}}) | <li> [Sqlserver](https://www.microsoft.com/sql-server): 2012, 2014, 2016, 2017, 2019 | JDBC Driver: 9.4.1.jre8 |
| [tidb-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/tidb-cdc" >}}) | <li> [TiDB](https://www.pingcap.com/): 5.1.x, 5.2.x, 5.3.x, 5.4.x, 6.0.0 | JDBC Driver: 8.0.27 |
| [db2-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/db2-cdc" >}}) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
| [vitess-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/vitess-cdc" >}}) | <li> [Vitess](https://vitess.io/): 8.0.x, 9.0.x | MySql JDBC Driver: 8.0.26 |
| Connector | Database | Driver |
|----------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------|
| [mongodb-cdc]({{< ref "docs/connectors/flink-sources/mongodb-cdc" >}}) | <li> [MongoDB](https://www.mongodb.com): 3.6, 4.x, 5.0 | MongoDB Driver: 4.3.4 |
| [mysql-cdc]({{< ref "docs/connectors/flink-sources/mysql-cdc" >}}) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.28 |
| [oceanbase-cdc]({{< ref "docs/connectors/flink-sources/oceanbase-cdc" >}}) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x, 4.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase): 2.x, 3.x, 4.x | OceanBase Driver: 2.4.x |
| [oracle-cdc]({{< ref "docs/connectors/flink-sources/oracle-cdc" >}}) | <li> [Oracle](https://www.oracle.com/index.html): 11, 12, 19, 21 | Oracle Driver: 19.3.0.0 |
| [postgres-cdc]({{< ref "docs/connectors/flink-sources/postgres-cdc" >}}) | <li> [PostgreSQL](https://www.postgresql.org): 9.6, 10, 11, 12, 13, 14 | JDBC Driver: 42.5.1 |
| [sqlserver-cdc]({{< ref "docs/connectors/flink-sources/sqlserver-cdc" >}}) | <li> [Sqlserver](https://www.microsoft.com/sql-server): 2012, 2014, 2016, 2017, 2019 | JDBC Driver: 9.4.1.jre8 |
| [tidb-cdc]({{< ref "docs/connectors/flink-sources/tidb-cdc" >}}) | <li> [TiDB](https://www.pingcap.com/): 5.1.x, 5.2.x, 5.3.x, 5.4.x, 6.0.0 | JDBC Driver: 8.0.27 |
| [db2-cdc]({{< ref "docs/connectors/flink-sources/db2-cdc" >}}) | <li> [Db2](https://www.ibm.com/products/db2): 11.5 | Db2 Driver: 11.5.0.0 |
| [vitess-cdc]({{< ref "docs/connectors/flink-sources/vitess-cdc" >}}) | <li> [Vitess](https://vitess.io/): 8.0.x, 9.0.x | MySql JDBC Driver: 8.0.26 |
## Supported Flink Versions
The following table shows the version mapping between Flink<sup>®</sup> CDC Connectors and Flink<sup>®</sup>:
@ -74,15 +74,15 @@ The following table shows the current features of the connector:
| Connector | No-lock Read | Parallel Read | Exactly-once Read | Incremental Snapshot Read |
|---------------------------------------------------------------------------------------|--------------|---------------|-------------------|---------------------------|
| [mongodb-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/mongodb-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [mysql-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/mysql-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [oracle-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/oracle-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [postgres-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/postgres-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [sqlserver-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/sqlserver-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [oceanbase-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/oceanbase-cdc" >}}) | ❌ | ❌ | ❌ | ❌ |
| [tidb-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/tidb-cdc" >}}) | ✅ | ❌ | ✅ | ❌ |
| [db2-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/db2-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [vitess-cdc]({{< ref "docs/connectors/legacy-flink-cdc-sources/vitess-cdc" >}}) | ✅ | ❌ | ✅ | ❌ |
| [mongodb-cdc]({{< ref "docs/connectors/flink-sources/mongodb-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [mysql-cdc]({{< ref "docs/connectors/flink-sources/mysql-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [oracle-cdc]({{< ref "docs/connectors/flink-sources/oracle-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [postgres-cdc]({{< ref "docs/connectors/flink-sources/postgres-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [sqlserver-cdc]({{< ref "docs/connectors/flink-sources/sqlserver-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [oceanbase-cdc]({{< ref "docs/connectors/flink-sources/oceanbase-cdc" >}}) | ❌ | ❌ | ❌ | ❌ |
| [tidb-cdc]({{< ref "docs/connectors/flink-sources/tidb-cdc" >}}) | ✅ | ❌ | ✅ | ❌ |
| [db2-cdc]({{< ref "docs/connectors/flink-sources/db2-cdc" >}}) | ✅ | ✅ | ✅ | ✅ |
| [vitess-cdc]({{< ref "docs/connectors/flink-sources/vitess-cdc" >}}) | ✅ | ❌ | ✅ | ❌ |
## Usage for Table/SQL API

4
docs/content.zh/docs/connectors/legacy-flink-cdc-sources/postgres-cdc.md → docs/content/docs/connectors/flink-sources/postgres-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "Postgres"
weight: 6
weight: 5
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/postgres-cdc
- /connectors/flink-sources/postgres-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

4
docs/content/docs/connectors/legacy-flink-cdc-sources/sqlserver-cdc.md → docs/content/docs/connectors/flink-sources/sqlserver-cdc.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "SQL Server"
weight: 7
weight: 4
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/sqlserver-cdc
- /connectors/flink-sources/sqlserver-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content/docs/connectors/legacy-flink-cdc-sources/tidb-cdc.md → docs/content/docs/connectors/flink-sources/tidb-cdc.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "TiDB"
weight: 8
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tidb-cdc
- /connectors/flink-sources/tidb-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

0
docs/content/docs/connectors/legacy-flink-cdc-sources/tutorials/_index.md → docs/content/docs/connectors/flink-sources/tutorials/_index.md
Unescape Escape View File

2
docs/content/docs/connectors/legacy-flink-cdc-sources/tutorials/build-real-time-data-lake-tutorial.md → docs/content/docs/connectors/flink-sources/tutorials/build-real-time-data-lake-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "Building a Real-time Data Lake with Flink CDC"
weight: 998
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/build-real-time-data-lake-tutorial.html
- /connectors/flink-sources/tutorials/build-real-time-data-lake-tutorial.html
---

2
docs/content/docs/connectors/legacy-flink-cdc-sources/tutorials/build-streaming-etl-tutorial.md → docs/content/docs/connectors/flink-sources/tutorials/build-streaming-etl-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "Building a Streaming ETL with Flink CDC"
weight: 999
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/mysql-postgres-tutorial.html
- /connectors/flink-sources/tutorials/mysql-postgres-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content/docs/connectors/legacy-flink-cdc-sources/tutorials/db2-tutorial.md → docs/content/docs/connectors/flink-sources/tutorials/db2-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "Db2 Tutorial"
weight: 2
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/db2-tutorial.html
- /connectors/flink-sources/tutorials/db2-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content/docs/connectors/legacy-flink-cdc-sources/tutorials/mongodb-tutorial.md → docs/content/docs/connectors/flink-sources/tutorials/mongodb-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "MongoDB Tutorial"
weight: 1
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/mongodb-tutorial.html
- /connectors/flink-sources/tutorials/mongodb-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content/docs/connectors/legacy-flink-cdc-sources/tutorials/oceanbase-tutorial.md → docs/content/docs/connectors/flink-sources/tutorials/oceanbase-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "OceanBase Tutorial"
weight: 3
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/oceanbase-tutorial.html
- /connectors/flink-sources/tutorials/oceanbase-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content/docs/connectors/legacy-flink-cdc-sources/tutorials/oracle-tutorial.md → docs/content/docs/connectors/flink-sources/tutorials/oracle-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "Oracle Tutorial"
weight: 4
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/oracle-tutorial.html
- /connectors/flink-sources/tutorials/oracle-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content/docs/connectors/legacy-flink-cdc-sources/tutorials/polardbx-tutorial.md → docs/content/docs/connectors/flink-sources/tutorials/polardbx-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "PolarDB-X Tutorial"
weight: 5
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/mongodb-tutorial.html
- /connectors/flink-sources/tutorials/mongodb-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content/docs/connectors/legacy-flink-cdc-sources/tutorials/sqlserver-tutorial.md → docs/content/docs/connectors/flink-sources/tutorials/sqlserver-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "SqlServer Tutorial"
weight: 6
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/sqlserver-tutorial.html
- /connectors/flink-sources/tutorials/sqlserver-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content/docs/connectors/legacy-flink-cdc-sources/tutorials/tidb-tutorial.md → docs/content/docs/connectors/flink-sources/tutorials/tidb-tutorial.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "TiDB Tutorial"
weight: 7
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/tidb-tutorial.html
- /connectors/flink-sources/tutorials/tidb-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content/docs/connectors/legacy-flink-cdc-sources/vitess-cdc.md → docs/content/docs/connectors/flink-sources/vitess-cdc.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "Vitess"
weight: 10
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/vitess-cdc
- /connectors/flink-sources/vitess-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

23
docs/content/docs/connectors/pipeline-connectors/_index.md
Unescape Escape View File

@ -0,0 +1,23 @@
---
title: Pipeline Connectors
bookCollapseSection: true
weight: 1
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

4
docs/content/docs/connectors/doris.md → docs/content/docs/connectors/pipeline-connectors/doris.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "Doris"
weight: 3
weight: 5
type: docs
aliases:
- /connectors/doris
- /connectors/pipeline-connectors/doris
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

4
docs/content/docs/connectors/kafka.md → docs/content/docs/connectors/pipeline-connectors/kafka.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "Kafka"
weight: 5
weight: 4
type: docs
aliases:
- /connectors/kafka
- /connectors/pipeline-connectors/kafka
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

2
docs/content/docs/connectors/mysql.md → docs/content/docs/connectors/pipeline-connectors/mysql.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "MySQL"
weight: 2
type: docs
aliases:
- /connectors/mysql
- /connectors/pipeline-connectors/mysql
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

21
docs/content/docs/connectors/overview.md → docs/content/docs/connectors/pipeline-connectors/overview.md
Unescape Escape View File

@ -3,7 +3,7 @@ title: "Overview"
weight: 1
type: docs
aliases:
- /connectors/overview
- /connectors/pipeline-connectors/overview
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
@ -33,11 +33,13 @@ definition.
## Supported Connectors
| Connector | Supported Type | External System |
|------------------------------------------------------|----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Apache Doris]({{< ref "docs/connectors/doris" >}}) | Sink | <li> [Apache Doris](https://doris.apache.org/): 1.2.x, 2.x.x |
| [MySQL]({{< ref "docs/connectors/mysql" >}}) | Source | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 |
| [StarRocks]({{< ref "docs/connectors/starrocks" >}}) | Sink | <li> [StarRocks](https://www.starrocks.io/): 2.x, 3.x |
| Connector | Supported Type | External System |
|--------------------------------------------------------------------------|----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Apache Doris]({{< ref "docs/connectors/pipeline-connectors/doris" >}}) | Sink | <li> [Apache Doris](https://doris.apache.org/): 1.2.x, 2.x.x |
| [Kafka]({{< ref "docs/connectors/pipeline-connectors/kafka" >}}) | Sink | <li> [Kafka](https://kafka.apache.org/) |
| [MySQL]({{< ref "docs/connectors/pipeline-connectors/mysql" >}}) | Source | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 |
| [Paimon]({{< ref "docs/connectors/pipeline-connectors/paimon" >}}) | Sink | <li> [Paimon](https://paimon.apache.org/): 0.6, 0.7, 0.8 |
| [StarRocks]({{< ref "docs/connectors/pipeline-connectors/starrocks" >}}) | Sink | <li> [StarRocks](https://www.starrocks.io/): 2.x, 3.x |
## Develop Your Own Connector
@ -46,11 +48,4 @@ your own connector to get your external system involved in Flink CDC pipelines.
Check out [Flink CDC APIs]({{< ref "docs/developer-guide/understand-flink-cdc-api" >}})
to learn how to develop your own connectors.
## Legacy Flink CDC Sources
Flink CDC sources introduces before 3.0 are still available as normal Flink
connector sources. You can find more details in the
[overview page]({{< ref "docs/connectors/legacy-flink-cdc-sources/overview" >}})
of legacy Flink CDC sources.
{{< top >}}

4
docs/content/docs/connectors/paimon.md → docs/content/docs/connectors/pipeline-connectors/paimon.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "Paimon"
weight: 6
weight: 3
type: docs
aliases:
- /connectors/paimon
- /connectors/pipeline-connectors/paimon
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

4
docs/content/docs/connectors/starrocks.md → docs/content/docs/connectors/pipeline-connectors/starrocks.md
Unescape Escape View File

@ -1,9 +1,9 @@
---
title: "StarRocks"
weight: 4
weight: 6
type: docs
aliases:
- /connectors/starrocks
- /connectors/pipeline-connectors/starrocks
---
<!--
Licensed to the Apache Software Foundation (ASF) under one

6
docs/content/docs/deployment/kubernetes.md
Unescape Escape View File

@ -48,7 +48,7 @@ If you have problems setting up a Kubernetes cluster, please take a look at [how
## Session Mode
Flink runs on all UNIX-like environments, i.e. Linux, Mac OS X, and Cygwin (for Windows).
You can refer [overview]({{< ref "docs/connectors/overview" >}}) to check supported versions and download [the binary release](https://flink.apache.org/downloads/) of Flink,
You can refer [overview]({{< ref "docs/connectors/pipeline-connectors/overview" >}}) to check supported versions and download [the binary release](https://flink.apache.org/downloads/) of Flink,
then extract the archive:
```bash
@ -134,8 +134,8 @@ pipeline:
```
You need to modify the configuration file according to your needs, refer to connectors more information.
- [MySQL pipeline connector]({{< ref "docs/connectors/mysql.md" >}})
- [Apache Doris pipeline connector]({{< ref "docs/connectors/doris.md" >}})
- [MySQL pipeline connector]({{< ref "docs/connectors/pipeline-connectors/mysql.md" >}})
- [Apache Doris pipeline connector]({{< ref "docs/connectors/pipeline-connectors/doris.md" >}})
Finally, submit job to Flink Standalone cluster using Cli.

6
docs/content/docs/deployment/standalone.md
Unescape Escape View File

@ -32,7 +32,7 @@ You will also run an example Flink CDC job and view it in the web UI.
## Preparation
Flink runs on all UNIX-like environments, i.e. Linux, Mac OS X, and Cygwin (for Windows).
You can refer [overview]({{< ref "docs/connectors/overview" >}}) to check supported versions and download [the binary release](https://flink.apache.org/downloads/) of Flink,
You can refer [overview]({{< ref "docs/connectors/pipeline-connectors/overview" >}}) to check supported versions and download [the binary release](https://flink.apache.org/downloads/) of Flink,
then extract the archive:
```bash
@ -111,8 +111,8 @@ pipeline:
```
You need to modify the configuration file according to your needs, refer to connectors more information.
- [MySQL pipeline connector]({{< ref "docs/connectors/mysql.md" >}})
- [Apache Doris pipeline connector]({{< ref "docs/connectors/doris.md" >}})
- [MySQL pipeline connector]({{< ref "docs/connectors/pipeline-connectors/mysql.md" >}})
- [Apache Doris pipeline connector]({{< ref "docs/connectors/pipeline-connectors/doris.md" >}})
Finally, submit job to Flink Standalone cluster using Cli.

2
docs/content/docs/deployment/yarn.md
Unescape Escape View File

@ -46,7 +46,7 @@ export HADOOP_CLASSPATH=`hadoop classpath`
## Session Mode
Flink runs on all UNIX-like environments, i.e. Linux, Mac OS X, and Cygwin (for Windows).
You can refer [overview]({{< ref "docs/connectors/overview" >}}) to check supported versions and download [the binary release](https://flink.apache.org/downloads/) of Flink,
You can refer [overview]({{< ref "docs/connectors/pipeline-connectors/overview" >}}) to check supported versions and download [the binary release](https://flink.apache.org/downloads/) of Flink,
then extract the archive:
```bash

Write Preview
Loading…
Cancel
Save