Commit Graph

813 Commits (7d7ba2dc19e95728719ef0dc64a53fe7329bc10c)
 

Author SHA1 Message Date
Kunni 7d7ba2dc19
[hotfix][cdc-connector][mysql] Skip SchemaChangeEvents that were not included in capturedTableFilter 1 year ago
Xin Gong e687349440
[docs][hotfix] Update dependency version from 2.5-SNAPSHOT to 3.0-SNAPSHOT(#2995)
This closes #2989.
1 year ago
skylines b463c1839c
[bp-3.0][hotfix][docs] Change the title to keep the document format uniform (#2889) 1 year ago
PengFei Li 1dc9143e92 [pipeline-connector][starrocks][docs] Fix the pictures do not display (#2860)
(cherry picked from commit f2c0a03269)
1 year ago
wudi c71ff51dde [docs][pipeline-connector][doris] Introduce Doris Pipeline Connector (#2788)
(cherry picked from commit 12836e5560)
1 year ago
PengFei Li 94e88a70bc [docs][pipeline-connector][starrocks] Add documents for StarRocks pipeline connector (#2840)
This close #2840.

(cherry picked from commit 78dc8ba9e7)
1 year ago
Kunni 91bbedb7d2 [3.0][docs] Add tutorial for mysql to starrocks pipeline (#2841)
(cherry picked from commit 8be7bdd25d)
1 year ago
Kunni fbc10a7aed
[3.0][docs] Add tutorial for mysql to doris pipeline. (#2838) 1 year ago
Hongshun Wang 657c93075e
[BP-3.0][docs] Introduce core concepts for CDC 3.0 pipeline (#2790) 1 year ago
Shawn Huang 2abb1f18ef
[3.0][docs] Add CDC pipeline example in README (#2836) (#2819)
* [3.0][docs] fix mysql pipeline connector option in readme (#2834)
1 year ago
Jiabao Sun ea3a34c479
[3.0][docs] Add documents for MySql pipeline connector (#2827) (#2835) 1 year ago
Kunni b18d29a863
[pipeline-connector][starrocks] Fix char/varchar length inconsistency between cdc and starrocks (#2830)
Co-authored-by: PengFei Li <lpengfei2016@gmail.com>
1 year ago
Qingsheng Ren f7dee46b8f
[cdc-cli][cdc-composer] Use "source-table" and "sink-table" as keywords of route (#2825) 1 year ago
gongzhongqiang c43ec5e1e8
[build][hotfix] Remove duplicate dependency (#2826) 1 year ago
Qingsheng Ren 73086c4f37
[cdc-cli][cdc-dist] Support loading global config from FLINK_CDC_HOME (#2822)
This closes #2822.
1 year ago
Qingsheng Ren 9415260a25
[cdc-common][hotfix] Remove redundant prefix of pipeline options (#2821)
This closes #2821.
1 year ago
He Wang 4616092195
[cdc-source-connector][oceanbase][e2e] Add OceanBase e2e test case (#2521)
This closes #981
1 year ago
Qingsheng Ren 0c8f5eaf0a [build] Use flink-shaded-force-shading to force all submodules to generate dependency-reduced-pom.xml
This will erase all properties in dependency-reduced-pom.xml and
pin all properties to a constant value. For example, ${flink.version}
will be replaced by the actual Flink version, which is safer for users
to reference it.
1 year ago
Qingsheng Ren daceb42168 [build] Use ${project.version} instead of ${revision} for dependency version
This is documented in https://maven.apache.org/maven-ci-friendly.html#dependencies
1 year ago
Qingsheng Ren c0ee1c188e [build] Skip maven-deploy-plugin for test modules and flink-cdc-dist 1 year ago
Jiabao Sun 8a08ded410
[cdc-pipeline-connector][mysql] Fix precision problem of BIT type conversion (#2820)
This closes #2820.
1 year ago
Dian Qi f970444efa
[cdc-source-connector][mysql] fix deserialization issue on table metadata binlog event (#2682)
Co-authored-by: EchoLee5 <39044001+EchoLee5@users.noreply.github.com>
1 year ago
Jiabao Sun 7d7c1af553
[pipeline-connector][mysql] Remove unnecessary serverTimeZone in DebeziumEventDeserializationSchema (#2816) 1 year ago
Leonard Xu 18fb0d8b7f
[build][hotfix] Remove useless suppressions. (#2817) 1 year ago
Josh Mahonin 63b39ba22c
[cdc-source-connector][mysql] Fix attempted load of isSuspended in MySqlSplitSerializer::deserializeSplit when cdc version <= 2.2.0 (#2550)
Co-authored-by: Hang Ruan <ruanhang1993@hotmail.com>
1 year ago
Jiabao Sun 8c75db145d
[cdc-pipeline-connector][mysql] Ensure the inference of MEDIUMINT_UNSIGNED type matches INT type in table schema (#2811) 1 year ago
He Wang a10e07fd36
[cdc-connector][oceanbase] Add option for obcdc extra configs (#2543)
This closes #2543.
1 year ago
Leonard Xu 0eae214183
[pipeline-connector][mysql] Enable send schema change by default (#2815)
This closes #2815.
1 year ago
Hang Ruan 30c8254af3
[3.0][cdc-runtime] Remove waitForFlushSuccess field in request handler (#2812) 1 year ago
PengFei Li 33b7c85093 [pipeline-connector][starrocks] Add starrocks pipeline connector (#2765)
This closes #2645.
1 year ago
Kunni 4abd86a500
[pipeline-connector][doris] add doris pipeline connector. (#2810)
Co-authored-by: wudi <676366545@qq.com>
1 year ago
Jiabao Sun 8ca3091805 [cdc-pipeline-connector][mysql] Ensure the precision inference of DECIMAL type matches table schema 1 year ago
Hang Ruan 1766e82d7c [3.0][cdc-common & runtime] Remove useless flink imports 1 year ago
Jiabao Sun 08c5dfed83 [cdc-pipeline-connector][mysql] Add tests for mysql pipeline connector
Co-authored-by: Hang Ruan <ruanhang1993@hotmail.com>
1 year ago
Jiabao Sun 174282d3ef [cdc-pipeline-connector][mysql] Support more mysql types 1 year ago
Hang Ruan ec3d558485 [cdc-pipeleine-connecotr][mysql] Send CreateTableEvent in the mysql pipeline connector 1 year ago
Hang Ruan 10ccbde023 [cdc-pipeline-connector][mysql] Parse the alter statement to generate SchemaChangeEvent 1 year ago
Jiabao Sun d0cce26b76 [cdc-pipeline-connector][mysql] Introduce mysql cdc pipeline DataSource 1 year ago
Qingsheng Ren a328f9895a
[cdc-cli][cdc-composer][cdc-dist] Support submitting job to general Flink environments (#2807)
This closes #2807.
1 year ago
Qingsheng Ren 9ce8450cf2
[cdc-runtime][hotfix] Setup waitFlushSuccess before responding operator to flush. (#2805)
This closes #2805.

There might be a case that sink flushes faster than schema operator requesting release upstream. In that case waitFlushSuccess has not been set when sink has flushed successfully, which will lead to a NullPointerException.
1 year ago
Leonard Xu 95921d5620
[cdc-common] Introduce "pipeline.local-time-zone" config option which help handle time zone well (#2797)
This closes #2797.
1 year ago
Hang Ruan e435004eda
[cdc-common][hotfix] Remove useless flink imports (#2803)
This closes #2803.
1 year ago
Qingsheng Ren cfb3fd206e
[build] Let surefire plugin to keep the full stacktrace on test failure (#2804)
This closes #2804.
1 year ago
Kunni b60322bef7
[pipeline-common] Use column name to check whether the schema contains specific column or not. (#2801)
This closes #2801.
1 year ago
Qingsheng Ren 3bf1428233 [cdc-composer][route][sink] Add name for route and improve name for sink 1 year ago
Leonard Xu a8f4ba8cc6 [cdc-composer][hotfix] Always chain operator by default 1 year ago
Leonard Xu f0c630d6f6 [cdc-composer][sink] Use correct name for sink operator 1 year ago
Qingsheng Ren 4843ab2432
[cdc-runtime] SchemaRegistry should complete the future after making the checkpoint (#2800)
This closes #2800.
1 year ago
Xin Gong 2d9dc018c5
[cdc-connector][sqlserver][tests] Fix UT errors by increasing the wait time after committing SQL (#2799)
This closes #2799.
1 year ago
Qingsheng Ren 173deb5248
[cdc-composer] Add common and runtime JAR when submit (#2795)
This closes #2795.
1 year ago