Commit Graph

289 Commits (master)

Author SHA1 Message Date
Jason Zhang 10f503ffa4
[hotfix][docs][pipeline-connector/maxcompute] Fix maxcompute connector typo in examples section
This closes #3875
1 week ago
Hang Ruan 606c9d20c6
[build] Update version to 3.4-SNAPSHOT and add release-3.3 docs
This closes #3870
1 week ago
Seung-Min Lee a1c9c19d7d [FLINK-36578][pipeline-connector/mysql] Introduce option to unify json type output between snapshot phase and binlog phase
This closes #3658
2 weeks ago
moses 947dffca31
[FLINK-37122][build] Try to be compatible with old flink version 1.18.x
This closes #3859
2 weeks ago
Olivier 7997f51c55
[FLINK-35067][cdc-connector][postgres] Adding metadata 'row_kind' for Postgres CDC Connector.
This closes #3716.

Co-authored-by: Leonard Xu <xbjtdcq@gmail.com>
2 weeks ago
Kunni a16abd5d24
[FLINK-36913][pipeline-connector][kafka] Introduce option to define custom mapping from upstream table id to downstream topic name
This closes #3805
2 weeks ago
Thorne 087195a3a6
[FLINK-36224][docs] Add the version mapping between pipeline connectors and flink
This closes  #3598
2 weeks ago
North Lin 9ad071c1fa
[FLINK-36282][pipeline-connector][cdc-connector][mysql]fix incorrect data type of TINYINT(1) in mysql pipeline connector (#3608)
* [FLINK-36282][pipeline-connector][cdc-connector][mysql]fix incorrect data type of TINYINT(1) in mysql pipeline connector

* reformat code

* Update MySqlPipelineITCase.java

* pass a boolean value instead of Properties

* uodate FAQ

* add a method to get tinyInt1isBit

* add new cdc config `treat-tinyint1-as-boolean`

* Update MySqlChunkSplitter.java

* change param name
2 weeks ago
North Lin b5d9673258
[FLINK-34865][pipeline-connector/mysql] Support sync table and column comment
This closes #3482

Co-authored-by: Leonard Xu <xbjtdcq@gmail.com>
2 weeks ago
wenmo d77cf7fa75 [FLINK-36647][transform] Support Timestampdiff and Timestampadd function in cdc pipeline transform
This closes #3698
2 weeks ago
Wink 551bdf395d
[FLINK-36636][transform] Supports timestamp comparison in cdc pipeline transform
This closes #3677
2 weeks ago
yuxiqian 865e14bfd7
[FLINK-35634][build] Add CdcUp playground CLI scripts
This closes #3605
2 weeks ago
moses 4dc31d8c61
[FLINK-37042][pipeline-connector/maxcompute] Rename maxcompute pipieline connector options to follow flink style
This closes  #3852
2 weeks ago
Wink 2fd03e683e
[FLINK-36865][cdc] Provide UNIX_TIMESTAMP series functions in YAML pipeline
This closes #3819.
3 weeks ago
yuxiqian 39fd00ce49
[FLINK-36610] MySQL CDC supports parsing gh-ost / pt-osc generated schema changes (#3668)
Co-authored-by: MOBIN-F <18814118038@163.com>
3 weeks ago
yuxiqian 486ee6f805
[FLINK-36956][transform] Append NOT NULL attribute for Primary Key columns
This closes  #3815
3 weeks ago
Jason Zhang 92081dfe58
[FLINK-35167][pipeline-connector/maxcompute] Introduce MaxCompute pipeline DataSink
This closes  #3254.
3 weeks ago
yuanoOo 6a345aaa80
[FLINK-34545][pipeline-connector/ob]Add OceanBase pipeline connector to Flink CDC
This closes #3360.
4 weeks ago
Hang Ruan c6cf46b75c
[FLINK-36879][runtime] Support to convert delete as insert in transform (#3804) 1 month ago
yuxiqian c2e301222c
[docs] Update download links to up-to-date cdc version (#3766)
* [docs] Update download links to up-to-date cdc version

Signed-off-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>

* fix: replace params to allow dead link checking to pass

Signed-off-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>

* fix: just don't check any interpolating urls

Signed-off-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>

---------

Signed-off-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>
2 months ago
Kunni 06154e9674
[FLINK-36525][transform] Support for AI Model Integration for Data Processing (#3753) 2 months ago
Leonard Xu c969957f02
[hotfix][docs][pipeline-connector/es] Add missed supported flink versions and Elasticsearch versions
This closes  #3752
2 months ago
Runkang He ba74c7c45f
[FLINK-36681][mysql-cdc][docs] Fix the wrong chunks splitting query in incremental snapshot reading section
This closes  #3703.
3 months ago
Timi 8b8330e245
[FLINK-36678][docs][deploy] Fix the typo in Flink CDC YARN deployment documentation
This closes #3706.
3 months ago
yuxiqian 57a1a7df93
[FLINK-36408][pipeline-connector][mysql] Fix MySQL pipeline connector failed to parse FLOAT type with precision (#3625) 3 months ago
MOBIN 1d2cfd789e
[FLINK-35985][transform] Correct the substring function in transform rule
This closes #3537.
3 months ago
yuxiqian 93c985150c
[FLINK-36221][docs] Add CAST function documentations (#3596) 3 months ago
Junbo wang e692e7b4bc
[FLINK-36052][docs][cdc-pipeline-connector/es] Add flink cdc pipeline elasticsearch connector docs
This closes #3649.

Co-authored-by: wangjunbo <wangjunbo@qiyi.com>
3 months ago
North Lin c42b925251
[FLINK-35544][docs][deploy] Add deployment documentations for Kubernetes Operator
This closes #3392.
3 months ago
yuxiqian a1781f432d
[FLINK-36151][docs] Add schema evolution related docs
This closes #3575.
4 months ago
Kunni 565032e3af
[FLINK-36115][pipeline-connector][mysql] Introduce scan.newly-added-table.enabled option for MySQL Source
This closes  #3560.
5 months ago
He Wang cbb33bb870
[FLINK-35638][OceanBase][test] Refactor OceanBase test cases and remove dependency on host network (#3439) 6 months ago
yuxiqian 3cb91fb51a
[FLINK-35805][transform] Add __data_event_type__ metadata column
This closes  #3468
6 months ago
gongzhongqiang b9c628d310
[hotfix][doc] Fix doris document dead links and typo
This closes #3527
6 months ago
yuxiqian e2bb917ad0
[FLINK-34876][transform] Support UDF functions in transform (#3465) 6 months ago
Kunni e0d6d1d1a8
[FLINK-35442][cdc-connect][kafka] add key.format and partition.strategy option to make sure the same record sending to the same partition. (#3522) 6 months ago
North Lin 09f36a4a05
[FLINK-35143][pipeline-connector][mysql] Expose newly added tables capture in mysql pipeline connector. (#3411)
Co-authored-by: Muhammet Orazov <916295+morazow@users.noreply.github.com>
Co-authored-by: north.lin <north.lin@yunlsp.com>
6 months ago
yuxiqian 81d916fc73
[FLINK-35272][cdc-runtime] Transform supports omitting and renaming computed column
This closes #3285.
6 months ago
Kunni 9d6154f123
[FLINK-35791][kafka] Add database and table info of Canal / Debezium json format for Kafka sink (#3461) 6 months ago
Leonard Xu 85bfd6ae52
[docs][minor] Add transform piece for pipeline example
This closes  #3496
6 months ago
yuxiqian 241eb03a74
[FLINK-35868][cdc-connector][mongodb] Bump dependency version to support MongoDB 7.0
This closes #3489.
6 months ago
SeungMin 91fc7bb506
[FLINK-35740][cdc-connector][mysql] Allow column as chunk key even it's not primary key
This closes #3448.
6 months ago
Kunni fcb4cd8f6c
[FLINK-35871][doc] Add missed "snapshot" mode to mysql connector startup options
This closes #3484.
6 months ago
FangXiangmin ec0becd6fa
[docs][minor] Specify the lib directory of Flink CDC in quick start docs
This closes #3479.

Co-authored-by: fangxiangmin <fangxiangmin>
Co-authored-by: Leonard Xu <leonard@apache.org>
Co-authored-by: yux <34335406+yuxiqian@users.noreply.github.com>
7 months ago
Kunni 274401e81b
[FLINK-35758][doc][cdc-connector][mysql] Add missed scan.startup.timestamp-millis option for mysql connector
This closes  #3453.
7 months ago
skylines a2539f5cfa
[hotfix][docs][postgres] Remove unsupported erroneous example code
This closes #3464
7 months ago
yuxiqian a098ddacba
[FLINK-35781][cli] Provide a default parallelism 1 for pipeline jobs
This closes #3458.
7 months ago
yuxiqian ad1f554c0e
[FLINK-35647][route] Support symbol replacement to enrich routing rules
This closes #3428.

Co-authored-by: 张田 <zhang_tian@inspur.com>
Co-authored-by: yangshuaitong <duguhoney@gmail.com>
7 months ago
ouyangwulin 302a691225
[FLINK-35354] Support host mapping in Flink tikv cdc (#3336)
* [FLINK-35337][cdc] Keep up with the latest version of tikv client

* [FLINK-35354]Support host mapping in Flink tikv cdc

* [FLINK-35354] Add doc for host mapping feature

* [FLINK-35354] fixed annotation import
7 months ago
Laffery 06bf14359a
[docs][minor] Optimize styles of the Flink CDC index page
This closes  #3420
8 months ago