Commit Graph

1072 Commits (3cb91fb51acd3228a0cb39fe514bc7c23a4883b5)
 

Author SHA1 Message Date
yuxiqian 3cb91fb51a
[FLINK-35805][transform] Add __data_event_type__ metadata column
This closes  #3468
6 months ago
yuxiqian 874ff4ff20
[hotfix][cdc-runtime] Invalidate cache correctly to avoid classloader leakage
This closes #3533
6 months ago
Qingsheng Ren ed832297a8
[release] Update version to 3.3-SNAPSHOT
This closes #3531.

Co-authored-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>
6 months ago
yuxiqian 104209558a [FLINK-36034][cdc-runtime] Get rid of Flink table planner dependency in cdc runtime module
This closes #3513.
6 months ago
yuxiqian 2885d12f57 [FLINK-35991][cdc-runtime] Resolve operator conflicts in transform SQL operator tables 6 months ago
wuzexian 8137f9d746 [FLINK-35894][pipeline-connector][es] Support for ElasticSearch 6, 7 versions
This closes #3495.
6 months ago
wuzexian ac14dba9fb [FLINK-35894][pipeline-connector][es] Introduce Elasticsearch Sink Connector for Flink CDC Pipeline 6 months ago
Junbo wang 15ab85860b
[FLINK-35884][pipeline-connector][mysql] MySQL pipeline connector supports to set chunk key-column (#3490)
Co-authored-by: wangjunbo <wangjunbo@qiyi.com>
Co-authored-by: Hang Ruan <ruanhang1993@hotmail.com>
6 months ago
Kunni 64f996b504
[FLINK-35891][pipeline-connector][paimon] Support dynamic bucket in Paimon sink
This closes  #3499.
6 months ago
MOBIN 17d21a957a
[FLINK-35984][cdc-runtime] Fix bug that metadata column name can not be used in transform rule
This closes  #3528.

Co-authored-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>
6 months ago
Hongshun Wang 5284df571f
[FLINK-36023][cdc-composer] Flink CDC K8S Native Application Mode add wrong jar url (#3523) 6 months ago
Kunni 3bb1f3ca45
[FLINK-35938][pipeline-connector][paimon] Use filterAndCommit API for retry Committable to avoid duplicate commit
This closes #3504.
6 months ago
gongzhongqiang b9c628d310
[hotfix][doc] Fix doris document dead links and typo
This closes #3527
6 months ago
MOBIN 84ef9d5daa
[FLINK-35981][cdc-runtime] Transform supports referencing one column more than once
This closes  #3515.
6 months ago
Hongshun Wang b361db58ab
[FLINK-36007][[cdc-composer] Loading factory and added jar in one search
This close #3520.
6 months ago
yuxiqian e2bb917ad0
[FLINK-34876][transform] Support UDF functions in transform (#3465) 6 months ago
Kunni e0d6d1d1a8
[FLINK-35442][cdc-connect][kafka] add key.format and partition.strategy option to make sure the same record sending to the same partition. (#3522) 6 months ago
Hongshun Wang 4bf5a395a5
[FLINK-34688][cdc-connector][mysql] Make scan newly table trigger condition strictly
This closes #3519.
6 months ago
Hang Ruan 07446d1f9d
[FLINK-35715][common] Ignore the compact optimize for mysql timestamp type in BinaryRecordData (#3511) 6 months ago
North Lin 09f36a4a05
[FLINK-35143][pipeline-connector][mysql] Expose newly added tables capture in mysql pipeline connector. (#3411)
Co-authored-by: Muhammet Orazov <916295+morazow@users.noreply.github.com>
Co-authored-by: north.lin <north.lin@yunlsp.com>
6 months ago
yuxiqian b937db25d5
[hotfix][cdc-connector][mongodb] Fix unstable test cases for snapshot back-filling (#3506) 6 months ago
ConradJam d6b687b61d
[FLINK-34853] Submit CDC Job To Flink K8S Native Application Mode (#3093) 6 months ago
yuxiqian 81d916fc73
[FLINK-35272][cdc-runtime] Transform supports omitting and renaming computed column
This closes #3285.
6 months ago
Wink 2dabfc0815
[FLINK-35730][cdc-cli] PipelineDefinitionParser supports parsing pipeline def in text format
This closes #3444.
6 months ago
yuxiqian af02ce1bc9
[build][e2e] Separate Pipeline and Source E2e tests and cover flink 1.20 version
This closes #3514.
6 months ago
Kunni 9d6154f123
[FLINK-35791][kafka] Add database and table info of Canal / Debezium json format for Kafka sink (#3461) 6 months ago
hk__lrzy 21fade0e22
[FLINK-35432][pipeline-connector][mysql] Support catch modify event in mysql to send AlterColumnTypeEvent. (#3352)
Co-authored-by: haoke <haoke@bytedance.com>
6 months ago
yuxiqian 5ed9e05bdf
[FLINK-35242][cdc-common][cdc-runtime] Support TRY_EVOLVE and LENIENT schema evolution behavior
This closes #3339.
6 months ago
yuxiqian 54c6a0b207
[hotfix][e2e] Add missing default values field in E2e test
This closes #3516.
6 months ago
Leonard Xu 03a2ae3ca7 [FLINK-35893][cdc-runtime] Write CURRENT_VERSION of TableChangeInfo to state
This closes #2944.
6 months ago
lvyanquan 4561a8a32b [FLINK-34638][cdc-common] Support column with default value 6 months ago
gongzhongqiang f06cc1f31b [hotfix][ci] Clean up disk space 6 months ago
gongzhongqiang 1778ef685a Revert "[hotfix][starrocks] Fix StarRocks FE startup failure due to insufficient disk space available "
This reverts commit 3315be3c8f.
6 months ago
Wink 7ca11359d7
[FLINK-35743][cdc-runtime] Correct the temporal function semantics
This closes #3449.

Co-authored-by: wenmo <32723967+wenmo@users.noreply.github.com>
Co-authored-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>
6 months ago
Hongshun Wang d76beecdb0
[FLINK-35234][minor][cdc-common] Fix potential NPE in ConfigurationUtils
This closes #3255.
6 months ago
yuxiqian 44dafe33bd
[FLINK-35813][cdc-runtime] Do not clear state field in TransformSchemaOperator until operator closed
This closes #3469.
6 months ago
yuxiqian 3315be3c8f
[hotfix][starrocks] Fix StarRocks FE startup failure due to insufficient disk space available
This closes #3508.
6 months ago
lipl f6d1d4810a
[FLINK-35912][cdc-connector] SqlServer CDC doesn't chunk UUID-typed columns correctly (#3497)
* resolve conficts

* polish code to trigger ci

---------

Co-authored-by: Kael <kael@fts.dev>
Co-authored-by: gongzhongqiang <gongzhongqiang@gigacloudtech.com>
6 months ago
Thorne 986f37b307
[FLINK-35674][cdc-connector][mysql]Fix blocking caused by searching for timestamp in binlog file (#3432) 6 months ago
Hongshun Wang 0c49959f67
[FLINK-35524][cdc-base] Clear connections pools when reader exist. (#3388) 6 months ago
Hongshun Wang 47f5660055
[FLINK-35344][cdc-base] Move same code from multiple subclasses to JdbcSourceChunkSplitter (#3319) 6 months ago
yuxiqian 8f2939e913
[hotfix][ci] Migrate to Docker Compose V2 (#3505) 6 months ago
Wink 1388cf9906
[FLINK-34877][cdc] Support type cast conversion in pipeline transform
This closes #3357.
6 months ago
yuxiqian a39959f4b3
[FLINK-35072][doris] Support applying AlterColumnTypeEvent to Doris pipeline sink
This closes  #3473
6 months ago
Leonard Xu 85bfd6ae52
[docs][minor] Add transform piece for pipeline example
This closes  #3496
6 months ago
yuxiqian 5917f78704
[FLINK-35736][test] Add migration test scripts & CI workflows
This closes #3447
6 months ago
Junbo wang b15a226ccd
[FLINK-35391][cdc-connector][paimon] Bump dependency of Paimon Pipeline connector to 0.8.2
This closes #3335

Co-authored-by: wangjunbo <wangjunbo@qiyi.com>
6 months ago
yuxiqian 241eb03a74
[FLINK-35868][cdc-connector][mongodb] Bump dependency version to support MongoDB 7.0
This closes #3489.
6 months ago
yuxiqian 768d528d9c
[minor][test] Add Flink CDC 3.1.1 version to migration test version list
This closes #3426.
6 months ago
SeungMin 91fc7bb506
[FLINK-35740][cdc-connector][mysql] Allow column as chunk key even it's not primary key
This closes #3448.
6 months ago