Commit Graph

101 Commits (master)

Author SHA1 Message Date
Sergei Morozov 5691b7bf97
[FLINK-36406][cdc-runtime] Close MetadataApplier when the job stops
This closes  #3623
2 weeks ago
yuxiqian e2b8f70e0d
[FLINK-37124][build] Simplify logs in test cases to avoid flooding GHA outputs
This closes  #3860
2 weeks ago
North Lin b5d9673258
[FLINK-34865][pipeline-connector/mysql] Support sync table and column comment
This closes #3482

Co-authored-by: Leonard Xu <xbjtdcq@gmail.com>
2 weeks ago
wenmo d77cf7fa75 [FLINK-36647][transform] Support Timestampdiff and Timestampadd function in cdc pipeline transform
This closes #3698
2 weeks ago
moses 9992584365
[FLINK-37011][cdc-transform] Improve get source field value by column name in PreTransformProcessor
This closes #3836
2 weeks ago
Wink 2fd03e683e
[FLINK-36865][cdc] Provide UNIX_TIMESTAMP series functions in YAML pipeline
This closes #3819.
3 weeks ago
yuxiqian 39fd00ce49
[FLINK-36610] MySQL CDC supports parsing gh-ost / pt-osc generated schema changes (#3668)
Co-authored-by: MOBIN-F <18814118038@163.com>
3 weeks ago
Jzjsnow 630e0d70c2
[FLINK-36701][cdc-runtime] Obtain latest evolvedSchema when SinkDataWriterOperator handles FlushEvent from failover
This closes #3802

Co-authored-by: jzjsnow <snow.jiangzj@gmail.com>
3 weeks ago
Shawn Huang dd865bf440
[FLINK-36970][cdc-common] Merge result of data type BIGINT and DOUBLE should be DOUBLE instead of STRING
This closes #3821
3 weeks ago
moses fc71888d7a
[hotfix][cdc-common] Remove duplicated code to improve performance
This closes #3840.

Co-authored-by: zhangchaoming.zcm <zhangchaoming.zcm@antgroup.com>
3 weeks ago
Umesh Dangat baba9c6ae6
[FLINK-37017][cdc-common] Supports map and array types for binary record data
This closes #3434.
4 weeks ago
yuxiqian 2e03f9ad49 [FLINK-36763][cdc-runtime] Introduce distributed schema evolution topology for sources with parallelized metadata
This closes #3801
1 month ago
yuxiqian a2e433ebdd [cdc-common] Extract column / schema type merging utility methods to `SchemaMergingUtils` 1 month ago
Hang Ruan a0ae3320fa
[FLINK-36784][common] Support to add metadata columns for data in the meta fields of DataChangeEvent at transform (#3758)
Co-authored-by: Kunni <lvyanquan.lyq@alibaba-inc.com>
1 month ago
Kunni 06154e9674
[FLINK-36525][transform] Support for AI Model Integration for Data Processing (#3753) 2 months ago
yuxiqian 4bd3a2347b
[FLINK-36565][transform] Route allows merging Decimals with various precisions
This closes #3651
3 months ago
Shawn Huang d9ceee050b
[FLINK-36699][cdc-common] fix nullability when converting cdc data type to flink data type (#3713) 3 months ago
yuxiqian b5ab385b8c
[FLINK-36461][transform] Fix schema evolution failure with un-transformed tables
This closes  #3632.
3 months ago
yuxiqian 84f97ed4c1
[FLINK-36474][route] Support merging timestamp columns when routing
This closes  #3636.
3 months ago
yuxiqian 2e938a92f5
[FLINK-36128][cdc-runtime] Fix potential unrecoverable in-flight data exception by promoting LENIENT as the default schema change behavior
This closes #3574.
5 months ago
yuxiqian 3837887819
[FLINK-36092][cdc-runtime] Fix schema evolution failure with wildcarded transform rule
This closes #3557.
5 months ago
Hongshun Wang 6205a5a0f1
[FLINK-36094][cdc-runtime] Improve the Exception that SchemaRegistryRequestHandler thrown
This closes #3558.
5 months ago
yuxiqian 78fda8b5ab
[FLINK-35243][cdc-common] Extends more schema change event types support
This close  #3521.
5 months ago
Qingsheng Ren d99e718951
[FLINK-36054][cdc][build] Fix Flink CDC parent pom and release scripts (#3540) 6 months ago
Kunni 64f996b504
[FLINK-35891][pipeline-connector][paimon] Support dynamic bucket in Paimon sink
This closes  #3499.
6 months ago
yuxiqian e2bb917ad0
[FLINK-34876][transform] Support UDF functions in transform (#3465) 6 months ago
Hang Ruan 07446d1f9d
[FLINK-35715][common] Ignore the compact optimize for mysql timestamp type in BinaryRecordData (#3511) 6 months ago
yuxiqian 81d916fc73
[FLINK-35272][cdc-runtime] Transform supports omitting and renaming computed column
This closes #3285.
6 months ago
yuxiqian 5ed9e05bdf
[FLINK-35242][cdc-common][cdc-runtime] Support TRY_EVOLVE and LENIENT schema evolution behavior
This closes #3339.
6 months ago
lvyanquan 4561a8a32b [FLINK-34638][cdc-common] Support column with default value 6 months ago
Wink 7ca11359d7
[FLINK-35743][cdc-runtime] Correct the temporal function semantics
This closes #3449.

Co-authored-by: wenmo <32723967+wenmo@users.noreply.github.com>
Co-authored-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>
6 months ago
Hongshun Wang d76beecdb0
[FLINK-35234][minor][cdc-common] Fix potential NPE in ConfigurationUtils
This closes #3255.
6 months ago
Leonard Xu 26ff6d2a08 [FLINK-35237][cdc-common] Improve the interfaces and reorganize the directory. 6 months ago
鼎昕 9a4dede3fe [FLINK-35237][cdc-common] Allow Sink to choose HashFunction in PrePartitionOperator 6 months ago
yuxiqian a098ddacba
[FLINK-35781][cli] Provide a default parallelism 1 for pipeline jobs
This closes #3458.
7 months ago
yuxiqian ad1f554c0e
[FLINK-35647][route] Support symbol replacement to enrich routing rules
This closes #3428.

Co-authored-by: 张田 <zhang_tian@inspur.com>
Co-authored-by: yangshuaitong <duguhoney@gmail.com>
7 months ago
yux 2bd2e4ce24
[FLINK-35121][common] Adds validation for pipeline definition options 8 months ago
North Lin c958daf8ab
[FLINK-35540][cdc-common] Fix table missed when database and table are with the same name
This closes  #3396.
8 months ago
yux 17ff659ff7
[hotfix][build] Remove test-jar from parent pom and only add to necessary modules
This closes  #3397.

Co-authored-by: Qingsheng Ren <renqschn@gmail.com>
8 months ago
joyCurry30 0214166049
[FLINK-35325][cdc-connector][paimon] Support specify column order when adding new columns to a table
This closes #3323.
8 months ago
Kunni c1e1aa48d9
[FLINK-34648][cdc] Avoid RPC timeout when applying SchemaChangeEvent to downstream external systems
This closes  #3128.
9 months ago
Kunni ef2eece256
[pipeline-connector][paimon] add paimon pipeline data sink connector. (#2916) 9 months ago
Qingsheng Ren 6017b16528
[FLINK-34653][cdc][route] Support table merging with route rules
This closes  #3129.
10 months ago
Qishang Zhong 18dd51fe22
[FLINK-34905][cdc-connector][mysql] Fix the default length of CHAR/BINARY data type of Add column DDL (#3145) 10 months ago
xleoken 6510e670fa
[FLINK-34959][cdc] Update old flink-cdc-connectors artifactId (#3200) 10 months ago
wenmo c92903016d [cdc-common] Introduce partitionKeys into schema and add related util classes 10 months ago
Qishang Zhong d099603ef1
[FLINK-34948][cdc][common] Fix cdc RowType can not convert to Flink type 10 months ago
xleoken 5f4f86fb31
[hotfix] Change old com.ververica dependencies to org.apache.flink (#3110) 10 months ago
Hang Ruan e10c8691ad
[FLINK-34584][cdc][build] Rename package from com.ververica to org.apache.flink
This closes #3089.
11 months ago
Hang Ruan eeb4bab92d [FLINK-34184][cdc][license] Update Copyright from Ververica to Apache 11 months ago