Commit Graph

117 Commits (947dffca3182722340048c913c286ea5d3d5956b)

Author SHA1 Message Date
moses 947dffca31
[FLINK-37122][build] Try to be compatible with old flink version 1.18.x
This closes #3859
2 weeks ago
Leonard Xu 77785c1c08
[hotfix[cdc-runtime] Close MetadataApplier in SchemaRegistry when the job stops
This closes #3864
2 weeks ago
MOBIN d83ccbeeb4
[FLINK-36754][transform] Projection should treated as an asterisk when projection expression is empty or null
This closes  #3749
2 weeks ago
yuxiqian e2b8f70e0d
[FLINK-37124][build] Simplify logs in test cases to avoid flooding GHA outputs
This closes  #3860
2 weeks ago
wenmo d77cf7fa75 [FLINK-36647][transform] Support Timestampdiff and Timestampadd function in cdc pipeline transform
This closes #3698
2 weeks ago
Wink 79e868b864
[FLINK-36741][transform] Fix the decimal precision and length lost during transform
This closes #3740
2 weeks ago
Wink 551bdf395d
[FLINK-36636][transform] Supports timestamp comparison in cdc pipeline transform
This closes #3677
2 weeks ago
moses 9992584365
[FLINK-37011][cdc-transform] Improve get source field value by column name in PreTransformProcessor
This closes #3836
2 weeks ago
Wink 2fd03e683e
[FLINK-36865][cdc] Provide UNIX_TIMESTAMP series functions in YAML pipeline
This closes #3819.
3 weeks ago
MOBIN f7f3901fa3
[FLINK-37012][transform] Fix argument type mismatch when metadata column used in function
This closes #3837
3 weeks ago
Jzjsnow 630e0d70c2
[FLINK-36701][cdc-runtime] Obtain latest evolvedSchema when SinkDataWriterOperator handles FlushEvent from failover
This closes #3802

Co-authored-by: jzjsnow <snow.jiangzj@gmail.com>
3 weeks ago
Wink 49dc957ac6
[FLINK-36981][transform] Considering sharding tables with different schema in transform projection
This closes #3826.
3 weeks ago
yuxiqian 486ee6f805
[FLINK-36956][transform] Append NOT NULL attribute for Primary Key columns
This closes  #3815
3 weeks ago
moses fc71888d7a
[hotfix][cdc-common] Remove duplicated code to improve performance
This closes #3840.

Co-authored-by: zhangchaoming.zcm <zhangchaoming.zcm@antgroup.com>
3 weeks ago
Umesh Dangat baba9c6ae6
[FLINK-37017][cdc-common] Supports map and array types for binary record data
This closes #3434.
4 weeks ago
yuxiqian 8424467239 [FLINK-36690][cdc-runtime] Fix schema operator hanging under extreme parallelized pressure
This closes #3680
1 month ago
yuxiqian 2e03f9ad49 [FLINK-36763][cdc-runtime] Introduce distributed schema evolution topology for sources with parallelized metadata
This closes #3801
1 month ago
Hang Ruan a0ae3320fa
[FLINK-36784][common] Support to add metadata columns for data in the meta fields of DataChangeEvent at transform (#3758)
Co-authored-by: Kunni <lvyanquan.lyq@alibaba-inc.com>
1 month ago
Hang Ruan c6cf46b75c
[FLINK-36879][runtime] Support to convert delete as insert in transform (#3804) 1 month ago
yuxiqian 2ad80066bc
[FLINK-36864][cdc-runtime] Fix unable to use numeric literals that goes beyond Int32 range (#3785)
Signed-off-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>
1 month ago
yuxiqian b0a7d920b9
[FLINK-36866][transform] Fix unable to narrow casting on numeric values
This closes #3786.
1 month ago
yuxiqian 39b560e377
[FLINK-36854][transform] Add missed comment and default value after transform
This closes #3780.
2 months ago
Kunni 06154e9674
[FLINK-36525][transform] Support for AI Model Integration for Data Processing (#3753) 2 months ago
ConradJam dd79a02f0f
[FLINK-36586][build] Update flink version to 1.19 (#3660)
Co-authored-by: ConradJam <czy006@apache.com>
Co-authored-by: Hang Ruan <ruanhang1993@hotmail.com>
2 months ago
yuxiqian 4bd3a2347b
[FLINK-36565][transform] Route allows merging Decimals with various precisions
This closes #3651
3 months ago
yuxiqian 51c679a914
[FLINK-36596][transform] Fix unable to schema evolve with project-less transform rules
This closes  #3665.
3 months ago
MOBIN 5db16ba30c
[FLINK-36093][transform] Fix preTransformoperator wrongly filters columns belong to different transforms
This closes #3572
3 months ago
yuxiqian b5ab385b8c
[FLINK-36461][transform] Fix schema evolution failure with un-transformed tables
This closes  #3632.
3 months ago
North Lin 8647fe4986
[FLINK-36375][cdc-runtime] Fix missed default value in AddColumnEvent/RenameColunEvent
This closes  #3622.
3 months ago
Sergei Morozov e15bf6f04a
[FLINK-36407][runtime] Shut down coordinatorExecutor upon closing SchemaRegistry (#3624) 3 months ago
MOBIN 1d2cfd789e
[FLINK-35985][transform] Correct the substring function in transform rule
This closes #3537.
3 months ago
yuxiqian 84f97ed4c1
[FLINK-36474][route] Support merging timestamp columns when routing
This closes  #3636.
3 months ago
lzshlzsh 2a5828c0ac
[FLINK-35291][runtime] Improve the ROW data deserialization performance of DebeziumEventDeserializationScheme (#3289)
Co-authored-by: liuzeshan <liuzeshan@bytedance.com>
3 months ago
yuxiqian 73977c127b
[FLINK-36183] Fix lenient schema evolution failure with route blocks (#3583) 5 months ago
yuxiqian 06fc939569
[FLINK-36184][transform] Fix transform operator swallows schema changes from tables not present in transform rules (#3589) 5 months ago
yuxiqian a876af2589
[minor][cdc-runtime] Run schema coordinator logic asynchronously to avoid blocking the main thread
This closes #3557
5 months ago
yuxiqian 0df63e26e8
[hotfix][cdc-runtime] Keep upstream pending requests in order to avoid checkpoint hanging & state inconsistency in timestamp startup mode
This closes  #3576.
5 months ago
yuxiqian 2e938a92f5
[FLINK-36128][cdc-runtime] Fix potential unrecoverable in-flight data exception by promoting LENIENT as the default schema change behavior
This closes #3574.
5 months ago
yuxiqian 060d2032e8
[hotfix][cdc-runtime] Fix schema registry hanging in multiple parallelism
This closes  #3567.
5 months ago
yuxiqian 3837887819
[FLINK-36092][cdc-runtime] Fix schema evolution failure with wildcarded transform rule
This closes #3557.
5 months ago
yuxiqian ee843e2f24
[FLINK-36114][cdc-runtime] Make SchemaRegistryRequestHandler thread safe by blocking subsequent schemaChangeEvent
This closes #3563.

Co-authored-by: Hongshun Wang <loserwang1024@gmail.com>
5 months ago
Hongshun Wang 6205a5a0f1
[FLINK-36094][cdc-runtime] Improve the Exception that SchemaRegistryRequestHandler thrown
This closes #3558.
5 months ago
yuxiqian 78fda8b5ab
[FLINK-35243][cdc-common] Extends more schema change event types support
This close  #3521.
5 months ago
Hongshun Wang d3473de4db
[FLINK-36076][minor][cdc-runtime] Set isSchemaChangeApplying as volatile for thread safe consideration
This closes #3556.
5 months ago
Qingsheng Ren d99e718951
[FLINK-36054][cdc][build] Fix Flink CDC parent pom and release scripts (#3540) 6 months ago
yuxiqian 3cb91fb51a
[FLINK-35805][transform] Add __data_event_type__ metadata column
This closes  #3468
6 months ago
yuxiqian 874ff4ff20
[hotfix][cdc-runtime] Invalidate cache correctly to avoid classloader leakage
This closes #3533
6 months ago
yuxiqian 104209558a [FLINK-36034][cdc-runtime] Get rid of Flink table planner dependency in cdc runtime module
This closes #3513.
6 months ago
yuxiqian 2885d12f57 [FLINK-35991][cdc-runtime] Resolve operator conflicts in transform SQL operator tables 6 months ago
MOBIN 17d21a957a
[FLINK-35984][cdc-runtime] Fix bug that metadata column name can not be used in transform rule
This closes  #3528.

Co-authored-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>
6 months ago