Commit Graph

1240 Commits (3d77dfc9070a4fb3d86e525543eb7383a6003481)
 

Author SHA1 Message Date
yuxiqian 3d77dfc907 fix: mysql ci failure 1 week ago
yuxiqian 9e98c49c29 test: get rid of fastjson 1 week ago
yuxiqian 856c664014 tests: fetch table results with timeout set properly 1 week ago
yuxiqian 414e3be054 build: do not fail fast & adjust timeout to a reasonable duration 1 week ago
Jason Zhang 10f503ffa4
[hotfix][docs][pipeline-connector/maxcompute] Fix maxcompute connector typo in examples section
This closes #3875
1 week ago
Hang Ruan bfcc46aede
[build] Update version to 3.4-SNAPSHOT and add release-3.3 docs
This closes #3870
1 week ago
Hang Ruan 606c9d20c6
[build] Update version to 3.4-SNAPSHOT and add release-3.3 docs
This closes #3870
1 week ago
yuxiqian 3e16a66972
[hotfix][build] Miscellaneous fixes on GHA workflows
This closes #3839
2 weeks ago
North Lin 8815f2b879
[FLINK-34865][pipeline-connector/mysql] Support sync newly added table's comment
This closes #3869
2 weeks ago
North Lin bbef6213a6
[hotfix][pipeline-connector][mysql] Fix missed optional option in MySqlDataSourceFactory
This closes #3867
2 weeks ago
Leonard Xu e81df946b9 [minor][ci] Set proper timeout for compile_and_test step of CI job 2 weeks ago
Seung-Min Lee a1c9c19d7d [FLINK-36578][pipeline-connector/mysql] Introduce option to unify json type output between snapshot phase and binlog phase
This closes #3658
2 weeks ago
moses 947dffca31
[FLINK-37122][build] Try to be compatible with old flink version 1.18.x
This closes #3859
2 weeks ago
Olivier 7997f51c55
[FLINK-35067][cdc-connector][postgres] Adding metadata 'row_kind' for Postgres CDC Connector.
This closes #3716.

Co-authored-by: Leonard Xu <xbjtdcq@gmail.com>
2 weeks ago
Kunni a16abd5d24
[FLINK-36913][pipeline-connector][kafka] Introduce option to define custom mapping from upstream table id to downstream topic name
This closes #3805
2 weeks ago
Leonard Xu 77785c1c08
[hotfix[cdc-runtime] Close MetadataApplier in SchemaRegistry when the job stops
This closes #3864
2 weeks ago
yuxiqian cf95dcab64
[hotfix][test] Make pipeline migration tests more robust
This closes #3866
2 weeks ago
wudi 2fa215e5c4
[FLINK-35600][pipeline-connector/mysql] Add timestamp for low and high watermark
This closes #3415
2 weeks ago
Kunni afd4a74c0a
[FLINK-35802][pipeline-connectors/mysql] Clean ChangeEventQueue to avoid deadlock when calling BinaryLogClient#disconnect method
This closes #3463
2 weeks ago
Petrichor 177da16fac
[FLINK-36351][pipeline-connector/doris] Support the conversion of Flink TIME type to Doris String type
This closes #3620
2 weeks ago
Jzjsnow 6b13868dd5
[FLINK-36620][cdc-cli] Add support for specifying the --flink-home parameter via an '=' character
This closes #3838
2 weeks ago
Thorne 087195a3a6
[FLINK-36224][docs] Add the version mapping between pipeline connectors and flink
This closes  #3598
2 weeks ago
Sergei Morozov 5691b7bf97
[FLINK-36406][cdc-runtime] Close MetadataApplier when the job stops
This closes  #3623
2 weeks ago
North Lin 9ad071c1fa
[FLINK-36282][pipeline-connector][cdc-connector][mysql]fix incorrect data type of TINYINT(1) in mysql pipeline connector (#3608)
* [FLINK-36282][pipeline-connector][cdc-connector][mysql]fix incorrect data type of TINYINT(1) in mysql pipeline connector

* reformat code

* Update MySqlPipelineITCase.java

* pass a boolean value instead of Properties

* uodate FAQ

* add a method to get tinyInt1isBit

* add new cdc config `treat-tinyint1-as-boolean`

* Update MySqlChunkSplitter.java

* change param name
2 weeks ago
Jason Zhang 47e5cd6d89
[hotfix][pipeline-connector][maxcompute] Fix MaxCompute Pipeline Connector tests by upgrading maxcompute-emulator
This closes  #3862
2 weeks ago
MOBIN d83ccbeeb4
[FLINK-36754][transform] Projection should treated as an asterisk when projection expression is empty or null
This closes  #3749
2 weeks ago
yuxiqian e2b8f70e0d
[FLINK-37124][build] Simplify logs in test cases to avoid flooding GHA outputs
This closes  #3860
2 weeks ago
North Lin b5d9673258
[FLINK-34865][pipeline-connector/mysql] Support sync table and column comment
This closes #3482

Co-authored-by: Leonard Xu <xbjtdcq@gmail.com>
2 weeks ago
Leonard Xu 50e6c82870 [build][minor] Upgrade the max file length from 3k to 4k 2 weeks ago
wenmo d77cf7fa75 [FLINK-36647][transform] Support Timestampdiff and Timestampadd function in cdc pipeline transform
This closes #3698
2 weeks ago
Wink 79e868b864
[FLINK-36741][transform] Fix the decimal precision and length lost during transform
This closes #3740
2 weeks ago
hiliuxg 8b554458d3
[FLINK-36977][pipeline-connector/paimon] Apply default value when process add_column schema change envent
This closes #3824.

Co-authored-by: Leonard Xu <xbjtdcq@gmail.com>
2 weeks ago
Wink 551bdf395d
[FLINK-36636][transform] Supports timestamp comparison in cdc pipeline transform
This closes #3677
2 weeks ago
Hongshun Wang bcd70df85c
[FLINK-35387][cdc-connector][postgres] PG CDC source support heart beat
This closes #3667
2 weeks ago
hiliuxg 6b65aed4c5
[FLINK-36974][cdc-cli] Support overwrite flink configuration via command line
This closes #3823

Co-authored-by: helloliuxg@gmail.com <xiaogenliu@kugou.net>
2 weeks ago
yuxiqian 865e14bfd7
[FLINK-35634][build] Add CdcUp playground CLI scripts
This closes #3605
2 weeks ago
moses 9992584365
[FLINK-37011][cdc-transform] Improve get source field value by column name in PreTransformProcessor
This closes #3836
2 weeks ago
Junbo wang 0f675061e1
[FLINK-36700][pipeline-connector][elasticsearch] Elasticsearch pipeline sink supports authentication
This closes  #3728
2 weeks ago
yuxiqian 56dbb1fd53
[FLINK-36193][pipeline-connectors][paimon/doris/starrocks] Supports applying TRUNCATE & DROP Table Event to Paimon, StarRocks and Doris
This closes #3673
2 weeks ago
moses 4dc31d8c61
[FLINK-37042][pipeline-connector/maxcompute] Rename maxcompute pipieline connector options to follow flink style
This closes  #3852
2 weeks ago
hiliuxg 32934393ce
[FLINK-36985][pipeline-connector/paimon] Tolerante ColumnAlreadyExistException when apply AddColumnEvent in paimon
This closes  #3828.
2 weeks ago
MOBIN a130718c96
[FLINK-36858][pipeline-connector][kafka] Fix JsonRowDataSerializationSchema compatibility issue with Flink 1.20
This closes  #3784

Co-authored-by: Leonard Xu <xbjtdcq@gmail.com>
3 weeks ago
Kunni 75b8a0cdf3
[FLINK-36964][pipeline-connector/paimon] Fix potential exception when SchemaChange in parallel with Paimon Sink
This closes #3818

Co-authored-by: yuxiqian.yxq <yuxiqian.yxq@alibaba-inc.com>
3 weeks ago
Wink 2fd03e683e
[FLINK-36865][cdc] Provide UNIX_TIMESTAMP series functions in YAML pipeline
This closes #3819.
3 weeks ago
yuxiqian 39fd00ce49
[FLINK-36610] MySQL CDC supports parsing gh-ost / pt-osc generated schema changes (#3668)
Co-authored-by: MOBIN-F <18814118038@163.com>
3 weeks ago
MOBIN f7f3901fa3
[FLINK-37012][transform] Fix argument type mismatch when metadata column used in function
This closes #3837
3 weeks ago
Shawn Huang 085684b773
[FLINK-36811][mysql] MySQL cdc setIsProcessingBacklog in snapshot phase and exit when snapshot phase finished (#3793) 3 weeks ago
Kunni 1fb68a8d62
[FLINK-35325][transform] Skip insufficient_quota error when running test case using ad model. (#3849) 3 weeks ago
Kunni 7b01d9ca40
[hotfix][ci] Fix CI failure due to implicit conflicts
This closes #3846
3 weeks ago
Jzjsnow 630e0d70c2
[FLINK-36701][cdc-runtime] Obtain latest evolvedSchema when SinkDataWriterOperator handles FlushEvent from failover
This closes #3802

Co-authored-by: jzjsnow <snow.jiangzj@gmail.com>
3 weeks ago