Commit Graph

63 Commits (ef78664a365509ba2ed6866d2dd20857bdca25b9)

Author SHA1 Message Date
yuxiqian dfbaf617e2
[FLINK-36558][source-connector/mysql] Fix column metadata parsing compatibility with MySQL 8.0.17&8.0.18
This closes  #3647.
2 months ago
Sergei Morozov b50d1728e0
[FLINK-36891[[source-connector][mysql] Fix corrupted state in case of serialization failure in MySQL CDC Source
This closes #3794.
3 months ago
Xin Gong 0037c4379e
[FLINK-36771][cdc-connector][base&mysql] Fix UT trigger error: Invalid assigner status {} [NEWLY_ADDED_ASSIGNING_FINISHED] (#3755) 3 months ago
Hang Ruan 23e8149e6a
[FLINK-36803][cdc-connector][base & mysql] Use the same format `tableId:chunkId` for splitId in SnapshotSplit (#3763) 3 months ago
Xin Gong 1d4e366969
[FLINK-36772][mysql][cdc-base] Fix error placeholder for errorMessageTemplate of Preconditions
This closes  #3754
3 months ago
Leonard Xu d0b2fe38a2
[hotfix][source-connector/mysql] Fix conflicts after flink version bumped to 1.19 (#3748) 3 months ago
Leonard Xu 01a1e5fba5
[FLINK-36656][mysql] Fix type conversion failure for newly-added sharding table with mysql boolean type (#3683) 3 months ago
ConradJam dd79a02f0f
[FLINK-36586][build] Update flink version to 1.19 (#3660)
Co-authored-by: ConradJam <czy006@apache.com>
Co-authored-by: Hang Ruan <ruanhang1993@hotmail.com>
3 months ago
ConradJam 13248faa22
[FLINK-35592][cdc-connector][mysql] Fix MysqlDebeziumTimeConverter miss timezone when convert to timestamp
This closes #3332

Co-authored-by: Hang Ruan <ruanhang1993@hotmail.com>
4 months ago
Xin Gong d5f1b9ace8
[FLINK-36247][cdc-connector][mysql] Fix potential transaction leak during MySQL snapshot phase (#3602) 4 months ago
Shawn Huang ecce75949a
[FLINK-36463][cdc-connector][mysql] Forbid to override some debezium options (#3631) 4 months ago
yuxiqian d5f549f91f
[FLINK-36326][cdc-connector][mysql] Send BinlogSplitUpdateRequestEvent only once to fix auto scan newly-added table failure (#3613)
Signed-off-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>
Co-authored-by: Hang Ruan <ruanhang1993@hotmail.com>
4 months ago
Hongshun Wang afe9c3c0ad
[FLINK-36150][pipeline-connector/mysql] tables.exclude should work even scan.binlog.newly-added-table.enabled is true
This closes  #3573.
6 months ago
Kevin Wang 77c63385d9
[FLINK-36081][cdc-connector][mysql] Remove the schemas of outdated tables in the BinlogSplit when restart (#3548)
Co-authored-by: 云时 <mingya.wmy@alibaba-inc.com>
6 months ago
Hongshun Wang 4bf5a395a5
[FLINK-34688][cdc-connector][mysql] Make scan newly table trigger condition strictly
This closes #3519.
7 months ago
Thorne 986f37b307
[FLINK-35674][cdc-connector][mysql]Fix blocking caused by searching for timestamp in binlog file (#3432) 7 months ago
Hongshun Wang 0c49959f67
[FLINK-35524][cdc-base] Clear connections pools when reader exist. (#3388) 7 months ago
SeungMin 91fc7bb506
[FLINK-35740][cdc-connector][mysql] Allow column as chunk key even it's not primary key
This closes #3448.
7 months ago
Zhongmin Qiao ea71b2302d
[FLINK-35874][cdc-connector][mysql] Check pureBinlogPhaseTables set before call getBinlogPosition method
This closes #3488.
7 months ago
ChengJie1053 9a97182fbb
[minor][cdc-connector][mysql] Code optimization for constants
This closes #3385
7 months ago
Shawn Huang 80920aaa28
[FLINK-35297][mysql] Add validation for option connect.timeout because of HikariConfig limitation (#3295) 8 months ago
yux 17ff659ff7
[hotfix][build] Remove test-jar from parent pom and only add to necessary modules
This closes  #3397.

Co-authored-by: Qingsheng Ren <renqschn@gmail.com>
9 months ago
Shawn Huang 19c382c0f2
[FLINK-35409][cdc][mysql] Request more splits if all splits are filtered from addSplits method
This closes #3342.
9 months ago
Shawn Huang b2cb30f240
[FLINK-35408][mysql] Introduce 30 minutes tolerance when validate the time-zone setting
This closes  #3341.
9 months ago
Shawn Huang 6350eec66c
[FLINK-35301][cdc] Avoid deadlock when loading driver classes
This closes  #3300.
9 months ago
Shawn Huang 18b0627e34
[FLINK-35295][mysql] Improve jdbc connection pool initialization failure message
This closes #3293.
9 months ago
Shawn Huang bddcaae784
[FLINK-35294][mysql] Use source config to check if the filter should be applied in timestamp starting mode
This closes #3291.
9 months ago
Shawn Huang 8e8fd304af
[FLINK-35298][cdc][metrics] Improve the fetch delay metrics
This closes #3298.
9 months ago
ConradJam 6232d84052
[FLINK-35173][cdc][mysql] Debezium custom time serializer for MySQL connector (#3240) 10 months ago
Leonard Xu ddf557c8e1 [tests][cdc-connector][db2] Improve the DB2 tests
This closes #2870
10 months ago
Hongshun Wang 48ca8623bb
[FLINK-34634][cdc-base][mysql] Fix that Restarting the job will not read the changelog anymore if it stops before the synchronization of meta information is complete and some table is removed (#3134) 11 months ago
L af7665d338
[FLINK-34689][MySQL][Feature] check binlog_row_value_options (#3148) 11 months ago
shikai93 80d461b1b8
[MySQL] Fix for DBZ-7037 and #2217 to not clear tableMapEventByTableId for fake Rotates Event (#3065) 11 months ago
xleoken 5f4f86fb31
[hotfix] Change old com.ververica dependencies to org.apache.flink (#3110) 11 months ago
Xin Gong 14b81dc6ec
[FLINK-34715][cdc][cdc-connector][mysql] Correctly close BinlogSplitReader in MySQL UTCase
This closes #3161.
12 months ago
Hang Ruan e10c8691ad
[FLINK-34584][cdc][build] Rename package from com.ververica to org.apache.flink
This closes #3089.
12 months ago
Hang Ruan eeb4bab92d [FLINK-34184][cdc][license] Update Copyright from Ververica to Apache 12 months ago
Hongshun Wang 1eeda75eaa [cdc-connector][cdc-base] Support capture newly added tables with un-blocking stream read for CDC Framework(#3024) 1 year ago
Hongshun Wang 7ea197242f
[debezium] Upgrade to Debezium 1.9.8.Final(#3033) (#3034) 1 year ago
Kunni 9abf2cf900
[hotfix][cdc-connector][mysql] Skip SchemaChangeEvents that were not included in capturedTableFilter (#2986)
This closes #2981
1 year ago
Hongshun Wang 3bf886b94d [cdc-connector][mysql] Add SNAPSHOT_ONLY mode for Mysql CDC Source (#2901) 1 year ago
gongzhongqiang a24c5d1b24
[cdc][e2e] Unify logic to clean docker image when connector e2e test finished (#2978) 1 year ago
FocusComputing b42bc60ab9
[source-connector][mysql] speedup searching binlog by timestamp (#2924)
* [feature] speedup search mysql binlog when mysql-cdc-connector consume binlog by timestamp

* improve code

* improve code

* code improve

* test case add and code improve

* code improve

* code improve

* code improve, ut add and bugfix for purged log

---------

Co-authored-by: TJX2014 <tangjinxin@deepexi.com>
1 year ago
Leonard Xu a5fa2dc6c8
[hotfix][cdc-connector][mysql] Expose row_kind metadata column for mysql cdc source. (#2927)
We use 'row_kind' name instead of 'op' or 'op_type', the operation type is a concept of database and database only has 'update'
type and hasn't 'update_before' and 'update_after' type.
1 year ago
Xianxun Ye 8eba910063
[cdc-connector][mysql] Support metadata 'operation' virtual column for MySql Connector. (#2913) 1 year ago
Hongshun Wang 2c557c6a78
[cdc-connector][sqlserver][tests] Fix UT errors by correcting right output (#2864) 1 year ago
Jiabao Sun 8a08ded410
[cdc-pipeline-connector][mysql] Fix precision problem of BIT type conversion (#2820)
This closes #2820.
1 year ago
Dian Qi f970444efa
[cdc-source-connector][mysql] fix deserialization issue on table metadata binlog event (#2682)
Co-authored-by: EchoLee5 <39044001+EchoLee5@users.noreply.github.com>
1 year ago
Josh Mahonin 63b39ba22c
[cdc-source-connector][mysql] Fix attempted load of isSuspended in MySqlSplitSerializer::deserializeSplit when cdc version <= 2.2.0 (#2550)
Co-authored-by: Hang Ruan <ruanhang1993@hotmail.com>
1 year ago
Jiabao Sun 08c5dfed83 [cdc-pipeline-connector][mysql] Add tests for mysql pipeline connector
Co-authored-by: Hang Ruan <ruanhang1993@hotmail.com>
1 year ago