Commit Graph

459 Commits (9bbfcb477dca1482b1728171730d5ce9c066b3d6)
 

Author SHA1 Message Date
Hang Ruan 9bbfcb477d [docs] Bump Flink version to 1.16.0 2 years ago
Hang Ruan db24ba59d4 [common] Bump Flink version to 1.16.0 2 years ago
gongzhongqiang 15392e8f84
[docs][db2] Add db2 to README.md (#1699) 2 years ago
wangxiaojing da5b322890
[tidb] Checkpoint is not updated long after a task has been running (#1686)
Co-authored-by: wangxiaojing <wangxiaojing@lixiang.com>
2 years ago
gongzhongqiang e377c339b6
[hotfix] Add method getMaxResolvedTs back to class CDCClient. (#1695)
Co-authored-by: gongzhongqiang <764629910@qq.com>
2 years ago
Hang Ruan 8685fd4027
[docs] Bump connector version to flink 1.15.2 in docs (#1684) 2 years ago
leozlliang 1111f0270e
[tidb] Fix data lost when region changed (#1632)
* [tidb] Fix data lost when region changed.

* [tidb] Add tidb cdc region change unit test and polish TiKVRichParallelSourceFunction.

Co-authored-by: liangzhili2 <liangzhili2@joyy.com>
Co-authored-by: gongzhongqiang <gongzhongqiang@gigacloudtech.com>
2 years ago
Jiabao Sun aee88f947c
[hotfix][docs] Correct reference link for DB2 docs (#1683) 2 years ago
Qingsheng Ren 4e63851c6a [mysql] Update docs of specifying starting offset feature of MySQL CDC source 2 years ago
Qingsheng Ren c07451884d [hotfix][mysql] Remove unused constructor in MySqlTableSource 2 years ago
Qingsheng Ren aca5643704 [mysql] Support starting reading binlog from earliest offset / specific offset / timestamp (#1661)
This closes #254
2 years ago
Jiabao Sun 301e5a8ab0 [mongodb] Support incremental snapshot for MongoDB CDC Connector 2 years ago
Qingsheng Ren ffdfd549b9 [cdc-base] Extract a higher level IncrementalSource independent of JDBC dialects 2 years ago
skylines 21ba3b05a2
[hotfix][docs] Fix db2 doc download link (#1681) 2 years ago
Leonard Xu f8f1a64081 [docs][db2] Add Db2 CDC connector doc link 2 years ago
Hang Ruan a5f6657046 [docs][db2] Add Db2 cdc connector docs 2 years ago
Hang Ruan f1b130cedf [build][db2] Add Db2 pipeline script 2 years ago
Hang Ruan 23e06df79d [e2e][db2] Add Db2 e2e test 2 years ago
Hang Ruan 3592dcc4ee [db2] Add sql connector for Db2 CDC (#1626)
This closes #450.
2 years ago
Hang Ruan 5d005809e9 [db2] Introduce Db2 cdc connector (#450) 2 years ago
xieyi888 a6fa1e694e
[tidb] Fix data loss in tidb-cdc when switch from snapshot read to change event read  (#1207)
Signed-off-by: Yi Xie <xieyi01@rd.netease.com>

Signed-off-by: Yi Xie <xieyi01@rd.netease.com>
Co-authored-by: Yi Xie <xieyi01@rd.netease.com>
2 years ago
Jiabao Sun fbf5eba03c
[hotfix][cdc-base] Fix OffsetFactory is not initialized when restored from checkpoint (#1657) 2 years ago
Tigran Manasyan 5b3dd9884e
[common] Bump Flink version to 1.15.2 (#1504)
This closes #1363. 

Co-authored-by: Hang Ruan <ruanhang1993@hotmail.com>
2 years ago
Jiabao Sun 8976f25925
[cdc-base] Abstract naming for variables of incremental framework (#1646) 2 years ago
Hang Ruan a4b969d4a3
[mysql] ChunkSplitter supports to snapshot when splitting chunks for a table (#1463)
This closes #1463.

Co-authored-by: Leonard Xu <xbjtdcq@gmail.com>
2 years ago
Jiabao Sun 21521506c0
[cdc-base] Fix StreamSplit's endingOffset of StreamSplitAssigner (#1659) 2 years ago
Jiabao Sun ddd0680009
[cdc-base] JdbcSourceConfig can be template type of DataSourceDialect (#1639) 2 years ago
lidoudou1993 8b2094c5f8
[mysql] Use the latest received meta of binlog split when request the next meta (#1479)
Co-authored-by: lidoudou3 <lidoudou1@jd.com>
2 years ago
sunny 40cd758656
[minor] Use Collections.singletonList instead of Arrays.asList if possible (#1569) 2 years ago
He Wang b1b092c97e
[oceanbase] Add new deserialization schema with runtime converter (#1356)
This closes #980.
2 years ago
Leonard Xu d3435381a6 [mysql] Avoid duplicate split requests when add new table.(#1156)
This closes #1149.
2 years ago
yurunchuan ad80e47c42 [mysql] Avoid duplicate split requests when add new table (#1156) 2 years ago
Leonard Xu fcbed91311 [postgres] Upgrade org.postgresql:postgresql from 42.2.14 to 42.2.26 for CVE-2022-31197 (#1616) 2 years ago
lxxawfl c37b9ed1e9 [test][e2e] Upgrade org.postgresql:postgresql from 42.3.1 to 42.2.26 for CVE-2022-31197(#1616) 2 years ago
Leonard Xu e1ec9906ce [hotfix][typo] Fix typo from schemaLess to schemaless 2 years ago
Hang Ruan 347911c7a0 [cdc-base] Reduce the memory usage of JM by sharing table schemas between splits 2 years ago
Hang Ruan f44d2092c7
[cdc-base] Release the scan fetcher thread resource after reading finished (#1619) 2 years ago
Hang Ruan 806a8505d3
[cdc-base] Optimize pure binlog phase check logic to improve performance (#1620)
This closes #1620.
2 years ago
Hang Ruan 8df5b110ab
[cdc-base] Let all records of one snapshot split don't cross checkpoints (#1622)
This closes #1622.
2 years ago
fbad aac6e31718 [postgres] Introduce changelog mode configuration with upsert/all modes 2 years ago
fbad 9eab82a2f8 [common] Introduce changelog mode configuration with upsert/all modes 2 years ago
hehuiyuan 7b4f31b4e4
[hotfix][docs] Fix docs for scan.incremental.snapshot.chunk.size (#1617) 2 years ago
Hang Ruan 48b8c366fa
[cdc-base] Adopt streaming merge during snapshotSplit scan to avoid OOM
This closes #1618.
2 years ago
Amber Moe 0e4954b4e6 [docs][oceanbase][zh] Add video tutorials that synchronizes data from OceanBase to Elasticsearch (#1604)
This closes #1601
2 years ago
Amber Moe 500aa85d92 [docs][oceanbase] Add video tutorials that synchronizes data from OceanBase to Elasticsearch (#1605)
This closes #1605
2 years ago
Hang Ruan f41faad8dc
[mysql] Fix parse error when ALTER SQL statement contains default string value and collate. (#1544)
This closes #1506
2 years ago
leixin 569c6a525b
[docs][mysql] Add regex examples for multiple tables (#1564)
Co-authored-by: leixin1 <leixin1@jd.com>
2 years ago
He Wang aabec52fcf
[oceanbase] Shade apache commons dependency to avoid class conflict (#1532)
This closes #1352.
2 years ago
gongzhongqiang 38cc01d0eb
[postgres] add geometry type support. (#1554) 2 years ago
gongzhongqiang cdfc86f670
[build] Polish CI scripts and remove useless code (#1540)
This closes #1540.
2 years ago