d0cce26b76
[cdc-pipeline-connector][mysql] Introduce mysql cdc pipeline DataSource
1 year ago
a328f9895a
[cdc-cli][cdc-composer][cdc-dist] Support submitting job to general Flink environments ( #2807 )
...
This closes #2807 .
1 year ago
9ce8450cf2
[cdc-runtime][hotfix] Setup waitFlushSuccess before responding operator to flush. ( #2805 )
...
This closes #2805 .
There might be a case that sink flushes faster than schema operator requesting release upstream. In that case waitFlushSuccess has not been set when sink has flushed successfully, which will lead to a NullPointerException.
1 year ago
95921d5620
[cdc-common] Introduce "pipeline.local-time-zone" config option which help handle time zone well ( #2797 )
...
This closes #2797 .
1 year ago
e435004eda
[cdc-common][hotfix] Remove useless flink imports ( #2803 )
...
This closes #2803 .
1 year ago
cfb3fd206e
[build] Let surefire plugin to keep the full stacktrace on test failure ( #2804 )
...
This closes #2804 .
1 year ago
b60322bef7
[pipeline-common] Use column name to check whether the schema contains specific column or not. ( #2801 )
...
This closes #2801 .
1 year ago
3bf1428233
[cdc-composer][route][sink] Add name for route and improve name for sink
1 year ago
a8f4ba8cc6
[cdc-composer][hotfix] Always chain operator by default
1 year ago
f0c630d6f6
[cdc-composer][sink] Use correct name for sink operator
1 year ago
4843ab2432
[cdc-runtime] SchemaRegistry should complete the future after making the checkpoint ( #2800 )
...
This closes #2800 .
1 year ago
2d9dc018c5
[cdc-connector][sqlserver][tests] Fix UT errors by increasing the wait time after committing SQL ( #2799 )
...
This closes #2799 .
1 year ago
173deb5248
[cdc-composer] Add common and runtime JAR when submit ( #2795 )
...
This closes #2795 .
1 year ago
5f318346ff
[cdc-connector][debezium] Supports conversion from binary bytes to base64 encoded string ( #2435 )
...
This closes #2435 .
1 year ago
d1c3f8500b
[cdc-connector][mysql] MysqlTableSource return metadata in a fixed order ( #2578 )
1 year ago
72d33c8965
[cdc-connector][mysql] Fix snapshot fetch size conf does not take effect ( #2766 )
1 year ago
5c00f76d0f
[cdc-connector][oracle] Expose ability to skip backfill in snapshot period for Oracle CDC ( #2793 )
...
This closes #2793 .
1 year ago
ef1c095763
[cdc-connector][oracle] Fix wrong table path during backfill phase which leads to potential data loss ( #2707 )
...
This closes #2707 .
1 year ago
cc9d368b63
[cdc-composer][tests] Add integration tests for FlinkPipelineComposer ( #2776 )
...
This closes #2766 .
1 year ago
100fb4a59e
[cdc-dist] Introduce flink-cdc.sh as the entrypoint of the Flink CDC ( #2791 )
1 year ago
1a5065e774
[3.0] [flink-cdc-dist] Collect all distro resources into a TAR ( #2636 )
...
This closes #2621 #2622
Co-authored-by: Qingsheng Ren <renqschn@gmail.com>
1 year ago
f57d6f6e67
[cdc-connector][sqlserver] Expose ability to skip backfill in snapshot period for SqlServer CDC.( #2783 )
...
This closes #2783 .
1 year ago
a6379358e4
[cdc-connector][mysql] Expose ability to skip backfill in snapshot period for MySQL CDC.
1 year ago
e7c713eca6
[cdc-connector][mongodb] Expose ability to skip backfill in snapshot period for Mongodb CDC.
1 year ago
c6cb3b2371
[cdc-connector][postgres] Expose ability to skip backfill in snapshot period for Postgres CDC.
1 year ago
4e248fad33
[cdc-connector][base] Support skip snapshot backfill for incremental snapshot connector
1 year ago
13be14fe3f
[cdc-connector][base][tests] Introduce snapshot phase hooks for better snapshot test
1 year ago
a771abef90
[cdc-composer] Add routing logic into the Flink composer
1 year ago
f22ac3e788
[cdc-cli][cdc-composer] Use sourceTable and sinkTable as keyword in route definition
1 year ago
416562c9ba
[cdc-runtime] Add RouteFunction for routing change events
1 year ago
2bde0beaec
[cdc-common] Introduce AssertJ-style assertion system for Flink CDC data structures
1 year ago
6eb980a614
[hotfix][mysql][tests] Avoid canceling a finished job in tests of SpecificStartingOffsetITCase ( #2687 )
...
This closes #2687 .
1 year ago
f7dda3b67b
[docs][zh][hotfix] Remove redundant sections from mysql-cdc documentation ( #2777 )
...
This closes #2777 .
1 year ago
cc9dbc6660
[mongodb][hotfix] Fix MongoDB's rewriteOutputBuffer to emit +I rather than +U ( #2760 ) ( #2760 )
1 year ago
a81d4a16b9
[cdc-common] Introduce createFieldGetters method in SchemaUtils to build FieldGetters of given Schema. ( #2762 )
...
This closes #2762 .
1 year ago
99ffbe03d2
[cdc-composer] use DataSinkWriterOperatorFactory to replace all SinkWriterOperatorFactory. ( #2773 )
1 year ago
8edf2a60f0
[cdc-pipeline-connector][values] Avoid using `default` as name of namespace and schema. ( #2774 )
1 year ago
da26c36d50
[cdc-runtime] Improve DataSinkWriterOperator logic that only emit latest schema for events that are not CreateTableEvent ( #2771 )
1 year ago
3806c72ef8
[cdc-common] add PublicEvolving annotation to SchemaUtils ( #2756 )
1 year ago
0cdfd36c08
[3.0][cdc-composer] Introduce pipeline.name config to support custom flink job name ( #2768 )
...
Co-authored-by: Leonard Xu <leonard@apache.org>
This closes #2769 .
1 year ago
57e4793b21
[cdc-base] Extract the common parts of each ScanFetchTask into AbstractScanFetcherTask.
...
This closes #2690 .
1 year ago
369be8c035
[docs][hotfix] Configuration 'slot.name' is required in mysql-postgres-tutorial-zh.md ( #2763 )
...
This closes #2763 .
1 year ago
a02454572f
[cdc-runtime] Improve partitioning logics and add tests for PrePartitionOperator( #2761 )
...
This closes #2761 .
1 year ago
1dae8674d4
[cdc-runtime][tests] Introduce Harness test utilities for CDC customized operators
1 year ago
ab3f7d8135
[cdc-runtime] Make sure CreateTableEvent is always sent before DataChangeEvent even if during restoration ( #2767 )
...
This closes #2767 .
1 year ago
c853514a0d
[docs][hotfix] Configuration 'slot.name' is required in mysql-postgres-tutorial.md ( #2764 )
...
This closes #2764 .
1 year ago
6e53733df2
[oceanbase] Use global timestamp to start log client ( #2565 )
...
* use global timestamp to start log client
* remove redundant fields
* update log info
1 year ago
4a5f2161aa
[cdc-pipeline-connector][values] Add event set MULTI_SPLITS_SINGLE_TABLE to mock source with multiple splits ( #2737 )
...
This closes #2737 .
1 year ago
071523eae6
[hotfix][cdc-composer] Use correct factory to discovery DataSink ( #2757 )
...
This closes #2757 .
1 year ago
70614be28c
[3.0][cdc-common] Introduce BinaryRecordDataGenerator to help to create BinaryRecordData
...
This closes #2734 .
1 year ago
Jiabao Sun
Qingsheng Ren
Leonard Xu
Hang Ruan
Kunni
Xin Gong
gongzhongqiang
Hongshun Wang
yunqingmo
ice
Hongshun Wang
l568288g
He Wang