2d9dc018c5
[cdc-connector][sqlserver][tests] Fix UT errors by increasing the wait time after committing SQL ( #2799 )
...
This closes #2799 .
1 year ago
173deb5248
[cdc-composer] Add common and runtime JAR when submit ( #2795 )
...
This closes #2795 .
1 year ago
5f318346ff
[cdc-connector][debezium] Supports conversion from binary bytes to base64 encoded string ( #2435 )
...
This closes #2435 .
1 year ago
d1c3f8500b
[cdc-connector][mysql] MysqlTableSource return metadata in a fixed order ( #2578 )
1 year ago
72d33c8965
[cdc-connector][mysql] Fix snapshot fetch size conf does not take effect ( #2766 )
1 year ago
5c00f76d0f
[cdc-connector][oracle] Expose ability to skip backfill in snapshot period for Oracle CDC ( #2793 )
...
This closes #2793 .
1 year ago
ef1c095763
[cdc-connector][oracle] Fix wrong table path during backfill phase which leads to potential data loss ( #2707 )
...
This closes #2707 .
1 year ago
cc9d368b63
[cdc-composer][tests] Add integration tests for FlinkPipelineComposer ( #2776 )
...
This closes #2766 .
1 year ago
100fb4a59e
[cdc-dist] Introduce flink-cdc.sh as the entrypoint of the Flink CDC ( #2791 )
1 year ago
1a5065e774
[3.0] [flink-cdc-dist] Collect all distro resources into a TAR ( #2636 )
...
This closes #2621 #2622
Co-authored-by: Qingsheng Ren <renqschn@gmail.com>
1 year ago
f57d6f6e67
[cdc-connector][sqlserver] Expose ability to skip backfill in snapshot period for SqlServer CDC.( #2783 )
...
This closes #2783 .
1 year ago
a6379358e4
[cdc-connector][mysql] Expose ability to skip backfill in snapshot period for MySQL CDC.
1 year ago
e7c713eca6
[cdc-connector][mongodb] Expose ability to skip backfill in snapshot period for Mongodb CDC.
1 year ago
c6cb3b2371
[cdc-connector][postgres] Expose ability to skip backfill in snapshot period for Postgres CDC.
1 year ago
4e248fad33
[cdc-connector][base] Support skip snapshot backfill for incremental snapshot connector
1 year ago
13be14fe3f
[cdc-connector][base][tests] Introduce snapshot phase hooks for better snapshot test
1 year ago
a771abef90
[cdc-composer] Add routing logic into the Flink composer
1 year ago
f22ac3e788
[cdc-cli][cdc-composer] Use sourceTable and sinkTable as keyword in route definition
1 year ago
416562c9ba
[cdc-runtime] Add RouteFunction for routing change events
1 year ago
2bde0beaec
[cdc-common] Introduce AssertJ-style assertion system for Flink CDC data structures
1 year ago
6eb980a614
[hotfix][mysql][tests] Avoid canceling a finished job in tests of SpecificStartingOffsetITCase ( #2687 )
...
This closes #2687 .
1 year ago
f7dda3b67b
[docs][zh][hotfix] Remove redundant sections from mysql-cdc documentation ( #2777 )
...
This closes #2777 .
1 year ago
cc9dbc6660
[mongodb][hotfix] Fix MongoDB's rewriteOutputBuffer to emit +I rather than +U ( #2760 ) ( #2760 )
1 year ago
a81d4a16b9
[cdc-common] Introduce createFieldGetters method in SchemaUtils to build FieldGetters of given Schema. ( #2762 )
...
This closes #2762 .
1 year ago
99ffbe03d2
[cdc-composer] use DataSinkWriterOperatorFactory to replace all SinkWriterOperatorFactory. ( #2773 )
1 year ago
8edf2a60f0
[cdc-pipeline-connector][values] Avoid using `default` as name of namespace and schema. ( #2774 )
1 year ago
da26c36d50
[cdc-runtime] Improve DataSinkWriterOperator logic that only emit latest schema for events that are not CreateTableEvent ( #2771 )
1 year ago
3806c72ef8
[cdc-common] add PublicEvolving annotation to SchemaUtils ( #2756 )
1 year ago
0cdfd36c08
[3.0][cdc-composer] Introduce pipeline.name config to support custom flink job name ( #2768 )
...
Co-authored-by: Leonard Xu <leonard@apache.org>
This closes #2769 .
1 year ago
57e4793b21
[cdc-base] Extract the common parts of each ScanFetchTask into AbstractScanFetcherTask.
...
This closes #2690 .
1 year ago
369be8c035
[docs][hotfix] Configuration 'slot.name' is required in mysql-postgres-tutorial-zh.md ( #2763 )
...
This closes #2763 .
1 year ago
a02454572f
[cdc-runtime] Improve partitioning logics and add tests for PrePartitionOperator( #2761 )
...
This closes #2761 .
1 year ago
1dae8674d4
[cdc-runtime][tests] Introduce Harness test utilities for CDC customized operators
1 year ago
ab3f7d8135
[cdc-runtime] Make sure CreateTableEvent is always sent before DataChangeEvent even if during restoration ( #2767 )
...
This closes #2767 .
1 year ago
c853514a0d
[docs][hotfix] Configuration 'slot.name' is required in mysql-postgres-tutorial.md ( #2764 )
...
This closes #2764 .
1 year ago
6e53733df2
[oceanbase] Use global timestamp to start log client ( #2565 )
...
* use global timestamp to start log client
* remove redundant fields
* update log info
1 year ago
4a5f2161aa
[cdc-pipeline-connector][values] Add event set MULTI_SPLITS_SINGLE_TABLE to mock source with multiple splits ( #2737 )
...
This closes #2737 .
1 year ago
071523eae6
[hotfix][cdc-composer] Use correct factory to discovery DataSink ( #2757 )
...
This closes #2757 .
1 year ago
70614be28c
[3.0][cdc-common] Introduce BinaryRecordDataGenerator to help to create BinaryRecordData
...
This closes #2734 .
1 year ago
cad7424a1d
[3.0][cdc-common] Use BinaryRecordData in DataChangeEvent instead of GenericRecordData
1 year ago
ac9bab3dce
[3.0][cdc-common] Introduce binaryRecordData to avoid type serialization and improve performance as well
...
This closes #2748 .
1 year ago
6a92546340
[build][hotfix] Use the correct format for labeler.yml ( #2758 )
...
This closes #2758 .
1 year ago
30031d0a14
[build] Add github labeler for pull request
...
This closes #2564 .
1 year ago
0bf378dd22
[docs] Add DataStream application package guidance ( #2583 )
...
This closes #2583 .
Co-authored-by: Leonard Xu <leonard@apache.org>
1 year ago
9c3c7faae4
[cdc-common] Introduce TableFilter to filter table objects from given TableId pattern ( #2673 )
...
This closes #2669 .
1 year ago
a26607a027
[cdc-composer] Initialize context with existing config of SourceDef and SinkDef ( #2749 )
...
This closes #2749 .
1 year ago
f697c5fff3
[hotfix][docs] Adjust the correct doc versions ( #2679 )
...
This closes #2679 .
1 year ago
9aed54d70f
[docs][oracle] Fix oracle cdc quick start ( #2727 )
...
This closes #2727 .
1 year ago
8ee3ea52d5
[cdc-runtime] Use CollectCoordinationResponse as a wrapper of CoordinationResponse
...
The reason we do this is that there is potentially a bug in Flink RPC system, that CoordinationResponse can only be deserialized by AppClassloader instead of user code classloader, so we can't use customize CoordinationResponse class in RPCs (will lead to ClassNotFoundException). As CollectCoordinationResponse is predefined in Flink and shipped in flink-dist, it will be always visible to AppClassloader, and we use its payload for holding the actual serialized custom CoordinationResponse.
This closes #2744
1 year ago
1973f64aa3
[cdc-common] MetadataApplier should extends Serializable
...
This closes #2743
1 year ago
Xin Gong
Qingsheng Ren
Hang Ruan
gongzhongqiang
Hongshun Wang
yunqingmo
ice
Hongshun Wang
Kunni
l568288g
He Wang
Kunni
skylines