4a5f2161aa
[cdc-pipeline-connector][values] Add event set MULTI_SPLITS_SINGLE_TABLE to mock source with multiple splits ( #2737 )
...
This closes #2737 .
1 year ago
071523eae6
[hotfix][cdc-composer] Use correct factory to discovery DataSink ( #2757 )
...
This closes #2757 .
1 year ago
70614be28c
[3.0][cdc-common] Introduce BinaryRecordDataGenerator to help to create BinaryRecordData
...
This closes #2734 .
1 year ago
cad7424a1d
[3.0][cdc-common] Use BinaryRecordData in DataChangeEvent instead of GenericRecordData
1 year ago
ac9bab3dce
[3.0][cdc-common] Introduce binaryRecordData to avoid type serialization and improve performance as well
...
This closes #2748 .
1 year ago
6a92546340
[build][hotfix] Use the correct format for labeler.yml ( #2758 )
...
This closes #2758 .
1 year ago
30031d0a14
[build] Add github labeler for pull request
...
This closes #2564 .
1 year ago
0bf378dd22
[docs] Add DataStream application package guidance ( #2583 )
...
This closes #2583 .
Co-authored-by: Leonard Xu <leonard@apache.org>
1 year ago
9c3c7faae4
[cdc-common] Introduce TableFilter to filter table objects from given TableId pattern ( #2673 )
...
This closes #2669 .
1 year ago
a26607a027
[cdc-composer] Initialize context with existing config of SourceDef and SinkDef ( #2749 )
...
This closes #2749 .
1 year ago
f697c5fff3
[hotfix][docs] Adjust the correct doc versions ( #2679 )
...
This closes #2679 .
1 year ago
9aed54d70f
[docs][oracle] Fix oracle cdc quick start ( #2727 )
...
This closes #2727 .
1 year ago
8ee3ea52d5
[cdc-runtime] Use CollectCoordinationResponse as a wrapper of CoordinationResponse
...
The reason we do this is that there is potentially a bug in Flink RPC system, that CoordinationResponse can only be deserialized by AppClassloader instead of user code classloader, so we can't use customize CoordinationResponse class in RPCs (will lead to ClassNotFoundException). As CollectCoordinationResponse is predefined in Flink and shipped in flink-dist, it will be always visible to AppClassloader, and we use its payload for holding the actual serialized custom CoordinationResponse.
This closes #2744
1 year ago
1973f64aa3
[cdc-common] MetadataApplier should extends Serializable
...
This closes #2743
1 year ago
d5981c9acb
[cdc-runtime] Rewrite DataSinkWriterOperator to avoid classloading issues using reflection
...
When the CDC job runs on Flink cluster, DataSinkWriterOperator is
loaded by user code classloader, while SinkWriterOperator is loaded
by app classloader, and the protection mechanism forbids us from
accessing SinkWriterOperator across different classloaders.
Therefore we have to manually load the SinkWriterOperator in user
code classloader, and use reflections to control it.
This closes #2742
Co-authored-by: lvyanquan <decq12ybhl@gmail.com>
1 year ago
f0bf38e0fe
[cdc-composer] Implementation of Flink pipeline composer for chaining everything together
...
This closes #2609
1 year ago
b52e88a43f
[cdc-composer] Introduce partitioning related runtime functions and translator
1 year ago
bf5914e6ac
[cdc-composer] Introduce data sink and schema operator translators
...
This closes #2618
1 year ago
9e64b8f0b6
[cdc-composer] Introduce DataSourceTranslator for composing source
...
This closes #2617
1 year ago
7ab1960466
[cdc-runtime] Introduce schema manager for managing schema version and handling schema changes
...
This closes #2741
1 year ago
5f0343e29e
[cdc-composer] Add FlinkEnvironmentUtils for adding JAR to StreamExecutionEnvironment
1 year ago
32de7776d2
[cdc-common] Define pipeline options and schema change behaviors
...
This closes #2740
1 year ago
6506ba788d
[cdc-common] Support adding Column directly in schema builder
...
This closes #2739
1 year ago
51554d8630
[hotfix][vitess] Fix invalidate package name
...
This closes #2735 .
1 year ago
b4deda97be
[cdc-pipeline-connector][values] Introduce `values` cdc pipeline connector
...
This closes #2675 .
1 year ago
e88bea3514
[3.0][cdc-common] Introduce TableSchemaState and its Serializer for DataSink to hold the table schemas.
1 year ago
40184ae19f
[3.0][cdc-common][minor] Add toString method for some classes.
1 year ago
be286110f1
[3.0][cdc-common] Add SchemaUtil to perform schema evolution ( #2732 )
...
This closes #2732 .
1 year ago
ace6080c93
[3.0][cdc-runtime] Provide SchemaOperator and SchemaRegistry to handle schema changes ( #2685 )
1 year ago
4770f360d0
[sqlserver] Fix timestamp type parse of sqlserver ( #2708 )
1 year ago
6c21046382
[3.0][cdc-runtime] Add TypeInformation for Event class ( #2728 )
1 year ago
70bbac99d5
[3.0][cdc-runtime][tests] Add tests for type serializers
...
This closes #2713 .
1 year ago
5223d6b69d
[3.0][cdc-runtime][hotfix] Move type serializers to runtime module
1 year ago
74f032aad6
[3.0][cdc-common] Introduce data type for record field in DataChangeEvent
1 year ago
fb9a8a4257
[3.0][cdc-common][minor] Improve public methods for schema event
1 year ago
9e149efcc1
[3.0][cdc-common][minor] Move FlushEvent to common module
1 year ago
18936a3676
[mongodb] Incremental snapshot source supports comma-separated format parameter for databases and collections ( #2724 )
1 year ago
3a80de7676
[3.0][cdc-common] Use built-in ConfigOption for cdc-common module ( #2716 )
...
This closes #2716 .
1 year ago
a1b67a7585
[3.0][cdc-common] Add ConText for Source/Sink initialization. ( #2672 )
1 year ago
af5a0cef3f
[3.0][cdc-common] Add TypeSerializer implementations ( #2683 )
1 year ago
00f8156671
[build] Add header copyright check for java files
1 year ago
dab7ba4cb5
[common] Introduce Configuration and related utilities ( #2681 )
1 year ago
7192e5300b
[oracle] Skip unsupported nested table for legacy debezium source function
1 year ago
0565bf844c
[oracle] Skip unsupported nested table for Incremental OracleSource
1 year ago
995d524416
[3.0][common] Use built-in API annotations to make common module free with flink
...
This closes #2695 .
1 year ago
ce4621a549
[3.0][common] Introduce built-in API annotations for project
...
This closes #2693 .
1 year ago
1e6b983bdd
[build] Bump Flink version to 1.18.0 ( #2463 )
...
This closes #2670 .
1 year ago
07818221d2
[build][tests] Enable CI tests for CDC 3.0 modules ( #2678 )
...
This closes #2671
1 year ago
4930698a9b
[3.0][cdc-common] Introduce internal data structures and generic implementations ( #2688 )
...
This closes #2651 .
1 year ago
98c929e6e4
[composer] Connector factory and JAR discovery utilities ( #2662 )
1 year ago
Kunni
Hang Ruan
Kunni
skylines
gongzhongqiang
Qingsheng Ren
Hongshun Wang
Shawn Huang
joyCurry30
Leonard Xu
zyi728
Maciej Bryński