CDC,schema change-Realtime Compute for Apache Flink(Flink)-阿里云帮助中心

The following legend applies to the tables in this topic:

✓ : The sink schema synchronizes this type of change.
✗ : The sink schema does not synchronize this type of change directly. The change may be synchronized in another way.
— : Not applicable. The framework does not allow the change or does not recognize it as a change.

Paimon Sink

Change event	LENIENT	EVOLVE	Description
Add column	✓	✓	Under LENIENT, the column is appended at the end and its type is changed to the nullable variant. Under EVOLVE, the corresponding column is automatically added to the result table.
Drop column	✗	✓	Under LENIENT, the column is not dropped and NULL is written for the column in subsequent data. Under EVOLVE, the column is dropped from the result table.
Rename column	✗	✓	Under LENIENT, this is handled as an add-column followed by a drop-column: the renamed column is added at the end, and existing data of the original column is automatically filled with NULL. Under EVOLVE, the original column is renamed in the result table.
Alter column type	✓	✓	The data type of the corresponding column is changed in the result table.
NOT NULL to NULLABLE	✓	✓	Paimon allows relaxing a column from NOT NULL to NULLABLE.
NULLABLE to NOT NULL	—	—	Both the framework and Paimon prohibit tightening a NULLABLE constraint. No event is generated for such a change.
Create table	✓	✓	The corresponding table is automatically created on the target side. If the table already exists, schema compatibility is validated.
Drop table	✗	✗	Under LENIENT, the drop-table change is not synchronized. Under EVOLVE, dropping the table is not supported, so the job fails.
Truncate table	✗	✗	Under LENIENT, the truncate-table change is not synchronized. Under EVOLVE, truncating the table is not supported, so the job fails.
Alter column comment	✗	✗	Changes to existing column comments on the source side are not synchronized. Comments included when a table is created or a column is added are synchronized.
Alter table comment	✗	✗	Changes to table comments on the source side are not synchronized. Table comments included when the table is created are synchronized.
Alter column order	✗	✓	Under LENIENT, the column order change is not synchronized, and the sink writes data in the original column order. Under EVOLVE, the column order change is synchronized.

Note

Paimon Sink automatically corrects column-name case: when a schema change fails due to case mismatch, it retries automatically.

Fluss Sink

Change event	LENIENT	EVOLVE	Description
Add column	✓	✓	Fluss supports adding a column only at the end (LAST position). Under LENIENT, the column is appended at the end and its type is changed to the nullable variant. Under EVOLVE, the corresponding column is automatically added to the result table. If the column is not added at the end, the job fails.
Drop column	✗	✗	Under LENIENT, the column is not dropped and NULL is written for the column in subsequent data. Under EVOLVE, dropping a column is not supported, so the job fails.
Rename column	✗	✗	Under LENIENT, this is handled as an add-column followed by a drop-column: the renamed column is added at the end, and existing data of the original column is automatically filled with NULL. Under EVOLVE, renaming a column is not supported, so the job fails.
Alter column type	✗	✗	Fluss does not support alter-column-type, so the Flink job fails.
NOT NULL to NULLABLE	✗	✗	Fluss does not support alter-column-type, so the Flink job fails.
NULLABLE to NOT NULL	—	—	The framework prohibits tightening a NULLABLE constraint. No event is generated for such a change.
Create table	✓	✓	The corresponding table and database are automatically created on the target side.
Drop table	✗	✓	Under LENIENT, the drop-table change is not synchronized. Under EVOLVE, the table is dropped on the target side.
Truncate table	✗	✗	Under LENIENT, the truncate-table change is not synchronized. Under EVOLVE, truncating the table is not supported, so the job fails.
Alter column comment	✗	✗	Changes to existing column comments on the source side are not synchronized. Comments included when a table is created or a column is added are synchronized.
Alter table comment	✗	✗	Changes to table comments on the source side are not synchronized. Table comments included when the table is created are synchronized.
Alter column order	✗	✗	Under LENIENT, the column order change is not synchronized, and the sink writes data in the original column order. Under EVOLVE, synchronizing a column order change is not supported, so the job fails.

Note

It is recommended to use the schema.change.behavior: LENIENT mode to handle schema changes not supported by Fluss more flexibly.

StarRocks Sink

Change event	LENIENT	EVOLVE	Description
Add column	✓	✓	Under LENIENT, the column is appended at the end and its type is changed to the nullable variant. Under EVOLVE, the corresponding column is automatically added to the result table.
Drop column	✗	✓	Under LENIENT, the column is not dropped and NULL is written for the column in subsequent data. Under EVOLVE, the column is dropped from the result table.
Rename column	✗	✗	Under LENIENT, this is handled as an add-column followed by a drop-column: the renamed column is added at the end, and existing data of the original column is automatically filled with NULL. Under EVOLVE, renaming a column is not supported, so the job fails.
Alter column type	✗	✗	StarRocks does not support alter-column-type, so the Flink job fails.
NOT NULL to NULLABLE	✗	✗	StarRocks does not support alter-column-type, so the Flink job fails.
NULLABLE to NOT NULL	—	—	The framework prohibits tightening a NULLABLE constraint. No event is generated for such a change.
Create table	✓	✓	The corresponding table is automatically created on the target side.
Drop table	✗	✓	Under LENIENT, the drop-table change is not synchronized. Under EVOLVE, the table is dropped on the target side.
Truncate table	✗	✓	Under LENIENT, the truncate-table change is not synchronized. Under EVOLVE, data of the table on the target side is truncated.
Alter column comment	✗	✗	Changes to existing column comments on the source side are not synchronized. Comments included when a table is created or a column is added are synchronized.
Alter table comment	✗	✗	Changes to table comments on the source side are not synchronized. Table comments included when the table is created are synchronized.
Alter column order	✗	✗	Under LENIENT, the column order change is not synchronized, and the sink writes data in the original column order. Under EVOLVE, synchronizing a column order change is not supported, so the job fails.

Note

StarRocks 3.3+ plans to support column rename and column order changes. The current connector version does not support them.

Iceberg Sink

Change event	LENIENT	EVOLVE	Description
Add column	✓	✓	Under LENIENT, the column is appended at the end and its type is changed to the nullable variant. Under EVOLVE, the corresponding column is automatically added to the result table.
Drop column	✗	✓	Under LENIENT, the column is not dropped and NULL is written for the column in subsequent data. Under EVOLVE, the column is dropped from the result table.
Rename column	✗	✓	Under LENIENT, this is handled as an add-column followed by a drop-column: the renamed column is added at the end, and existing data of the original column is automatically filled with NULL. Under EVOLVE, the original column is renamed in the result table.
Alter column type	✓	✓	The data type of the corresponding column is changed in the result table.
NOT NULL to NULLABLE	✗	✗	Iceberg does not currently support changing NOT NULL to NULLABLE.
NULLABLE to NOT NULL	—	—	The framework prohibits tightening a NULLABLE constraint. No event is generated for such a change.
Create table	✓	✓	The corresponding table and namespace are automatically created on the target side.
Drop table	✗	✗	Under LENIENT, the drop-table change is not synchronized. Under EVOLVE, dropping the table is not supported, so the job fails.
Truncate table	✗	✗	Under LENIENT, the truncate-table change is not synchronized. Under EVOLVE, truncating the table is not supported, so the job fails.
Alter column comment	✗	✗	Changes to existing column comments on the source side are not synchronized. Comments included when a table is created or a column is added are synchronized.
Alter table comment	✗	✗	Changes to table comments on the source side are not synchronized. Table comments included when the table is created are synchronized.
Alter column order	✗	✗	Under LENIENT, the column order change is not synchronized, and the sink writes data in the original column order. Under EVOLVE, synchronizing a column order change is not supported, so the job fails.

Hologres Sink

Change event	LENIENT	EVOLVE	Description
Add nullable column	✓	✓	Under LENIENT, the column is appended at the end and its type is changed to the nullable variant. Under EVOLVE, the corresponding column is automatically added to the result table.
Add non-nullable column	✓	✓	Hologres does not support adding a non-nullable column. Adding a non-nullable column is downgraded as follows: the corresponding column is appended to the end of the result table schema, the new column is nullable by default, and the column is automatically set to NULL for existing data.
Drop column	✗	✗	Hologres does not currently support dropping columns. Under LENIENT, the column is not dropped, NULL is written for the column in subsequent data, and no error is reported. Under EVOLVE, dropping a column is not supported, so the job fails.
Rename column	✗	✓	Under LENIENT, this is handled as an add-column followed by a drop-column: the renamed column is added at the end, and existing data of the original column is automatically filled with NULL. Under EVOLVE, the column is renamed directly.
Alter column type	✗	✗	Hologres does not support alter-column-type, so the Flink job fails. Use it together with `sink.type-normalize-strategy` to avoid job failures.
NOT NULL to NULLABLE	✗	✗	Hologres does not support alter-column-type, so the Flink job fails.
NULLABLE to NOT NULL	—	—	The framework prohibits tightening a NULLABLE constraint. No event is generated for such a change.
Create table	✓	✓	The corresponding table is automatically created on the target side.
Drop table	✗	✓	Under LENIENT, the drop-table change is not synchronized. Under EVOLVE, the table is dropped on the target side.
Truncate table	✗	✓	Under LENIENT, the truncate-table change is not synchronized. Under EVOLVE, data of the table on the target side is truncated.
Alter column comment	✗	✗	Changes to existing column comments on the source side are not synchronized. Comments included when a table is created or a column is added are synchronized.
Alter table comment	✗	✗	Changes to table comments on the source side are not synchronized. Table comments included when the table is created are synchronized.
Alter column order	✗	✗	Under LENIENT, the column order change is not synchronized, and the sink writes data in the original column order. Under EVOLVE, synchronizing a column order change is not supported, so the job fails.

Note

Hologres Sink does not support the TRY_EVOLVE strategy. When adding a column, only appending at the end (LAST position) is supported, and adding a non-NULLABLE column is not supported. It also does not synchronize changes to constraints such as primary keys or indexes, dropping non-null columns, or the NOT NULL to NULLABLE change.

SelectDB Sink

Change event	LENIENT	EVOLVE	Description
Add column	✓	✓	Under LENIENT, the column is appended at the end and its type is changed to the nullable variant. Under EVOLVE, the corresponding column is automatically added to the result table.
Drop column	✗	✓	Under LENIENT, the column is not dropped and NULL is written for the column in subsequent data. Under EVOLVE, the column is dropped from the result table.
Rename column	✗	✓	Under LENIENT, this is handled as an add-column followed by a drop-column: the renamed column is added at the end, and existing data of the original column is automatically filled with NULL. Under EVOLVE, the original column is renamed in the result table.
Alter column type	✓	✓	The data type of the corresponding column is changed in the result table.
NOT NULL to NULLABLE	✗	✗	When the MODIFY COLUMN statement does not explicitly specify NULL, Doris keeps the existing NULLABLE constraint unchanged.
NULLABLE to NOT NULL	—	—	The framework prohibits tightening a NULLABLE constraint. No event is generated for such a change.
Create table	✓	✓	The corresponding table is automatically created on the target side.
Drop table	✗	✓	Under LENIENT, the drop-table change is not synchronized. Under EVOLVE, the table is dropped on the target side.
Truncate table	✗	✓	Under LENIENT, the truncate-table change is not synchronized. Under EVOLVE, data of the table on the target side is truncated.
Alter column comment	✗	✗	Changes to existing column comments on the source side are not synchronized. Comments included when a table is created or a column is added are synchronized.
Alter table comment	✗	✗	Changes to table comments on the source side are not synchronized. Table comments included when the table is created are synchronized.
Alter column order	✗	✗	Under LENIENT, the column order change is not synchronized, and the sink writes data in the original column order. Under EVOLVE, synchronizing a column order change is not supported, so the job fails.

Note

In batch mode (sink.enable-batch-mode: true), ALTER TABLE-type schema changes (add column, drop column, rename column, alter column type) wait for the schema change to complete before data processing resumes. The timeout is configured through schema.evolution.timeout. SelectDB is built on the Apache Doris engine.

MaxCompute Sink

Change event	LENIENT	EVOLVE	Description
Add column	✓	✓	Under LENIENT, the column is appended at the end and its type is changed to the nullable variant. Under EVOLVE, the corresponding column is automatically added to the result table.
Drop column	✗	✓	Under LENIENT, the column is not dropped and NULL is written for the column in subsequent data. Under EVOLVE, the column is dropped from the result table.
Rename column	✗	✓	Under LENIENT, this is handled as an add-column followed by a drop-column: the renamed column is added at the end, and existing data of the original column is automatically filled with NULL. Under EVOLVE, the original column is renamed in the result table.
Alter column type	✓	✓	The data type of the corresponding column is changed in the result table.
NOT NULL to NULLABLE	✓	✓	Relaxing the column constraint from NOT NULL to NULLABLE is synchronized to the target side.
NULLABLE to NOT NULL	—	—	The framework prohibits tightening a NULLABLE constraint. No event is generated for such a change.
Create table	✓	✓	The corresponding table is automatically created on the target side. If the table already exists, schema and primary key consistency are validated. An inconsistency causes the job to fail.
Drop table	✗	✓	Under LENIENT, the drop-table change is not synchronized. Under EVOLVE, the table is dropped on the target side.
Truncate table	✗	✓	Under LENIENT, the truncate-table change is not synchronized. Under EVOLVE, data of the table on the target side is truncated.
Alter column comment	✗	✗	Changes to existing column comments on the source side are not synchronized. Comments included when a table is created or a column is added are synchronized.
Alter table comment	✗	✗	Changes to table comments on the source side are not synchronized. Table comments included when the table is created are synchronized.
Alter column order	✗	✗	Under LENIENT, the column order change is not synchronized, and the sink writes data in the original column order. Under EVOLVE, synchronizing a column order change is not supported, so the job fails.

Kafka Sink

Change event	LENIENT	EVOLVE	Description
Create table	✓	✓	The corresponding topic is automatically created in Kafka (a compacted topic is created in Upsert mode).
Add column	✓	✓	Kafka does not maintain a schema. Under LENIENT, data of the new column is appended at the end. Under EVOLVE, data of the new column is added in the specified column order.
Drop column	✗	✓	Kafka does not maintain a schema. Under LENIENT, NULL is written for the dropped column in subsequent data. Under EVOLVE, the dropped column stops being written.
Rename column	✗	✓	Kafka does not maintain a schema. Under LENIENT, this is handled as an add-column followed by a drop-column: the renamed column is added at the end, and existing data of the original column is automatically filled with NULL. Under EVOLVE, data is synchronized according to the changed schema.
Alter column type	✓	✓	Kafka does not maintain a schema. Subsequent data is written in the changed type.
NOT NULL to NULLABLE	✓	✓	Kafka does not maintain a schema. No extra handling is performed for the NOT NULL to NULLABLE change.
NULLABLE to NOT NULL	—	—	The framework prohibits tightening a NULLABLE constraint. No event is generated for such a change.
Drop table	✗	✗	The drop-table change is not handled.
Truncate table	✗	✗	The truncate-table change is not handled.
Alter column comment	✗	✗	The alter-column-comment change is not handled.
Alter table comment	✗	✗	The alter-table-comment change is not handled.
Alter column order	✓	✓	Kafka does not maintain a schema. Under LENIENT, the column order change is not synchronized, and the sink writes data in the original column order. Under EVOLVE, data is synchronized according to the changed schema.

Note

As a message queue, Kafka does not maintain a table schema. Schema change events are delivered downstream as part of the message payload, but no actual schema change operations are performed on the Kafka side.