Flink CDC pipeline module

Parameter

Description

Required

Data type

Default value

Note

name

The name of the Flink CDC data ingestion job.

No

STRING

Flink CDC Pipeline Job

-

schema.change.behavior

The strategy to handle schema changes.

No

STRING

LENIENT

Valid values are listed below. For more information, see Schema evolution configurations.

• LENIENT (default)

• EXCEPTION

• EVOLVE

• TRY_EVOLVE

• IGNORE

schema-operator.rpc-timeout

The timeout period for SchemaOperator to wait for the completion of a downstream schema change event application.

No

DURATION

3min

The timeout period for the schema change coordination process.

local-time-zone

The local time zone for job execution.

No

STRING

System time zone

This setting affects time-related calculations in the transform module.

user-defined-function

The user-defined functions (UDFs) to register for the job.

No

LIST

-

For more information, see User-defined functions.

transform.allow.trimming.pk-columns

Allows transform rules to reduce upstream primary key constraints

No

BOOLEAN

false

Removing upstream PK constraints can cause multiple data entries to merge, potentially leading to data losses. Before using this setting, ensure the uniqueness of the new PK (if defined).

transform.decimal.precision.mode

Specifies the maximum Decimal precision used by the Transform module.

No

STRING

UP_TO_19

Valid values:

• UP_TO_19: The maximum precision of the Decimal type returned by expressions is 19. This is the default behavior in VVR 11.6 and earlier.

• UP_TO_38: The maximum precision of the Decimal type returned by expressions is 38.

sink.partitioning.strategy

Specifies the partitioning strategy that distributes data change events to the downstream subtasks of the sink.

No

STRING

SINK_DEFINED

Valid values:

• SINK_DEFINED: Uses the partitioning strategy defined by the sink. This is the default value. The Paimon, Fluss, and MaxCompute sinks support only SINK_DEFINED.

• PRIMARY_KEY: Partitions events by table ID and primary key value. Events with the same primary key are routed to the same subtask.

• TABLE_ID: Partitions events by table ID. All events of the same table are routed to the same subtask.