ApsaraMQ for Kafka allows you to synchronize message data from a source Kafka instance to a target Kafka instance. You can create synchronization tasks in same-account or cross-account scenarios, and configure the topics, compute resources, task concurrency, compression type, and offset reset strategy as needed. Data synchronization is designed for long-term inter-instance data transfer scenarios such as data distribution, backup, and disaster recovery.
Usage limits
This feature is currently available by request. To enable it, submit a support ticket.
Billing
Data synchronization is billed based on CU. For details, see CU Postpaid.
If cross-account or cross-VPC data synchronization is involved, you must also pay VPC data transfer fees. For details, see VPC data transfer postpaid.
Prerequisites
Before creating a cross-account data synchronization task, ensure that the source or target instance account has completed authorization. For more information, see Authorization management. After authorization, the authorized account can select the corresponding instance as the source or target when creating a synchronization task.
Create a data synchronization task
-
Log on to the ApsaraMQ for Kafka console.
-
In the left-side navigation pane, select Data Synchronization.
-
On the Data Synchronization page, click Create Task.
-
On the Create Synchronization Task page, complete the configuration as instructed.
Step 1: Configure basic information
|
Parameter |
Description |
|
Task name |
Enter a task name. We recommend that you use a name that identifies the business purpose, source instance, and target instance to simplify O&M and troubleshooting. |
After configuration, click Next.
Step 2: Configure the source service
|
Parameter |
Description |
|
Cross-account instance |
Specify whether the source instance belongs to another Alibaba Cloud account. If yes, the source instance account must complete authorization in advance. |
|
Source cluster instance |
Select the source Kafka instance to read data from. |
|
Sync all Topics |
Specify whether to synchronize all topics from the source instance. If you select No, specify the topics to synchronize as prompted. |
|
CU |
Select the compute resource specification. A higher CU value generally provides stronger synchronization capability. For details, see the specifications on the page. |
|
Number of tasks |
Select the number of concurrent synchronization tasks. |
|
Compression type |
Select the compression type for messages written to the target instance. Choose based on your compatibility requirements and compression efficiency. |
|
Message offset reset strategy |
Select the message offset reset strategy for the task. |
After configuration, click Next.
Step 3: Configure the target service
|
Parameter |
Description |
|
Target cluster |
Select the target Kafka instance to receive the synchronized data. If the target instance belongs to another account, that account must complete authorization in advance. |
After confirming the configuration, click Create Synchronization Task.
Next steps
After the task is created, you can view its status, synchronization progress, and runtime details on the Data Synchronization page. We recommend that you verify the topics, message writing status, and consumption chain in the target instance meet expectations before switching traffic.