DataWorks now offers Serverless resource groups, which allow you to perform data synchronization, task scheduling and execution, and API service management from a single resource group. To help you leverage the elasticity and maintenance-free benefits of Serverless resource groups, DataWorks provides a one-click migration tool to seamlessly upgrade your existing exclusive Data Integration resource groups to Serverless resource groups.
This feature is in a canary release. To use this feature, contact technical support to add your account to the allowlist.
Migration process
The migration process involves a four-day canary period to gradually shift task traffic from the original exclusive resource group to the new Serverless resource group.
-
Confirm migration: The system creates a new Serverless resource group based on the configuration of the original exclusive resource group.
-
Connectivity test: Before the canary release begins, you must run a connectivity test between your data sources and the new resource group.
-
Canary decision: After the network test passes and you confirm the start of the canary release, the system enters a four-day canary period. During this period, task traffic is gradually shifted to the new resource group in increments of 20%, 50%, and finally 100%. You can choose to complete the switch immediately or stop the migration at any time. If you take no action during the canary period, the system automatically completes the full traffic shift and releases the original exclusive resource group after four days.
Impact and limitations
Before you start the migration, carefully read the following limitations and risk warnings:
-
Type limitation: This feature only supports migrating an exclusive resource group for Data Integration to a Serverless resource group.
-
Resource group lock: After the migration starts, the original exclusive resource group for Data Integration is locked. You cannot modify its VPC, edit host mappings, or make other changes until the migration is completed or stopped. After the migration is complete, the name, remarks, and attached VPC network configurations of the original resource group are inherited by the new resource group. No reconfiguration is required.
-
Real-time task interruption: During the final cutover phase, real-time synchronization tasks running on the resource group may be interrupted for several minutes. Evaluate the potential business impact and choose a suitable time for the migration.
-
Irreversible operation: Once the migration starts, it enters a four-day canary period. After this period, the resource group is permanently converted to a Serverless resource group and cannot be rolled back to an exclusive resource group.
-
Network requirements: A Serverless resource group must be attached to a VPC for network access. For details about network connectivity configuration, see Network connectivity configuration.
Billing
-
During migration: The original exclusive resource group continues to be billed based on its subscription method. The newly created Serverless resource group does not incur any charges.
-
After migration: The new Serverless resource group is billed based on the billing method (subscription or pay-as-you-go) you selected during the migration.
-
If you select the subscription billing method and the payment fails due to an insufficient account balance, the billing method automatically switches to pay-as-you-go.
-
In addition to resource usage fees, Serverless resource groups also incur task scheduling fees based on the number of instances that run successfully each day. For more information, see Task scheduling billing.
-
-
If the original exclusive resource group used host mappings, the PrivateZone service is automatically enabled on the designated VPC after the migration, and the corresponding DNS records are created. For information about related billing, see Product Billing.
Migration status transitions
|
Status |
Description |
Next step or trigger |
|
Not migrated |
The original exclusive resource group is running normally. |
User action: Click Migrate to Serverless Resource Group. |
|
Creating migration resource group |
The system is creating a new Serverless resource group based on the original configuration. |
System trigger: After the resource group is created, the status automatically changes to Pending start. |
|
Pending Migration Start |
The new Serverless resource group is created. You must run a connectivity test before you can proceed. |
User action: |
|
Starting canary release |
The canary release is confirmed. The system is preparing to shift traffic. |
System trigger: After preparations are complete, the status changes to Migrating (Canary release) X%. |
|
Migrating (Canary release) X% |
The resource group has entered the four-day canary period. Each day, the system automatically shifts a greater percentage of task traffic to the new resource group. |
User action (available anytime): |
|
Skipping canary release (Full migration) |
You chose to skip the canary period and migrate immediately. |
Wait for the migration to complete. |
|
Completing migration |
The final cutover has been triggered by you or the system. The system is finishing the traffic shift and releasing the original resource group. |
After the cutover is complete, the status changes to Running. |
|
Running |
The migration is complete. The original exclusive resource group has been released, and the new Serverless resource group handles all tasks. |
The resource group is now a Serverless resource group and is operating normally. |
Procedure
The migration has three phases: initiation, execution, and management.
Phase 1: Initiate migration
-
Go to the migration entry point
-
Log on to the DataWorks console, go to the resource group list, and switch to the target region.
-
In the Resource Groups, find the target exclusive resource group. Click the
icon in the Actions column and select Migrate to Serverless Resource Group.
-
-
Confirm migration details and configure the new resource group
In the Migrate to Serverless Resource Group panel, confirm and configure the following information:-
Current Resource Group: The system displays the type, region, specifications, and fees for the current exclusive resource group. Verify that the information is correct.
-
Serverless Resource Group: This is the core configuration section for the migration. The system estimates the cost of the new resource group based on the original group's usage over the past 30 days.
ImportantThe estimated cost is based on list prices and does not include applicable discounts. The value is for reference only.
-
Select a billing method: You can choose Subscription or Pay-as-you-go. Select the billing method that best suits your resource consumption and usage frequency. For more information about Serverless resource group billing, see Serverless resource group billing.
-
Subscription: Suitable for scenarios with stable and high resource usage. You must specify the resource group specifications. The minimum specification is 2 CU.
-
Pay-as-you-go: Suitable for scenarios with fluctuating resource usage. You are billed for the resources you actually use, without pre-selecting specifications.
NoteA Serverless resource group incurs task scheduling fees based on the number of instances that are generated and run successfully each day. For more information, see Task scheduling billing.
-
-
Confirm specifications and fees:
Based on your original resource specifications, the system recommends a suitable number of CUs (1 CU = 1 vCPU + 4 GiB memory) for the Serverless resource group and estimates the post-migration cost. Any cost change (savings or increase) is highlighted for your review. -
Confirm network configuration:
Set the default VPC and vSwitch for the Serverless resource group.Note-
A Serverless resource group requires a VPC for network access. For more information, see Network connectivity configuration.
-
If the original resource group used host mappings, the PrivateZone service is automatically enabled on the selected default VPC after migration, and the corresponding DNS records are created.
-
If the original exclusive resource group is already attached to a VPC, you can select it as the default VPC.
-
If the original exclusive resource group is not attached to a VPC, you can select an existing VPC under your Alibaba Cloud account as the default.
-
-
-
View affected tasks:
The panel lists all tasks that will be affected by the migration. Review this list carefully.ImportantA task appears in this list only if it has successfully run on the selected resource group. The list is updated the next day (T+1).
-
-
Start the migration
Click Confirm Migration to start the migration process.
Phase 2: Execute migration
After clicking Confirm Migration, the resource group undergoes the following changes:
-
Create a Serverless resource group
-
The system creates a new Serverless resource group. This takes a few minutes. You can close the dialog box without interrupting the background process.
-
After the Serverless resource group is successfully created, its status changes to Pending start.
-
-
Test data source connectivity
In the Data Sources of the migration interface, run a Connectivity test for all relevant data sources with the new Serverless resource group. You can start the canary release only after all data sources pass the connectivity test. If a test fails, use the connectivity diagnostics tool to troubleshoot. For more information about network configuration for data sources and resource groups, see Network connectivity configuration.
-
Enter the canary period
-
After all data source connectivity tests pass, click Start Canary Now to begin the four-day canary period. After a few minutes of initialization, the status changes to Migrating (Canary release) 20%.
-
The canary period lasts for four days. During this time, tasks are scheduled to both the original and new resource groups according to a daily shifting ratio. The status reflects the current traffic distribution.
Canary phase
Day 1
Day 2
Day 3
Day 4
Traffic ratio
20%
50%
100%
100% (cooldown)
During this period, monitor your tasks closely to ensure they run successfully.
-
Phase 3: Manage migration
During the canary period, you can manage the migration at any time.
-
Open the migration management panel
On the resource group list page, find the resource group that is in the canary period. Click the
icon in the Actions column and select Serverless Resource Group Migration Management. -
Make a decision in the management panel
The management panel displays migration details and provides two core actions:-
Action 1: Complete the migration
-
Description: Click Stop Canary Release and Migrate All Now to immediately shift all remaining tasks to the new Serverless resource group. This action is irreversible.
-
Impact: This cutover process may interrupt running real-time tasks for several minutes.
-
Timing: Perform this action during off-peak business hours.
-
-
Action 2: Stop the migration
-
Description: If you encounter issues during the canary period and need to roll back, click Stop & Roll Back on the Serverless Resource Group Migration Management panel.
-
Impact: The system deletes the new Serverless resource group and restores all configurations to their pre-migration state. The original exclusive resource group returns to the Running state. You can initiate the migration again later.
-
-
If you take no action by the end of the four-day canary period, the system automatically completes the migration. After the cutover, the original exclusive resource group is released, and its Purpose in the resource group list changes to General-purpose.
Next steps
After the migration is complete, tasks that used the original exclusive resource group are automatically switched to the new Serverless resource group.
To switch the resource group for individual tasks, see Switch legacy resource groups.