Cross-region disaster recovery
MaxCompute provides a cross-region disaster recovery feature to ensure that your jobs run continuously. This feature backs up your projects, including data and metadata, to a remote location. The backup location is more than 100 km away from the data source to improve data security. You can select a destination region for your backups and monitor the backup progress. If a region-level failure occurs, you can quickly switch your project to the backup region and resume your business operations using the backed-up data. This topic describes the cross-region disaster recovery feature and how to use it to back up a MaxCompute project.
Function overview
MaxCompute cross-region disaster recovery creates a disaster recovery cluster in a remote location more than 100 km away from the current region. This feature provides a complete remote backup for your project's data. This feature protects the integrity and availability of your data and reduces the risk of data loss caused by regional natural disasters.
Backup scope:
Data: tables, partitions, Delta Lake tables, and materialized views.
Metadata: version information, user roles, and access policies.
Backup cycle: Backups are performed on an hourly cycle.
Incremental data backup: Incremental data is identified by changes in the file's metadata version. Operations in the primary region, such as overwriting data, merging small files, or deleting data, create new versions. Cross-region replication then synchronizes the latest data to the backup region.
Backup order: The system automatically identifies the latest versions of all unbacked-up data and starts parallel backup jobs. The time required to complete a backup depends on the amount of data and the cross-region data transmission bandwidth. The backup cycle is controlled by the automatic scheduling policy for cross-region replication. This policy does not guarantee that every incremental version of the data is backed up. Instead, it copies the latest version of your data each time a backup job starts.
Backup region selection: MaxCompute provides data replication tasks and ensures that a project can be backed up to multiple regions. You can choose to:
Back up all projects to the same region.
Back up data from a new project to any other region that already contains backups.
If a link is broken or returns an error, you can provide feedback by submitting a ticket.
How it works
Disaster recovery method
Cross-region disaster recovery uses file-level incremental backups and copies column-compressed binary data files. This method improves backup efficiency, reduces transmission costs, and achieves an hourly backup cycle.
Fault recovery
If a disaster occurs and the service in the primary region becomes unavailable, you can switch the project to the backup region. You can then assign compute resources in the backup region to the project and connect your upstream services to the Endpoint of the backup region to immediately resume business operations.
The MaxCompute control region and metadata achieve cross-region high availability (HA). The service does not stop if a single region fails. As shown in the following figure, a project in the China (Beijing) region is configured for disaster recovery. The destination region for the backup is China (Shanghai). This means a replication task from the China (Beijing) region to the China (Shanghai) region is selected. The cross-region replication feature then synchronizes data from the China (Beijing) region (primary region) to the China (Shanghai) region (backup region).
Trigger a project switchover
If a disaster occurs in the primary region and Alibaba Cloud confirms that the region cannot be recovered, a button to switch the project to the backup region becomes available. You can then choose whether to switch regions based on your business needs. After the switchover, the project's default region changes from the primary region to the backup region. The latest data version available at the time of the switchover becomes the visible and usable version.
Billing
For projects with cross-region disaster recovery enabled, MaxCompute charges for cross-region data replication and cross-region disaster recovery storage on a pay-as-you-go basis. For more information about billing, see Cross-region disaster recovery fees (pay-as-you-go).
Compute resource usage: If a project uses subscription resources in the primary region and a disaster occurs, the project defaults to using pay-as-you-go resources after you switch it to the backup region. You can purchase subscription resources in the backup region later and change the project's default compute quota to the new subscription quota.
Compute performance guarantee: When many disaster recovery projects are migrated to a backup region, MaxCompute ensures that compute resources are available after the switchover. MaxCompute also supplements compute resources, but performance may be affected. If you have high requirements for compute performance, you can purchase subscription compute resources in the destination region. After the region switchover, you can change the project's default compute quota to the purchased subscription quota.
Limits
Region limits: The cross-region disaster recovery feature is available only in the China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), and China (Chengdu) regions.
Project type limits: Only internal projects support cross-region disaster recovery. An internal project is a project for which Project Type is set to managed on the Projects page.
A project's data can be backed up to only one destination region.
Only data from standard tables, Delta Lake tables, and materialized views can be backed up. Resource backup is not supported. For Delta Lake tables, Time Travel queries on disaster recovery data have version limits. You can only perform a Time Travel query on versions from the latest backup version to the version when a compact major operation was last triggered by you or the system. For more information, see Time travel.
Point-in-time recovery (PITR) is not supported. Backups are performed at the file granularity. Cross-region replication ensures that each file is the latest version, but it does not guarantee that a snapshot of all data in the entire project is backed up at the same time. Therefore, PITR is not supported. The backup job scheduling determines that the file-level recovery point objective (RPO) is one day or less. You must consider the data version time alignment issues that arise from the lack of PITR support.
Use of related products:
If you resume production in the backup region and the project was associated with other products, such as DataWorks and OSS, before the switchover, you must activate these products in the destination region. You must then re-bind or re-configure the foreign tables for the migrated project in the destination region as if it were a new project. For example:
DataWorks: You must create a DataWorks workspace in the new region and re-bind the migrated MaxCompute project using the Endpoint of the new region. The original DataWorks workspace, even if it is still accessible, uses the service role identity authentication or the Endpoint of the original region to connect to the migrated project. This will cause authentication errors or connection issues.
OSS: If you used OSS before the migration, the OSS in the original region may also be inaccessible in a disaster recovery scenario. You must modify the MaxCompute foreign table configuration in the new region to point to the OSS Endpoint and file storage location of the new region. Even if the original OSS is still accessible, there is no guarantee that the migrated MaxCompute project can read OSS data across regions.
You are responsible for backing up or migrating DataWorks tasks, OSS data, and other assets using the respective backup features of those products.
Procedure
(Optional) Activate MaxCompute.
You must activate MaxCompute in the destination region. This means you must activate pay-as-you-go resources or purchase subscription resources.
If you have already purchased pay-as-you-go or subscription resources in the destination region, skip this step.
If MaxCompute is not activated, the destination region cannot be selected.
Add a cross-region backup project.
Log in to the MaxCompute console and select a region in the upper-left corner.
In the left-side navigation pane, choose .
On the Disaster Recovery page, click the Cross-region Disaster Recovery tab.
On the Cross-region Disaster Recovery tab, click Add Cross-region Backup Project.
In the Add Cross-region Backup Project dialog box, Project to Be Backed Up and select a Destination Secondary Region. Note: You must first activate MaxCompute in the backup destination region.
If the destination backup region you need is not available, submit a ticket for feedback.
Click Enable Cross-region Backup.
(Optional) Switch to the backup region.
After a failure in the primary region, Alibaba Cloud confirms that the primary region cannot be recovered and enables the ability to switch the project to the backup region. The switchover button then becomes available, and you can trigger the switchover. Use the switchover feature with caution. A switchover may cause data loss because backups are not real-time and do not support PITR. If you want to switch back to the original region after switching to the destination region, you must re-create the backup configuration for the project, perform a full data transfer, and then perform another switchover to complete the migration. The steps are as follows:
On the Cross-region Disaster Recovery tab, in the Actions column for the target project, click Disaster Recovery.
In the Disaster Recovery dialog box, enter the project name and click OK to switch the project's home region.
After the switchover is successful, the project is visible on the Projects page under Manage Configurations in the backup region.
Modify the Endpoint: After the project's region is switched, you must change the Endpoint in your connected products or applications to the Endpoint of the destination region to access the project. Using a cross-region Endpoint to connect to the project may cause connection stability or performance issues.
Related operations
Delete a backup
On the Cross-region Disaster Recovery tab, in the Actions column for the target project, click Delete.
In the Are you sure that you want to cancel the backup of this cross-region disaster recovery project and delete the backup? dialog box, enter the project name and click OK.
This operation stops cross-region data disaster recovery for the project and deletes the backup data in the backup region.
Use the delete feature with caution. This operation immediately deletes the replicated data in the backup region. To restore the backup, you must re-create the disaster recovery configuration and transfer all data from the primary region to the backup region again.
Monitor disaster recovery
On the Cross-region Disaster Recovery tab, in the Actions column for the target project, click Monitor.
You can view the current stored data size in the primary and backup regions and the historical cumulative traffic generated by cross-region replication. This information is generated by the scheduling of cross-region replication tasks and is not guaranteed to be real-time. The information may be up to 24 hours old.