Archive data using DMS

Features

DMS lets you flexibly archive business data, such as production logs and transaction records.AnalyticDB for PostgreSQL in Serverless mode serves as the data archiving engine and provides the following benefits:

• Supports key archiving capabilities, including table-level archiving, custom archiving conditions, data mapping, and data cleanup.

• Supports custom periodic archiving to meet auditing and historical data analysis requirements.

• When you select the Cost-effective option, the instance automatically starts when an archiving task begins and pauses after it completes. This approach significantly reduces costs by using compute resources only on demand.

• AnalyticDB for PostgreSQL in Serverless mode delivers powerful data analysis performance. Compared to storage media like Object Storage Service (OSS), it helps you manage archived data in a structured way and run efficient analysis on demand.

• Supports analyzing archived data directly in AnalyticDB for PostgreSQL.

Comparison of data archiving solutions

Prerequisites

• You have created an AnalyticDB for PostgreSQL instance in Serverless mode as the archive destination. For more information, see Create an instance.

• You have created an account for the archive destination database. For more information, see Create a database account.

Usage notes

• We recommend using an instance in Serverless mode as the destination data archiving engine. This allows you to benefit from ultra-low storage prices and high-performance data analysis. When you create a data archiving ticket, select Cost-effective to automatically start the instance when the task begins and pause it after the task completes.

• If you choose to automatically clean up data from the source table during the archiving process, ensure the source table has enough reserved space to prevent errors.

• AnalyticDB for PostgreSQL supports instances in both Elastic Storage mode and Serverless mode.

• To run a periodic archiving task, the control mode for both the source and destination databases must be Secure Collaboration or Stable Change. If you only need to run the archiving task once, there are no restrictions on the database control mode.

  Note

  If the control mode of an instance is Stable Change, we recommend enabling secure access or changing the control mode to Secure Collaboration before running a periodic archiving task. This prevents task failure due to an expired logon. For more information, see Enable secure access and Change the control mode.

Step 1: Access data archiving

1. Log on to the AnalyticDB for PostgreSQL console.
2. In the upper-left corner of the console, select a region.
3. Find the instance that you want to manage and click the instance ID.

4. On the Basic Information page, click Log On to Database in the upper-right corner.

5. On the Logon Instance page, enter the Database Account and Database Password.

6. Click Log On.

7. In the top menu bar, choose Solutions > Archive Data.

Step 2: Create an archiving ticket

1. In the top navigation bar, choose Solution > Data Archiving.

   Note

   If you use the DMS console in simple mode, move the pointer over the icon in the upper-left corner of the DMS console and choose All Features > Solution > Data Archiving.

2. In the upper-right corner of the page, click Data Archiving.

3. On the New Data Archiving Ticket page, configure the parameters.

   Parameter	Required	Description
   Task Name	Yes	Enter a clear and descriptive name for the task for easy identification.
   Archive Destination	Yes	Select AnalyticDB for PostgreSQL.
   ADB Instance	Yes	Search for and select the target AnalyticDB for PostgreSQL instance. Note If you select Cost-effective for a pay-as-you-go instance in Serverless mode, you gain the following advantages: The instance starts automatically when the archiving task begins. The instance pauses automatically after the archiving task completes. While the archiving task is running, the instance is in the Running state, and you are charged for compute nodes and storage. For more information, see Billing. After the archiving task completes, the instance is in the Paused state, and you are charged only for storage.
   Database	Yes	Search for and select the source database.
   Archiving Configuration	Yes	Select the tables you want to archive data from. Optional: Enter filter conditions for the data to be archived. Note If you need to use time variables, such as archiving data from six months ago, you must first set up Variable Configuration.
   Archive Table Mapping	No	You can click Edit in the Actions column for a table to edit the target table name, column information, sharding key, and partition key.
   Variable Configuration	No	Configure time variables. For more information, see Configure time variables.
   Post-execution Action	No	If you select Cleanse Archived Data from Source Table (delete-lock-free), the system automatically deletes the archived data from the source table after the task completes. This operation uses a lock-free DELETE statement. During this process, a temporary backup table is created in the source database. Ensure you have enough space in the source database to prevent the instance from becoming unavailable. To clean up the temporary backup table, you can create a common data change ticket after confirming the data has been archived correctly. If you do not select Cleanse Archived Data from Source Table (delete-lock-free), you must manually delete the data from the source table and optimize the table space after the data has been archived. Cleanse source table data: Create a common data change ticket to clean up the archived data from the database. For more information, see Common data change. Optimize source table space: Create a lock-free change ticket to optimize the source table space. For more information, see Implement lock-free schema changes by using lock-free change tickets.
   Run Mode	Yes	Select a run mode. The following modes are supported: One-time Execution: The archiving task runs only once after the ticket is approved. Periodic Scheduling: The archiving task runs periodically according to the configured schedule once the ticket is approved. For more information about the configuration, see Periodic Scheduling.

4. In the Preview Archived Data dialog box that appears, click OK.

5. Once the ticket is approved, the system automatically runs the data archiving task.

   If the task fails, go to the Execution section of the ticket and click Details in the Actions column to view the logs and identify the cause. If the failure was due to a temporary issue, such as a network or database connection error, you can click Resume from Breakpoint to retry the task.

6. The task completes.

Step 4: Query archived data

1. Log on to the AnalyticDB for PostgreSQL console.
2. In the upper-left corner of the console, select a region.

3. Find your target instance and click the instance ID.

4. If you selected Cost-effective when configuring the ticket parameters, you must start the instance. On the Basic Information page, choose Manage Instance > Resume Instance in the upper-right corner.

5. On the Basic Information page, click Log On to Database in the upper-right corner.

6. On the Logon Instance page, enter the Database Account and Database Password.

7. Click Log On.

8. In the Logged-in Instances list on the left side of the DMS console, find the archive instance and database, and then double-click the database.

9. On the SQL Console tab, run a SELECT statement to query the archived data.

   SELECT * FROM customer;

Periodic scheduling

When Operation Mode is set to Cyclic scheduling, configure the following parameters.