Lifecycle
The MaxCompute table lifecycle feature automatically reclaims table or partition data that remains unchanged for a specified period after its last update. This specified period is the lifecycle. Setting a lifecycle enables automatic data cleanup or retention, reducing storage costs.
Basic Definitions and Functions
-
Unit and Value: The lifecycle unit is day. The value must be a positive integer.
-
Function: Enables periodic automatic reclamation of tables or partitions.
Reclamation Rules
-
Start Time: Lifecycle calculation begins at the last data modification time (
LastModifiedTime) of the table or partition. -
Trigger Conditions:
Reclamation occurs only when the
LastModifiedTimeexceeds the specified lifecycle duration. A table or partition is eligible for reclamation when(current time - LastModifiedTime) > lifecycle days. -
Execution Mechanism:
The system starts reclamation daily at a scheduled time and scans all tables and partitions. Actual reclamation timing depends on service load, and immediate execution after expiration is not guaranteed. After the data lifecycle reaches the deletion point, the data is deleted within 24 hours.
-
Example:
Assume a partitioned table has a lifecycle of 1 day and the last modification time for data in a partition is
February 17, 2026, 15:00. If the table is scanned beforeFebruary 18, 2026, 15:00(less than one day), the partition will not be reclaimed. If a scan on February 19, 2026, finds that the partition'sLastModifiedTimeexceeds the specified lifecycle duration, the partition will be reclaimed.
Applicable Rules for Different Table Types
-
Non-partitioned Tables
-
The lifecycle applies to the entire table.
-
You can specify the lifecycle when you create the table. If you do not specify a lifecycle, MaxCompute does not automatically reclaim the table or its partitions based on lifecycle rules.
-
If table data is not modified within its lifecycle, MaxCompute automatically reclaims the entire table after the specified number of days, similar to a DROP TABLE operation.
-
After deleting a table, all its property information—including the lifecycle—is deleted. When you create a new table with the same name, the table's lifecycle follows the newly set properties.
-
-
Partitioned Tables
-
The lifecycle applies to each individual partition. However, the lifecycle can be set only at the table level and cannot be set at the partition level. The lifecycle specified for a partitioned table applies to all its partitions.
-
You can specify the lifecycle when you create the table. If you do not specify a lifecycle, MaxCompute does not automatically reclaim partitions based on lifecycle rules.
-
Partitions not modified within their lifecycle are reclaimed individually.
-
Unlike non-partitioned tables, a partitioned table is not deleted after its last partition is reclaimed.
-
After deleting a table, all its property information—including the lifecycle—is deleted. When you create a new table with the same name, the table's lifecycle follows the newly set properties.
-
Auditing and Monitoring
-
Executor:
The Alibaba Cloud service account
maxcompute.aliyuncs.comperforms lifecycle data reclamation operations. -
Operation Records:
You can retrieve relevant operation records from the ActionTrail service. For more information, see Get started with the event query feature. For example, you might see records such as 'Table
bettergithubanalytics.test_lifecycleautomatically reclaimed by MaxCompute based on its lifecycle'.Automatic Reclamation of Non-partitioned Tables
Table
bettergithubanalytics.test_lifecycleis automatically reclaimed by MaxCompute based on its lifecycle. You can see the following records.On the event details page in ActionTrail, the event interpretation indicates that an Alibaba Cloud service (maxcompute.aliyuncs.com) called the DropTable operation of MaxCompute. In the basic information, the event name is DropTable, the username is maxcompute.aliyuncs.com (which indicates that the operation was automatically initiated by a system service), and the event time is December 11, 2022, 11:21:11. The related resources include two records:
Table (ACS::MaxCompute::Table):
test_lifecycleProject (ACS::MaxCompute::Project):
bettergithubanalytics
{ "eventType": "TableEvent", "referencedResources": { // List of resources affected by the event "ACS::MaxCompute::Table": [ "test_lifecycle" ], "ACS::MaxCompute::Project": [ "bettergithubanalytics" ] }, "userIdentity": { // Identity information of the requester "accountId": "xxx", // ID of the Alibaba Cloud account "type": "system", // Alibaba Cloud service "userName": "maxcompute.aliyuncs.com" }, "serviceName": "MaxCompute", // Name of the cloud service related to the event "additionalEventData": { "TableName": "test_lifecycle", "ProjectName": "bettergithubanalytics", "OperationText": "RECYCLE_TABLE" }, "requestId": "3b85e8ed-6c4a-4680-b386-4afd90dd6972", "eventTime": "2022-12-11T03:21:11Z", // Time when the event occurred (UTC) "isGlobal": false, "acsRegion": "cn-beijing", // Alibaba Cloud region "eventName": "DropTable" // Event name }Automatic Reclamation of Partitioned Table Partitions
Partition
sale_date=2013/region=chinaof tablebettergithubanalytics.sale_detailis automatically reclaimed by MaxCompute based on its lifecycle. You can see the following records.On the event details page in ActionTrail, the event interpretation shows that an Alibaba Cloud service called the ChangeTable operation of MaxCompute. In the basic information, the event name is ChangeTable, the event source is MaxCompute, and the username is
maxcompute.aliyuncs.com. The Related Resources section lists two associated resources:Resource type Table (ACS::MaxCompute::Table), resource name
sale_detailResource type Project (ACS::MaxCompute::Project), resource name
bettergithubanalytics
The following JSON example shows the details of an event in which a system service (
type: system) performs theRECYCLE_PARTITIONoperation. The valuesystemofuserIdentity.typeindicates that the operation was automatically triggered by an Alibaba Cloud service:{ "referencedResources": { "ACS::MaxCompute::Table": [ "sale_detail" ], "ACS::MaxCompute::Project": [ "bettergithubanalytics" ] }, "userIdentity": { "accountId": "xxx", "type": "system", "userName": "maxcompute.aliyuncs.com" }, "serviceName": "MaxCompute", "additionalEventData": { "TableName": "sale_detail", "Partition": "sale_date=2013/region=china", "ProjectName": "bettergithubanalytics", "OperationText": "RECYCLE_PARTITION" }, "requestId": "884d49e0-7343-422e-899a-599628229617", "eventTime": "2022-12-11T03:21:11Z", "isGlobal": false, "acsRegion": "cn-beijing", "eventName": "ChangeTable" }
References
-
For operations such as specifying a lifecycle during table creation, modifying the lifecycle of a table, and modifying the
LastModifiedTimeof a table, see Table operations. -
For more lifecycle operations, such as setting, disabling, or recovering lifecycles for existing or new tables, see Manage table lifecycle.
-
You can also view reclamation operation records for table partitions using the Data Map feature in DataWorks. For more information, see Overview. The displayed operator account is the MaxCompute system account
odps_user@aliyun.com, which is the Alibaba Cloud service namedmaxcompute.aliyuncs.com.The Data Change Records table shows the change details of partition operations, including the following fields:
Change Description: for example,
Delete partition:sale_date=2013/region=chinaChange Type: for example, Delete Partition
Granularity: for example, PARTITION
Time: the time when the operation was performed
Operator: the account that performed the operation, for example,
odps_user@aliyun.com