Data versions and TTL

Max versions

If you do not specify the Max Versions parameter when creating a data table, it defaults to 1. You can modify this value later by calling the UpdateTable operation.

Data versions that exceed the Max Versions limit are considered invalid. You cannot read these versions, even if the system has not yet deleted them.

• If you decrease the Max Versions value, the system asynchronously deletes any older versions that now exceed the new limit.

• If you increase the Max Versions value, previously inaccessible older versions might become readable again if they fall within the new limit and have not yet been deleted.

Version number

After you configure Max Versions for a data table, Tablestore creates a new version of a value each time you update it in an attribute column. This version is assigned a version number, which is a timestamp in milliseconds.

When you write data to an attribute column, you can either specify a version number or allow the system to automatically generate one. Because version numbers are millisecond-based timestamps, they must be divided by 1,000 to be converted into seconds for TTL and Max Version Offset calculations.

• When the system automatically generates a version number, it uses the current time as a millisecond timestamp, calculated from 1970-01-01 00:00:00 UTC.

• When you specify a custom version number, it must be a 64-bit millisecond timestamp that falls within the valid version range.

Version numbers enable the following features:

• TTL

  The version number can be used to define the lifecycle of values in an attribute column. The system asynchronously deletes a data version once its age exceeds the configured TTL.

  For example, if a data version has a version number of 1468944000000 (which corresponds to 2016-07-20 00:00:00 UTC+8) and the data table's TTL is set to 86400 (one day), this version expires at 2016-07-21 00:00:00 UTC+8, and the system asynchronously deletes it.

• Read data by version

  When reading row data, you can specify the maximum number of versions to read per column or a specific range of version numbers. For more information, see Read data.

TTL

TTL is a property of a data table that specifies the data retention period in seconds. The system asynchronously deletes a data version in an attribute column when its age exceeds the configured TTL. If all data versions in all attribute columns of a row expire, the system asynchronously deletes the entire row.

For example, if a data table's TTL is set to 86400 (one day), then at 2016-07-21 00:00:00 UTC+8, all attribute column data with a version number less than 1468944000000 expires. This timestamp corresponds to 2016-07-20 00:00:00 UTC+8 after converting to seconds. The system then asynchronously deletes the expired data.

If you do not set a TTL when creating a data table, it defaults to -1, meaning the data never expires. You can later modify the TTL by calling the UpdateTable operation.

Data that exceeds its TTL is considered expired and becomes unreadable, even if the system has not yet deleted it.

• If you decrease the TTL value, some data in the data table may expire, and the system will asynchronously delete the expired data.

• If you increase the TTL, older versions that expired under the previous TTL might become readable again if they fall within the new TTL period and have not yet been deleted.

Max version offset

To prevent data from expiring immediately upon being written, which can happen if a custom timestamp is older than the table's TTL, you can configure a Max Version Offset.

Max Version Offset is the maximum allowed deviation, in seconds, between a custom version number and the current system time. It is a positive integer.

To ensure a successful write operation, the system checks the version number of the data for each attribute column. Data can be written only if its version number, converted from milliseconds to seconds, falls within the valid version range of [max{write_time - Max Version Offset, write_time - TTL}, write_time + Max Version Offset).

For example, if a data table's Max Version Offset is 86400 (one day), at 2016-07-21 00:00:00 UTC+8, you can only write data with a version number that is greater than 1468944000000 (which corresponds to 2016-07-20 00:00:00 UTC+8 after converting to seconds) and less than 1469116800000 (which corresponds to 2016-07-22 00:00:00 UTC+8 after converting to seconds). If a version number for an attribute column in a row is 1468943999000 (which corresponds to 2016-07-19 23:59:59 UTC+8 after converting to seconds), the write operation for that row fails.

If you do not configure Max Version Offset when creating a data table, it defaults to 86400. You can modify this value later by calling the UpdateTable operation.

Configuration methods

You can configure parameters such as Max Versions and TTL when you create a data table. You can also modify these parameters after the table is created. The following examples show how to modify the configuration of an existing table.

alter -t mytable --ttl 86400 --version 1

private static void updateTable(SyncClient client) {
    // The expiration time of data, in seconds. A value of -1 means the data never expires.
    int timeToLive = -1;
    // Update Max Versions to 5.
    int maxVersions = 5; 
    // Update Max Version Offset to 86400, in seconds.
    int maxTimeDeviation=86400;
    TableOptions tableOptions = new TableOptions(timeToLive, maxVersions,maxTimeDeviation);
    // Set the table name.
    UpdateTableRequest request = new UpdateTableRequest("<TABLE_NAME>");
    request.setTableOptionsForUpdate(tableOptions);
    client.updateTable(request);
}

Billing

• Using multiple data versions increases storage costs because you are charged for storing both the version numbers and the data of each version.

• Data cleanup based on TTL is free of charge. However, enabling TTL adds a timestamp as a version number to each attribute column, which increases storage usage.

For more information, see Data storage.

FAQ

• What do I do if the "[table ttl] must be bigger than or equal search index ttl" error is returned when I modify the TTL of a data table?

• Why did the data table size increase after I configured TTL?