Change compression and encoding methods during migration

Prerequisites

• Verify network connectivity among the source cluster, the destination cluster, and LTS.

• You have added data sources for the source and destination instances. For instructions, see Add a Lindorm wide table data source.

• The LTS version must be 3.8.2 or later.

  Note

  You can view the LTS version of your instance on the Instance Details page in the Lindorm console. If the version does not meet the requirements, you can upgrade it in the console. For instructions, see Upgrade the minor version of an instance.

Considerations

• Ensure the target instance has sufficient capacity to prevent task failures.

• Renaming tables is not supported. If you specify the renaming format (tableA/tableB), secondary index tables are automatically ignored.

Procedure

1. Log on to LTS.

2. In the left-side navigation pane, choose Lindorm/HBase Migration > One-click Migration.

3. Click Create Task.

4. Select the source cluster and destination cluster.

5. Select the following operations: schema migration, real-time data replication, and full data migration.

   • table schema migration: Creates tables in the destination cluster with a consistent schema and region information. If a table already exists, it is skipped.

   • real-time data replication: Synchronizes incremental data from the source cluster in real time.

   • historical data migration: Performs a file-level physical migration of all data.

6. In the Tables to synchronize or migrate section, enter the tables that you want to migrate. Use the format <database_name>:<table_name>, for example, default:shop_item_relation. To migrate multiple tables, enter each table on a new line.

7. Based on your migration scenario, add the required parameters in the Advanced settings section.

   Migration scenario	Configuration	Example
   Change only the compression method during data migration	{"createTableCompression":"<compression_method>","needCompress":true}	{"createTableCompression":"ZSTD","needCompress":true}
   Change only the encoding method during data migration	{"createTableBlockEncoding":"<encoding_method>","needCompress":true}	{"createTableBlockEncoding":"INDEX","needCompress":true}
   Change both the compression and encoding methods during data migration	{"createTableCompression":"<compression_method>","createTableBlockEncoding": "<encoding_method>","needCompress":true}	{"createTableCompression":"ZSTD","createTableBlockEncoding": "INDEX","needCompress":true}

   Parameters

   Parameter	Description
   createTableCompression	The compression method to use when the table schema is created in the destination cluster. Valid values: ZSTD SNAPPY LZ4
   createTableBlockEncoding	The encoding method to use when the table schema is created in the destination cluster. Valid values: DIFF INDEX
   needCompress	Specifies whether to apply the new compression and encoding settings to the data during migration. Valid values: true: The new settings are applied. false: The new settings are not applied.

   Important

   If the source cluster is a self-managed cluster, you must also add "exporterEnable":false to the Advanced settings. For example: { "createTableCompression":"ZSTD","needCompress":true,"exporterEnable":false}.

8. Click Create.

Next steps

Migration progress

On the One-click Migration page, click the name of the target task to view the progress of the batch migration task. If the task's Status is SUCCEEDED, the batch migration task is complete. The task details page shows the status for each stage: The schema creation task status is SUCCEEDED and was completed on 2024-02-01 16:13:33; the real-time task status is RUNNING, with a replication lag of about 2,394,533 ms; and the batch migration task (Total: 1, Completed: 1, Failed: 0) Status is SUCCEEDED.

Compression and encoding methods

Go to the overview page of the cluster management system and click the name of the target table under the destination database. In the Current details table area, click View table properties to check the values of the COMPRESSION and DATA_BLOCK_ENCODING parameters to verify that the changes were successful. For instructions on how to access the cluster management system, see Log on to the cluster management system.