Adjust the disk capacity of an ApsaraDB for ClickHouse cluster.-ApsaraDB for ClickHouse-阿里云帮助中心

This topic describes how to reduce the storage capacity of an ApsaraDB for ClickHouse Community-Compatible Edition cluster.

Prerequisites

The cluster is a Community-Compatible Edition cluster.
The cluster status is Running.
The cluster has no pending renewal orders.

Note
Log on to the Alibaba Cloud ClickHouse console. In the upper-right corner, choose Fee > Expenses and Costs. In the left-side navigation pane, choose Order>My Orders, and then pay for or cancel the order.

Considerations

The storage capacity per node ranges from 100 GB to 32,000 GB.
After a disk downgrade, historical data in tables that use a MergeTree engine is migrated to the new cluster and automatically redistributed.
Migrated content:
- Databases, data dictionaries, and materialized views.
- Table schemas: All table schemas except for those of tables that use the Kafka or RabbitMQ engine.
- Data: Data from tables that use MergeTree-family engines is incrementally migrated.
Content that is not migrated:
- Tables that use the Kafka or RabbitMQ engine and their data.
  
  Important
  During a configuration change, data is migrated to a new instance, and traffic is eventually switched to the new instance. To prevent Kafka and RabbitMQ data from being split, first delete the Kafka and RabbitMQ engine tables from the source cluster. Recreate them after the configuration change is complete.
- Data from non-MergeTree tables, such as external tables and Log-family tables.
Important
You must manually handle the unsupported items during the disk downgrade.
Do not perform any DDL operation during the disk downgrade. Otherwise, data validation may fail at the end of the process, which can cause the downgrade to fail.
The internal node IP addresses change after the disk downgrade. If your application relies on node IP addresses for data writes and access, you must obtain the new VPC CIDR block IP addresses for the cluster. For more information, see Obtain the VPC CIDR block of a cluster.
After a configuration change, a period of high-frequency merge operations will increase I/O usage and may increase service request latency. Plan ahead to mitigate the potential impact of increased request latency. For information about how to calculate the merge duration, see Calculate the merge duration after migration.
During the disk downgrade, CPU and memory usage on the cluster increases. The expected resource usage per node is less than 5 cores and 20 GB.

Billing

Changing the configuration of a cluster will change the fees. The actual fees are subject to the prices displayed in the console. For more information, see Billing for configuration changes.

Procedure

Step 1: Record and clean up Kafka/RabbitMQ engine tables

Kafka and RabbitMQ engine tables cannot be directly migrated through the console scaling feature. You must record the DDL statements of these tables and delete them from the cluster before scaling to prevent interference with the scaling task.

Log on to the cluster and run the following statement to query the tables that need to be handled. For information about how to log on to the cluster, see Connect to an Alibaba Cloud ClickHouse cluster by using a DMS client.
```
SELECT * FROM  `system`.`tables` WHERE engine IN ('RabbitMQ', 'Kafka');
```
View the CREATE TABLE statement for the target table and back it up.
```
SHOW CREATE TABLE <aim_table_name>;
```
If a materialized view was created without the TO clause, the system automatically generates an implicit target table prefixed with .inner or .inner_id. These implicit tables are migrated to the new nodes during scaling, but due to internal naming mechanisms, directly recreating the original materialized view may cause naming conflicts. Therefore, you must first rename the implicit target tables to regular table names.
```
-- Rename implicit target tables to regular table names (execute on all nodes)
RENAME TABLE <database>.`.inner_id.<uuid>` TO <database>.<new_target_table_name> ON CLUSTER default;
```
Delete the materialized views and Kafka/RabbitMQ engine tables.

Important
You must delete the materialized views that reference the Kafka/RabbitMQ tables first, and then delete the Kafka/RabbitMQ engine tables. Otherwise, the scaling operation fails.
```
-- Delete materialized views first
DROP TABLE <database>.<materialized_view_name> ON CLUSTER default;

-- Then delete Kafka/RabbitMQ engine tables
DROP TABLE <database>.<kafka_or_rabbitmq_table_name> ON CLUSTER default;
```

Step 2: Back up non-MergeTree data

Log on to the cluster and run the following statement to view the non-MergeTree tables whose data needs to be migrated.

SELECT
    `database` AS database_name,
    `name` AS table_name,
    `engine`
FROM `system`.`tables`
WHERE (`engine` NOT LIKE '%MergeTree%') AND (`engine` != 'Distributed') AND (`engine` != 'MaterializedView') AND (`engine` NOT IN ('Kafka', 'RabbitMQ')) AND (`database` NOT IN ('system', 'INFORMATION_SCHEMA', 'information_schema')) AND (`database` NOT IN (
    SELECT `name`
    FROM `system`.`databases`
    WHERE `engine` IN ('MySQL', 'MaterializedMySQL', 'MaterializeMySQL', 'Lazy', 'PostgreSQL', 'MaterializedPostgreSQL', 'SQLite')
))

Back up the data.

You need to back up the business data from the non-MergeTree tables. For more information, see Back up data to OSS.

Step 3: Console disk downgrade operation

Log on to the ApsaraDB for ClickHouse console.
In the upper-left corner of the page, select the region where the cluster is deployed.
On the Clusters page, click the Clusters of Community-compatible Edition tab.
In the Actions column of the target Cluster ID, click Change Configurations.
In the Change Configurations dialog box, select Downgrade Disk Specification and click OK.
In the disk downgrade check dialog box, review the check results.
- If the check is successful, click Next.
- If the check fails, fix the issues as prompted on the page and click Retry Detection. After the check is successful, click Next.
  During a disk downgrade, the check can fail for the following common reasons:
  Missing unique distributed table: A local table does not have a corresponding distributed table. You must create one.
  
  The corresponding distributed table is not unique: A local table has more than one distributed table. Delete the extra distributed tables and keep only one.
  
  Unsupported Kafka/RabbitMQ engine tables: The cluster contains Kafka or RabbitMQ tables. You must delete them.
  
  Non-replicated *MergeTree tables in a master-replica instance: Data inconsistency between replicas will cause a data migration exception during scaling.
  
  Columns of the distributed table and the local table are inconsistent: You must make the columns consistent. Otherwise, an exception will occur during data migration in the scaling process.
  
  The table is missing on some nodes: You must create tables with the same name on different shards. For the inner table of a materialized view, we recommend that you rename the inner table and then rebuild the materialized view to point to the renamed inner table. For more information, see The inner table of a materialized view is inconsistent across shards.
On the Change Specification page, configure the Storage Capacity and Stop-writing Window parameters based on your business requirements.
1. Storage capacity: The storage capacity per node ranges from 100 GB to 32,000 GB.
2. Stop-writing window: A disk downgrade involves data migration. To improve the success rate of the migration, you must stop data writes to the cluster.
  Note
  Note the following requirements for the stop-writing window:
  - Set the stop-writing window to at least 30 minutes.
  - The disk downgrade must be completed within five days after the configuration change is created. Therefore, the end date of the Time of Stopping Data Writing must be no later than Current Date + 5.
  - To reduce the impact on your business, schedule the stop-writing window during off-peak hours.
Click Buy and Start and complete the payment.
On the Paid page, click Console.
On the Clusters of Community-compatible Edition tab, view the status of the target cluster in the Status column. When the cluster status changes from Scaling Horizontally to Running, the disk downgrade is complete.

Note

A disk downgrade usually takes at least 30 minutes. The actual duration depends on the amount of data. You can monitor the progress by checking the cluster status in the console.

Important

If the cluster contains Kafka/RabbitMQ engine tables: When the downgrade task enters the Data Migration phase (after table schema migration is complete), perform Step 4 to recreate the Kafka/RabbitMQ engine tables on the cluster so that incremental data resumes flowing in and is synchronized to the new nodes.
Shortly before the configured write suspension window, perform Step 5 to drop the Kafka/RabbitMQ engine tables and their downstream materialized views from the cluster. This prevents message backlog during the write suspension period, which could cause data inconsistency.

Step 4: Rebuild Kafka/RabbitMQ engine tables

After the scaling task enters the data migration phase (that is, table schema migration is complete), use the previously saved DDL statements to rebuild the Kafka/RabbitMQ engine tables and their downstream materialized views. Once rebuilt, incremental data resumes flowing and is automatically synchronized to the new nodes.

Important

If you performed a RENAME operation on implicit target tables in Step 1, use the TO clause pointing to the renamed target table when rebuilding the materialized view. For more information about the TO clause, see CREATE MATERIALIZED VIEW.

-- Rebuild Kafka/RabbitMQ engine tables
CREATE TABLE <database>.<kafka_or_rabbitmq_table_name> (...)
ENGINE = Kafka/RabbitMQ
SETTINGS ...;

-- Rebuild materialized view (using TO clause pointing to the renamed target table)
CREATE MATERIALIZED VIEW <database>.<materialized_view_name> TO <database>.<new_target_table_name>
AS SELECT ... FROM <database>.<kafka_or_rabbitmq_table_name>;

Step 5: Delete Kafka/RabbitMQ engine tables

Shortly before the configured write-stop window, delete the Kafka/RabbitMQ engine tables and their downstream materialized views on the cluster to stop incremental data writes and ensure consistency of the final data synchronization.

-- Delete materialized views first
DROP TABLE <database>.<materialized_view_name> ON CLUSTER default;

-- Then delete Kafka/RabbitMQ engine tables
DROP TABLE <database>.<kafka_or_rabbitmq_table_name> ON CLUSTER default;

Step 6: Rebuild Kafka/RabbitMQ engine tables on the scaled cluster

After the scaling task is complete, use the previously saved DDL statements to rebuild the Kafka/RabbitMQ engine tables and their downstream materialized views on the scaled cluster to restore the incremental data consumption pipeline. For information about how to log on to the cluster, see Connect to an Alibaba Cloud ClickHouse cluster by using a DMS client.

Important

-- Rebuild Kafka/RabbitMQ engine tables
CREATE TABLE <database>.<kafka_or_rabbitmq_table_name> (...)
ENGINE = Kafka/RabbitMQ
SETTINGS ...;

-- Rebuild materialized view (using TO clause pointing to the renamed target table)
CREATE MATERIALIZED VIEW <database>.<materialized_view_name> TO <database>.<new_target_table_name>
AS SELECT ... FROM <database>.<kafka_or_rabbitmq_table_name>;

Step 7: Import non-MergeTree data

Log on to the new cluster and import from OSS the data that you backed up in Step 2: Back up non-MergeTree data. For more information, see Import data from OSS.