Migrate from Object(JSON) to the new JSON type
ApsaraDB for ClickHouse Enterprise Edition has replaced the experimental Object(JSON) data type with the production-ready JSON type. If you used the Object(JSON) type in version 25.10 or earlier, you must complete the migration before upgrading to version 25.12.
Background
The legacy semi-structured data type Object(JSON) (also written as Object('json')) has been deprecated and completely removed from version 25.12. All instances that use this type must be migrated to the new JSON type.
The core objective of this migration is to convert all Object(JSON) columns to JSON columns in place, without data loss or prolonged service interruption. The migration can be completed with a single ALTER TABLE ... MODIFY COLUMN mutation.
The migration must be completed on a version that still supports Object(JSON) (≤25.10) before you upgrade to 25.12. After you upgrade to 25.12, the migration operation is no longer available.
Version and settings before migration
We recommend that you upgrade to version 25.10 before migration. The new JSON type is more stable in this version.
The migration has the following version requirements:
|
Cluster version |
Migration instructions |
|
25.3 to 25.10 |
The new |
|
24.10 to 25.2 |
You must explicitly enable |
|
24.8 to 24.9 |
You must explicitly enable |
The following table describes the related parameters:
|
Parameter |
Description |
Default value changes |
Migration action |
|
|
Enables the new |
Introduced in 24.10 (default false) → default true from 25.3 → now obsolete (always true) |
Not required for ≥25.3; for 24.10–25.2, set |
|
|
Legacy name of |
Introduced in 24.8 (default false) → default true from 25.3 → now obsolete |
Required for 24.8–24.9: set |
|
|
Makes the |
Changed from true to false in 24.8 → now obsolete (always false) |
Keep the default value false. Do not enable this setting; otherwise, |
|
|
Allows creation or use of the legacy |
Default false → now obsolete |
Not required for migration; the target is the new type |
Procedure
Step 1: Identify columns to migrate
If you manage multiple instances, you can first run an aggregate query to identify instances and tables that still use the legacy type:
SELECT
uniqExact(spoken_name) AS instances,
groupUniqArray(spoken_name),
uniqExact(database, name) AS tables,
any(create_table_query)
FROM merge('tables.*')
WHERE create_table_query LIKE '%Object(\'json\')%'
AND spoken_name NOT LIKE '%stress%'
AND scrape_time_microseconds > now() - INTERVAL 40 DAY
AND database NOT IN ('system', 'information_schema', 'INFORMATION_SCHEMA')
On a single instance, run the following SQL query to list all columns that use the legacy Object(JSON) type:
SELECT database, table, name AS column, type
FROM system.columns
WHERE type LIKE '%Object(%json%)%'
AND database NOT IN ('system', 'information_schema', 'INFORMATION_SCHEMA')
ORDER BY database, table, name;
The query returns the database name, table name, column name, and current type for all columns that require migration.
Step 2: Run the migration
Before you run the migration, take note of the following items:
-
We recommend that you first verify query compatibility in a test environment or on a cloned table. The path access syntax of the new
JSONtype differs from the legacyObjecttype. For more information, see Step 3: Adapt queries after migration. -
The mutation reads the entire column and rewrites data parts. Large tables consume significant memory and disk I/O. We recommend that you run the migration during off-peak hours. For large tables, migrate data partition by partition.
You can choose one of the following methods based on your business requirements.
Method 1: In-place column modification (recommended)
For each column to migrate, execute an ALTER TABLE ... MODIFY COLUMN statement to change the type to the new JSON:
ALTER TABLE <db>.<table>
MODIFY COLUMN <column> JSON
SETTINGS mutations_sync = 1;
-- On versions earlier than 25.3, enable enable_json_type = 1 in the session
Parameter description:
-
mutations_sync = 1: waits for the mutation to complete synchronously on the current replica. -
This mutation rewrites all data parts that contain the column, converting the legacy
Objectstorage format to the newJSONstorage format.
Method 2: Create a new table and migrate data
If you prefer to migrate data by creating a new table, follow these steps:
1. Create a new table with the same structure as the old table:
CREATE TABLE new_table AS old_table;
2. Modify the column type of the new table to the new JSON:
ALTER TABLE new_table MODIFY COLUMN `your_json_column` JSON;
3. Insert data from the old table into the new table:
INSERT INTO new_table SELECT * FROM old_table;
Step 3: Adapt queries after migration
After the migration, write operations (INSERT) typically do not require changes, but read operations (SELECT) may need adaptation. The key difference is that path access on the legacy Object type returns inferred concrete types, while path access on the new JSON type returns the Dynamic type.
Write operations (no changes required)
Both types accept JSON documents as input. Insert code typically does not require changes. The only exception is when your code hardcodes the type name as a string, such as in DDL statements or CAST target types that use Object('json'). In these cases, change the type name to JSON. Pure data write paths are not affected.
|
Write method |
Legacy Object(JSON) |
New JSON |
|
Row-format import |
|
Identical |
|
Import entire column as string |
|
Identical |
|
|
|
Identical |
|
CAST from |
|
|
Read operations (review required)
Review and adapt your queries based on the following scenarios:
|
Scenario |
Legacy Object syntax |
New JSON syntax |
|
Retrieve values for display or passthrough (no computation) |
|
|
|
Scalar path in expressions, functions, or comparisons |
|
|
|
|
|
|
|
Object arrays |
|
The array is |
|
Read nested sub-objects |
|
Use |
Examples of the new syntax:
-- Scalar type annotation
SELECT json.a.g.:Float64, json.d.:Date FROM t;
-- GROUP BY / ORDER BY requires explicit settings
SELECT json.repo.name, count() AS c
FROM t
GROUP BY json.repo.name
ORDER BY c DESC
SETTINGS allow_suspicious_types_in_group_by = 1, allow_suspicious_types_in_order_by = 1;
-- Object arrays: [] denotes the array level, compatible with ARRAY JOIN
SELECT json.payload.commits[].author.name FROM t;
-- Read entire sub-object
SELECT json.^metadata FROM t;
Step 4: Verify the migration
Confirm that no Object(JSON) columns remain in the cluster:
SELECT count() FROM system.columns
WHERE type LIKE '%Object(%json%)%'
AND database NOT IN ('system', 'information_schema', 'INFORMATION_SCHEMA');
Confirm that all JSON-related mutations are complete:
SELECT count() FROM system.mutations
WHERE command ILIKE '%JSON%' AND is_done = 0;
If both queries return 0, the migration is complete for the instance.
FAQ
Memory limit exceeded on large tables
Large tables may trigger a memory limit error when the entire column is cast to JSON. A typical error message is as follows:
latest_fail_reason: (total) memory limit exceeded: would use 7.20 GiB
... while executing 'FUNCTION _CAST(<column> :: 0, 'JSON' :: 1) -> _CAST(<column>, 'JSON') JSON'
(while reading from part .../all_1_1_0 located on disk s3WithKeeperDisk of type s3)
While executing MergeTreeSequentialSource
latest_fail_error_code_name: MEMORY_LIMIT_EXCEEDED
You can use the following methods to mitigate this issue:
-
Scale up or temporarily increase the instance memory limit.
-
Upgrade to a later version (we recommend upgrading to 25.10) and retry.
-
Migrate large tables partition by partition to reduce peak memory usage per mutation.
A failed mutation does not cause data loss. After you resolve the issue, the mutation automatically retries on the failed part until is_done = 1.
Monitor migration progress
Run the following query to check the progress of in-progress JSON-related mutations:
SELECT *
FROM system.mutations
WHERE command ILIKE '%JSON%'
AND is_done = 0;
Pay attention to the parts_to_do (remaining parts), is_done, and latest_fail_reason fields.