当JSON列的查询需要解析整个JSON数据时,列存索引(IMCI)提供JSON索引功能,通过JSON分词器将JSON数据拆解并构建为倒排索引,避免加载和解析完整JSON对象,提升json_overlaps、json_contains和json_extract等表达式的查询效率。
列存索引JSON索引目前处于灰度阶段,若您有相关需求,请提交工单联系我们为您处理。
创建JSON索引
JSON索引通过JSON分词器将JSON对象拆解出数组项或键值对并添加到倒排索引,查询时直接通过索引判断是否命中,避免加载和解析整个JSON对象。JSON索引仅支持JSON类型的列,包含两种索引类型:
JSON数组索引:用于数组类型的JSON列,仅支持标量数组(即数组元素为整型、字符串或时间等),不适用于对象数组或嵌套数组。
JSON键值索引:用于对象类型的JSON列,且键值对的值类型为标量(即整型、字符串或时间等)或标量数组。通过
json_paths参数指定需要索引的键。
创建JSON索引的本质是创建倒排索引,用法与全文索引一致。更多信息,请参见IMCI全文索引使用说明。
操作步骤
开启JSON索引功能:使用JSON索引前,需要先开启全局参数
imci_enable_fts_json。参数名
级别
说明
imci_enable_fts_jsonGlobal
控制是否开启JSON索引功能。
ON(默认):开启。
OFF:关闭。
创建语法:
CREATE TABLE table_name ( column_name JSON COMMENT "imci_fts(type=4 mode=MODE json_paths={$.PATH1,$.PATH2,...})" ) COMMENT 'columnar=1';参数说明:
type=4:表示使用JSON分词器。mode:指定JSON索引类型。1为JSON数组索引,2为JSON键值索引。json_paths:指定JSON键值索引的键信息,仅在mode=2时使用。例如,对于'{"k1":"v1","k2":2}',可以指定json_paths={$.k1}、json_paths={$.k2}或json_paths={$.k1,$.k2}。
JSON数组索引
JSON数组索引用于加速数组类型JSON列的查询,支持json_overlaps和json_contains表达式。
创建语法
ALTER TABLE table_name MODIFY COLUMN column_name JSON DEFAULT NULL COMMENT 'imci_fts(type=4 mode=1)';加速json_overlaps查询
参数名 | 级别 | 说明 |
| Global/Session | 控制是否开启
|
开启后,json_overlaps表达式利用FtsTableScan加速。JSON分词器将目标值分解后分别在倒排索引中查询,最后求其并集(任意元素匹配即命中)。
mysql> EXPLAIN SELECT * FROM t1 WHERE json_overlaps(title, '[300, "301"]');
+----+----------------------+------+-----------------------------------------------------------------------------------------------------------------+
| ID | Operator | Name | Extra Info |
+----+----------------------+------+-----------------------------------------------------------------------------------------------------------------+
| 1 | Select Statement | | IMCI Execution Plan (max_dop = 1, max_query_mem = 1073741824) |
| 2 | └─Compute Scalar | | |
| 3 | └─FtsTableScan | t1 | Term: ("300(json_type(2))", "301(json_type(5))") Fallback: (JSON_OVERLAPS(t1.title, "[300, "301"](json)") <> 0) |
+----+----------------------+------+-----------------------------------------------------------------------------------------------------------------+加速json_contains查询
参数名 | 级别 | 说明 |
| Global/Session | 控制是否开启
|
开启后,json_contains表达式利用FtsTableScan加速。JSON分词器将目标值分解后分别在倒排索引中查询,最后求其交集(所有元素都匹配才命中)。
mysql> EXPLAIN SELECT * FROM t1 WHERE json_contains(title, '[300, "301"]');
+----+----------------------+------+-----------------------------------------------------------------------------------------------------------+
| ID | Operator | Name | Extra Info |
+----+----------------------+------+-----------------------------------------------------------------------------------------------------------+
| 1 | Select Statement | | IMCI Execution Plan (max_dop = 1, max_query_mem = 1073741824) |
| 2 | └─Compute Scalar | | |
| 3 | └─FtsTableScan | t1 | Term: ("300(json_type(2))", "301(json_type(5))") Fallback: (JSON_CONTAINS(t1.title, "[300, "301"]") <> 0) |
+----+----------------------+------+-----------------------------------------------------------------------------------------------------------+JSON键值索引
JSON键值索引用于加速对象类型JSON列中特定键的等值查询,支持json_extract表达式。索引创建时通过json_paths指定需要索引的键,查询时根据键值对的值在倒排索引中快速定位匹配行。
创建语法
ALTER TABLE table_name MODIFY COLUMN column_name JSON DEFAULT NULL COMMENT 'imci_fts(type=4 mode=2 json_paths={$.k1,$.k2})';加速json_extract查询
参数名 | 级别 | 说明 |
| Global/Session | 控制是否开启
|
开启后,json_extract等值表达式利用FtsTableScan加速。以下示例查询$.k1值等于'1'的行:
mysql> EXPLAIN SELECT * FROM t1 WHERE json_extract(title, "$.k1") = '1';
+----+----------------------+------+------------------------------------------------------------------------------------------------+
| ID | Operator | Name | Extra Info |
+----+----------------------+------+------------------------------------------------------------------------------------------------+
| 1 | Select Statement | | IMCI Execution Plan (max_dop = 1, max_query_mem = 1073741824) |
| 2 | └─Compute Scalar | | |
| 3 | └─FtsTableScan | t1 | Term: ("1(json_index(0),json_type(5))") Fallback: (JSON_EXTRACT(t1.title, "$.k1") = "1(json)") |
+----+----------------------+------+------------------------------------------------------------------------------------------------+多个键值条件可以在同一查询中同时加速:
mysql> EXPLAIN SELECT * FROM t1 WHERE json_extract(title, "$.k1") = '1' AND json_extract(title, "$.k2") = 2;
+----+----------------------+------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ID | Operator | Name | Extra Info |
+----+----------------------+------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 1 | Select Statement | | IMCI Execution Plan (max_dop = 1, max_query_mem = 1073741824) |
| 2 | └─Compute Scalar | | |
| 3 | └─FtsTableScan | t1 | Term: ("1(json_index(0),json_type(5))") Term: ("2(json_index(1),json_type(2))") Fallback: ((JSON_EXTRACT(t1.title, "$.k1") = "1(json)") AND (JSON_EXTRACT(t1.title, "$.k2") = "2(json)")) |
+----+----------------------+------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+JSON嵌套索引
JSON嵌套索引用于加速对象数组类型JSON列中特定键的数组查询,支持json_overlaps/json_contains与json_extract嵌套查询。索引创建时通过json_paths指定对象数组中的索引键,查询时根据键值对的值在倒排索引中快速定位匹配行。
创建语法
ALTER TABLE table_name MODIFY COLUMN column_name JSON DEFAULT NULL COMMENT 'imci_fts(type=4 mode=2 json_paths={$[*].k1,$[*].k2})';加速JSON嵌套查询
json_overlaps、json_contains与json_extract等嵌套查询可以利用FtsTableScan加速。以[{"id": 1, "name": "张三"}, {"id": 2, "name": "李四"}, {"id": 3, "name": "陈一"}] 为例,可以通过json_paths={$[*].id, $[*].name}将JSON对象数组中对应键值解析出来添加到索引来加速查询。
mysql> EXPLAIN SELECT * FROM t1 WHERE json_overlaps(title->'$[*].id', '[1, 2]');
+----+----------------------+------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ID | Operator | Name | Extra Info |
+----+----------------------+------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 1 | Select Statement | | IMCI Execution Plan (max_dop = 32, max_query_mem = 137438953472) |
| 2 | └─Compute Scalar | | |
| 3 | └─FtsTableScan | t1 | Term: (OR(1(json_index(0),json_type(2)), 2(json_index(0),json_type(2)))) Fallback: (JSON_OVERLAPS(JSON_EXTRACT(t1.title, "$[*].id"), "[1, 2](json)") <> 0) |
+----+----------------------+------+------------------------------------------------------------------------------------------------------------------------------------------------------------+mysql> EXPLAIN SELECT * FROM t1 WHERE json_contains(title->'$[*].name', '["张三", "陈一"]');
+----+----------------------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ID | Operator | Name | Extra Info |
+----+----------------------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 1 | Select Statement | | IMCI Execution Plan (max_dop = 32, max_query_mem = 137438953472) |
| 2 | └─Compute Scalar | | |
| 3 | └─FtsTableScan | t1 | Term: (AND(张三(json_index(1),json_type(5)), 陈一(json_index(1),json_type(5)))) Fallback: (JSON_CONTAINS(JSON_EXTRACT(t1.title, "$[*].name"), "["张三", "陈一"](json)") <> 0) |
+----+----------------------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+