Array type (Public Preview)
The array data type is used to store variable-length fields. The Lindorm wide table engine supports arrays for any non-primary key column. The supported element types include integer (INT/INTEGER, BIGINT, SMALLINT, and TINYINT), floating-point (FLOAT and DOUBLE), string (VARCHAR), and Boolean (BOOLEAN). This topic describes how to use the array type.
Engine and version
The array type is available only for wide table engine versions 2.7.8 and later.
The array data type is currently in public preview. To use this feature, contact Lindorm technical support on DingTalk (ID: s0s3eg3) to enable it.
Lindorm SQL version 2.8.7.0 or later is required. To view your Lindorm SQL version, see SQL version guide.
Prerequisites
Limits
The array type is not supported for primary key columns or index columns in Lindorm wide tables.
DDL
When you create or modify a table, you can specify the data type of a column as an array. For more information about the syntax, see CREATE TABLE.
The following are examples:
-- Create the sample table t1
CREATE TABLE t1 (id INT, intArr ARRAY<INT>, integerArr ARRAY<INTEGER>, bigintArr ARRAY<BIGINT>, smallintArr ARRAY<SMALLINT>, tinyintArr ARRAY<TINYINT>, floatArr ARRAY<FLOAT>, doubleArr ARRAY<DOUBLE>, varcharArr ARRAY<VARCHAR>, booleanArr ARRAY<BOOLEAN>, PRIMARY KEY(id));
-- Create the sample table t2
CREATE TABLE t2 (id INT, intArr INT[], integerArr INTEGER[], bigintArr BIGINT[], smallintArr SMALLINT[], tinyintArr TINYINT[], floatArr FLOAT[], doubleArr DOUBLE[], varcharArr VARCHAR[], booleanArr BOOLEAN[], PRIMARY KEY(id));In the preceding examples, tables t1 and t2 have the same structure. The id column is the primary key column, and its data type is INT. The intArr, integerArr, bigintArr, smallintArr, tinyintArr, floatArr, doubleArr, varcharArr, and booleanArr columns are non-primary key columns. Their data types are INT array, INT array, BIGINT array, SMALLINT array, TINYINT array, FLOAT array, DOUBLE array, VARCHAR array, and BOOLEAN array, respectively.
Lindorm currently supports only one-dimensional arrays.
Execute the following statement to view the structure of a sample table:
DESCRIBE <sample_table_name>;For table t1, the result is as follows:
+--------------+------------+-------------+-----------------+----------------+------------+-------------------+-----------------+
| TABLE_SCHEMA | TABLE_NAME | COLUMN_NAME | TYPE | IS_PRIMARY_KEY | SORT_ORDER | IS_AUTO_INCREMENT | IS_LOCAL_UNIQUE |
+--------------+------------+-------------+-----------------+----------------+------------+-------------------+-----------------+
| default | t1 | id | INT | true | ASC | false | false |
| default | t1 | intArr | ARRAY<INT> | false | none | false | false |
| default | t1 | integerArr | ARRAY<INT> | false | none | false | false |
| default | t1 | bigintArr | ARRAY<BIGINT> | false | none | false | false |
| default | t1 | smallintArr | ARRAY<SMALLINT> | false | none | false | false |
| default | t1 | tinyintArr | ARRAY<TINYINT> | false | none | false | false |
| default | t1 | floatArr | ARRAY<FLOAT> | false | none | false | false |
| default | t1 | doubleArr | ARRAY<DOUBLE> | false | none | false | false |
| default | t1 | varcharArr | ARRAY<VARCHAR> | false | none | false | false |
| default | t1 | booleanArr | ARRAY<BOOLEAN> | false | none | false | false |
+--------------+------------+-------------+-----------------+----------------+------------+-------------------+-----------------+You can view the structure of the sample table t2 in the same way. The structure of t2 is identical to that of t1.
DML
UPSERT
You can write array data in one of the following two ways.
Use square brackets ([]) or curly braces (
{}) to enclose the array elements and write the array as a string. For example:'[element1, element2, ...]'or'{element1, element2, ...}'.NoteIf an element is a string, enclose it in double quotation marks (""). For example,
'["a", "b", "c"]'.The following are examples:
UPSERT INTO t1(id, intArr, integerArr, bigintArr, smallintArr, tinyintArr, floatArr, doubleArr, varcharArr, booleanArr) VALUES (1, '[1, 2]', '[1, 2]', '[1, 2]', '[1, 2]', '[1, 2]', '[1.1, 2.2]', '[1.1, 2.2]', '["a", "b"]', '[true, false]'); UPSERT INTO t2(id, intArr, integerArr, bigintArr, smallintArr, tinyintArr, floatArr, doubleArr, varcharArr, booleanArr) VALUES (1, '{1, 2}', '{1, 2}', '{1, 2}', '{1, 2}', '{1, 2}', '{1.1, 2.2}', '{1.1, 2.2}', '{"a", "b"}', '{true, false}');Use the array constructor:
ARRAY[element1, element2, ...].NoteIf an element is a string, enclose it in single quotation marks (''). For example,
ARRAY['a', 'b', 'c'].The following is an example:
UPSERT INTO t2(id, intArr, integerArr, bigintArr, smallintArr, tinyintArr, floatArr, doubleArr, varcharArr, booleanArr) VALUES (2, ARRAY[3, 4], ARRAY[3, 4], ARRAY[3, 4], ARRAY[3, 4], ARRAY[3, 4], ARRAY[3.3, 4.4], ARRAY[3.3, 4.4], ARRAY['c', 'd'], ARRAY[true, false]);
UPDATE
Use the array_append function to add elements to the end of an array. You can add up to four elements at a time.
Example 1: Use the
array_appendfunction to add one element to the end of an array.UPDATE t1 SET intArr = ARRAY_APPEND(intArr, 3), integerArr = ARRAY_APPEND(integerArr, 3), bigintArr = ARRAY_APPEND(bigintArr, 3), smallintArr = ARRAY_APPEND(smallintArr, 3), tinyintArr = ARRAY_APPEND(tinyintArr, 3), floatArr = ARRAY_APPEND(floatArr, 3.3), doubleArr = ARRAY_APPEND(doubleArr, 3.3), varcharArr = ARRAY_APPEND(varcharArr, 'c'), booleanArr = ARRAY_APPEND(booleanArr, true) WHERE id = 1;Example 2: Use the
array_appendfunction to add four elements to the end of an array.UPDATE t1 SET intArr = ARRAY_APPEND(intArr, 4, 5, 6, 7), integerArr = ARRAY_APPEND(integerArr, 4, 5, 6, 7), bigintArr = ARRAY_APPEND(bigintArr, 4, 5, 6, 7), smallintArr = ARRAY_APPEND(smallintArr, 4, 5, 6, 7), tinyintArr = ARRAY_APPEND(tinyintArr, 4, 5, 6, 7), floatArr = ARRAY_APPEND(floatArr, 4.4, 5.5, 6.6, 7.7), doubleArr = ARRAY_APPEND(doubleArr, 4.4, 5.5, 6.6, 7.7), varcharArr = ARRAY_APPEND(varcharArr, 'd', 'e', 'f', 'g'), booleanArr = ARRAY_APPEND(booleanArr, false, false, false, false) WHERE id = 1;
You can add null elements to an array. For example, ARRAY_APPEND(intArr, 1, null).
SELECT
Query all columns
The array is returned as a JSON array string.
Example 1: Query all data in table
t1.SELECT * FROM t1;Result:
+------+-----------------+-----------------+-----------------+-----------------+-----------------+-------------------------------+-------------------------------+-------------------------------+-------------------------------------------+ | id | intArr | integerArr | bigintArr | smallintArr | tinyintArr | floatArr | doubleArr | varcharArr | booleanArr | +------+-----------------+-----------------+-----------------+-----------------+-----------------+-------------------------------+-------------------------------+-------------------------------+-------------------------------------------+ | 1 | [1,2,3,4,5,6,7] | [1,2,3,4,5,6,7] | [1,2,3,4,5,6,7] | [1,2,3,4,5,6,7] | [1,2,3,4,5,6,7] | [1.1,2.2,3.3,4.4,5.5,6.6,7.7] | [1.1,2.2,3.3,4.4,5.5,6.6,7.7] | ["a","b","c","d","e","f","g"] | [true,false,true,false,false,false,false] | +------+-----------------+-----------------+-----------------+-----------------+-----------------+-------------------------------+-------------------------------+-------------------------------+-------------------------------------------+Example 2: Query all data in table
t2.SELECT * FROM t2;Result:
+------+--------+------------+-----------+-------------+------------+-----------+-----------+------------+--------------+ | id | intArr | integerArr | bigintArr | smallintArr | tinyintArr | floatArr | doubleArr | varcharArr | booleanArr | +------+--------+------------+-----------+-------------+------------+-----------+-----------+------------+--------------+ | 1 | [1,2] | [1,2] | [1,2] | [1,2] | [1,2] | [1.1,2.2] | [1.1,2.2] | ["a","b"] | [true,false] | | 2 | [3,4] | [3,4] | [3,4] | [3,4] | [3,4] | [3.3,4.4] | [3.3,4.4] | ["c","d"] | [true,false] | +------+--------+------------+-----------+-------------+------------+-----------+-----------+------------+--------------+
Query a specific element in an array
You can use arr to retrieve the i-th element from the arr array. The index is 1-based. If the index is less than 1 or greater than the array length, NULL is returned.
Example 1: Retrieve the i-th element of an array.
SELECT intArr[1], integerArr[2], bigintArr[3], smallintArr[4], tinyintArr[5], floatArr[6], doubleArr[7], varcharArr[1], booleanArr[1], booleanArr[2] FROM t1 WHERE id = 1;Result:
+-----------+---------------+--------------+----------------+---------------+-------------+--------------+---------------+---------------+---------------+ | intArr[1] | integerArr[2] | bigintArr[3] | smallintArr[4] | tinyintArr[5] | floatArr[6] | doubleArr[7] | varcharArr[1] | booleanArr[1] | booleanArr[2] | +-----------+---------------+--------------+----------------+---------------+-------------+--------------+---------------+---------------+---------------+ | 1 | 2 | 3 | 4 | 5 | 6.6 | 7.7 | a | 1 | 0 | +-----------+---------------+--------------+----------------+---------------+-------------+--------------+---------------+---------------+---------------+Example 2: (Abnormal use case) Retrieve the i-th element of an array where
iis less than1or greater than the array length.SELECT intArr[0], integerArr[10], doubleArr[-1], varcharArr[100] FROM t1 WHERE id = 1;Result:
+-----------+----------------+---------------+-----------------+ | intArr[0] | integerArr[10] | doubleArr[-1] | varcharArr[100] | +-----------+----------------+---------------+-----------------+ | NULL | NULL | NULL | NULL | +-----------+----------------+---------------+-----------------+
Equality query
Use the equivalent condition array = array_literal to retrieve data that meets the condition.
Two arrays are considered equal only if they are identical, including the order of their elements.
The array constant
array_literalmust be represented using the array constructor:ARRAY[element1, element2, ...].
Example 1: Query for rows where
intArris equal to the array[1,2,3,4,5,6,7]and return their IDs.SELECT id FROM t1 WHERE id >= 0 AND id <= 10 AND intArr = ARRAY[1,2,3,4,5,6,7];Result:
+------+ | id | +------+ | 1 | +------+Example 2: Query for rows where
varcharArris equal to the array['c', 'd']and return their IDs.SELECT id FROM t2 WHERE id >= 0 AND id <= 10 AND varcharArr = ARRAY['c', 'd'];Result:
+------+ | id | +------+ | 2 | +------+
Functions
Function overview
For array data, Lindorm provides the following functions to support more complex query requirements.
Function categorization | Function | Description |
element = ANY(arr) | Checks if an array contains a specified element.
| |
ARRAY_CONTAINS(arr, element) | ||
ARRAY_CONTAINS(arr, array_literal) | Checks if an array contains every element of a specified array
| |
ARRAYS_OVERLAP(arr, array_literal) | Checks if the array
| |
CARDINALITY(arr) | Returns the number of elements in the array. | |
ARRAY_SIZE(arr) |
Containment check (array contains element)
element = ANY(arr)
Checks if the array column arr contains the element element. If element is null, the function returns null. If arr is empty, it returns false.
Example 1: Check if the INT array column
intArrcontains1.SELECT id, intArr FROM t1 WHERE id >= 0 AND id <= 10 AND 1 = ANY(intArr);Result:
+------+-----------------+ | id | intArr | +------+-----------------+ | 1 | [1,2,3,4,5,6,7] | +------+-----------------+Example 2: Check if the DOUBLE array column
doubleArrcontains1.1.SELECT id, doubleArr FROM t1 WHERE id >= 0 AND id <= 10 AND 1.1 = ANY(doubleArr);Result:
+------+-------------------------------+ | id | doubleArr | +------+-------------------------------+ | 1 | [1.1,2.2,3.3,4.4,5.5,6.6,7.7] | +------+-------------------------------+Example 3: Check if the VARCHAR array column
varcharArrcontainsc.SELECT id, varcharArr FROM t2 WHERE id >= 0 AND id <= 10 AND 'c' = ANY(varcharArr);Result:
+------+------------+ | id | varcharArr | +------+------------+ | 2 | ["c","d"] | +------+------------+Example 4: Check if the BOOLEAN array column contains the Boolean constant true.
SELECT id, booleanArr FROM t2 WHERE id >= 0 AND id <= 10 AND true = ANY(booleanArr);Result:
+------+--------------+ | id | booleanArr | +------+--------------+ | 1 | [true,false] | | 2 | [true,false] | +------+--------------+
ARRAY_CONTAINS(arr, element)
Checks if the array column arr contains the element element.
Example 1: Check if the INT array column
intArrcontains1.SELECT id, intArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(intArr, 1);Result:
+------+-----------------+ | id | intArr | +------+-----------------+ | 1 | [1,2,3,4,5,6,7] | +------+-----------------+Example 2: Check if the DOUBLE array column
doubleArrcontains1.1.SELECT id, doubleArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(doubleArr, 1.1);Result:
+------+-------------------------------+ | id | doubleArr | +------+-------------------------------+ | 1 | [1.1,2.2,3.3,4.4,5.5,6.6,7.7] | +------+-------------------------------+Example 3: Check if the VARCHAR array column
varcharArrcontainsc.SELECT id, varcharArr FROM t2 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(varcharArr, 'c');Result:
+------+------------+ | id | varcharArr | +------+------------+ | 2 | ["c","d"] | +------+------------+Example 4: Check if the BOOLEAN array column contains the Boolean constant true.
SELECT id, booleanArr FROM t2 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(booleanArr, false);Result:
+------+--------------+ | id | booleanArr | +------+--------------+ | 1 | [true,false] | | 2 | [true,false] | +------+--------------+
Containment check (array contains array)
Use the ARRAY_CONTAINS(arr, array_literal) function to check if the array column arr contains every element of the array array_literal.
This function checks if arr contains every element in array_literal. If either arr or array_literal is null, the function returns null. If array_literal contains a null element, the function also returns null.
A return value of true does not mean that array_literal is a subset of arr. For example, if arr is ARRAY[1] and array_literal is ARRAY[1, 1, 1], the function returns true. However, array_literal has more elements than arr and is not a subset of arr.
Example 1: Check if the INT array column
intArrcontains every element ofARRAY[1,2].SELECT id, intArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(intArr, ARRAY[1,2]);Result:
+------+-----------------+ | id | intArr | +------+-----------------+ | 1 | [1,2,3,4,5,6,7] | +------+-----------------+Example 2: Check if the DOUBLE array column
doubleArrcontains every element ofARRAY[1.1,1.1].SELECT id, doubleArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(doubleArr, ARRAY[1.1, 1.1]);Result:
+------+-------------------------------+ | id | doubleArr | +------+-------------------------------+ | 1 | [1.1,2.2,3.3,4.4,5.5,6.6,7.7] | +------+-------------------------------+Example 3: Check if the VARCHAR array column
varcharArrcontains every element ofARRAY['c', 'b', 'a'].SELECT id, varcharArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(varcharArr, ARRAY['c', 'b', 'a']);Result:
+------+-------------------------------+ | id | varcharArr | +------+-------------------------------+ | 1 | ["a","b","c","d","e","f","g"] | +------+-------------------------------+Example 4: Check if the BOOLEAN array column
booleanArrcontains every element ofARRAY[true, true, true].SELECT id, booleanArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(booleanArr, ARRAY[true, true, true]);Result:
+------+-------------------------------------------+ | id | booleanArr | +------+-------------------------------------------+ | 1 | [true,false,true,false,false,false,false] | +------+-------------------------------------------+
Array overlap check
Use the ARRAYS_OVERLAP(arr, array_literal) function to check if the array column arr and the array array_literal have any elements in common. If either parameter is null, the function returns null. If there are no common elements, or if the only common elements are null, the function returns false.
Example 1: Check if the INT array column
intArrand the arrayARRAY[1, 10, 100]have common elements.SELECT id, intArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAYS_OVERLAP(intArr, ARRAY[1, 10, 100]);Result:
+------+-----------------+ | id | intArr | +------+-----------------+ | 1 | [1,2,3,4,5,6,7] | +------+-----------------+Example 2: Check if the DOUBLE array column
doubleArrand the arrayARRAY[1.1, 1.1]have common elements.SELECT id, doubleArr FROM t2 WHERE id >= 0 AND id <= 10 AND ARRAYS_OVERLAP(doubleArr, ARRAY[1.1, 1.1]);Result:
+------+-----------+ | id | doubleArr | +------+-----------+ | 1 | [1.1,2.2] | +------+-----------+Example 3: Check if the VARCHAR array column
varcharArrand the arrayARRAY['a', 'c']have common elements.SELECT id, varcharArr FROM t2 WHERE id >= 0 AND id <= 10 AND ARRAYS_OVERLAP(varcharArr, ARRAY['a', 'c']);Result:
+------+------------+ | id | varcharArr | +------+------------+ | 1 | ["a","b"] | | 2 | ["c","d"] | +------+------------+Example 4: Check if the BOOLEAN array column
booleanArrand the arrayARRAY[true]have common elements.SELECT id, booleanArr FROM t2 WHERE id >= 0 AND id <= 10 AND ARRAYS_OVERLAP(booleanArr, ARRAY[true]);Result:
+------+--------------+ | id | booleanArr | +------+--------------+ | 1 | [true,false] | | 2 | [true,false] | +------+--------------+
Count array elements
CARDINALITY(arr)
Returns the number of elements in the array column arr.
Example 1: Return the number of elements in the INT array column
intArr.SELECT id, CARDINALITY(intArr) AS intArraySize FROM t1;Result:
+------+--------------+ | id | intArraySize | +------+--------------+ | 1 | 7 | +------+--------------+Example 2: Count the rows in the sample table
t2where the ID is between 0 and 10, and the number of elements inintArris greater than 5.SELECT COUNT(*) FROM t2 WHERE id >= 0 AND id <= 10 AND CARDINALITY(intArr) > 5;Result:
+--------+ | EXPR$0 | +--------+ | 0 | +--------+Example 3: Count the rows in the sample table
t1where the ID is between 0 and 10, and the number of elements inintArris greater than 5.SELECT COUNT(*) FROM t1 WHERE id >= 0 AND id <= 10 AND CARDINALITY(intArr) > 5;Result:
+--------+ | EXPR$0 | +--------+ | 1 | +--------+
ARRAY_SIZE(arr)
The ARRAY_SIZE(arr) function returns the number of elements in the array column arr.
The following example returns the number of elements in the VARCHAR array column varcharArr.
SELECT id, ARRAY_SIZE(varcharArr) AS varcharArraySize FROM t1;Result:
+------+------------------+
| id | varcharArraySize |
+------+------------------+
| 1 | 7 |
+------+------------------+