Array type (Public Preview)

Updated at:

The array data type is used to store variable-length fields. The Lindorm wide table engine supports arrays for any non-primary key column. The supported element types include integer (INT/INTEGER, BIGINT, SMALLINT, and TINYINT), floating-point (FLOAT and DOUBLE), string (VARCHAR), and Boolean (BOOLEAN). This topic describes how to use the array type.

Engine and version

The array type is available only for wide table engine versions 2.7.8 and later.

Important
  • The array data type is currently in public preview. To use this feature, contact Lindorm technical support on DingTalk (ID: s0s3eg3) to enable it.

  • Lindorm SQL version 2.8.7.0 or later is required. To view your Lindorm SQL version, see SQL version guide.

Prerequisites

Limits

The array type is not supported for primary key columns or index columns in Lindorm wide tables.

DDL

When you create or modify a table, you can specify the data type of a column as an array. For more information about the syntax, see CREATE TABLE.

The following are examples:

-- Create the sample table t1
CREATE TABLE t1 (id INT, intArr ARRAY<INT>, integerArr ARRAY<INTEGER>, bigintArr ARRAY<BIGINT>, smallintArr ARRAY<SMALLINT>, tinyintArr ARRAY<TINYINT>, floatArr ARRAY<FLOAT>, doubleArr ARRAY<DOUBLE>, varcharArr ARRAY<VARCHAR>, booleanArr ARRAY<BOOLEAN>, PRIMARY KEY(id));

-- Create the sample table t2
CREATE TABLE t2 (id INT, intArr INT[], integerArr INTEGER[], bigintArr BIGINT[], smallintArr SMALLINT[], tinyintArr TINYINT[], floatArr FLOAT[], doubleArr DOUBLE[], varcharArr VARCHAR[], booleanArr BOOLEAN[], PRIMARY KEY(id));

In the preceding examples, tables t1 and t2 have the same structure. The id column is the primary key column, and its data type is INT. The intArr, integerArr, bigintArr, smallintArr, tinyintArr, floatArr, doubleArr, varcharArr, and booleanArr columns are non-primary key columns. Their data types are INT array, INT array, BIGINT array, SMALLINT array, TINYINT array, FLOAT array, DOUBLE array, VARCHAR array, and BOOLEAN array, respectively.

Note

Lindorm currently supports only one-dimensional arrays.

Execute the following statement to view the structure of a sample table:

DESCRIBE <sample_table_name>;

For table t1, the result is as follows:

+--------------+------------+-------------+-----------------+----------------+------------+-------------------+-----------------+
| TABLE_SCHEMA | TABLE_NAME | COLUMN_NAME | TYPE            | IS_PRIMARY_KEY | SORT_ORDER | IS_AUTO_INCREMENT | IS_LOCAL_UNIQUE |
+--------------+------------+-------------+-----------------+----------------+------------+-------------------+-----------------+
| default      | t1         | id          | INT             | true           | ASC        | false             | false           |
| default      | t1         | intArr      | ARRAY<INT>      | false          | none       | false             | false           |
| default      | t1         | integerArr  | ARRAY<INT>      | false          | none       | false             | false           |
| default      | t1         | bigintArr   | ARRAY<BIGINT>   | false          | none       | false             | false           |
| default      | t1         | smallintArr | ARRAY<SMALLINT> | false          | none       | false             | false           |
| default      | t1         | tinyintArr  | ARRAY<TINYINT>  | false          | none       | false             | false           |
| default      | t1         | floatArr    | ARRAY<FLOAT>    | false          | none       | false             | false           |
| default      | t1         | doubleArr   | ARRAY<DOUBLE>   | false          | none       | false             | false           |
| default      | t1         | varcharArr  | ARRAY<VARCHAR>  | false          | none       | false             | false           |
| default      | t1         | booleanArr  | ARRAY<BOOLEAN>  | false          | none       | false             | false           |
+--------------+------------+-------------+-----------------+----------------+------------+-------------------+-----------------+

You can view the structure of the sample table t2 in the same way. The structure of t2 is identical to that of t1.

DML

UPSERT

You can write array data in one of the following two ways.

  • Use square brackets ([]) or curly braces ({}) to enclose the array elements and write the array as a string. For example: '[element1, element2, ...]' or '{element1, element2, ...}'.

    Note

    If an element is a string, enclose it in double quotation marks (""). For example, '["a", "b", "c"]'.

    The following are examples:

    UPSERT INTO t1(id, intArr, integerArr, bigintArr, smallintArr, tinyintArr, floatArr, doubleArr, varcharArr, booleanArr) VALUES (1, '[1, 2]', '[1, 2]', '[1, 2]', '[1, 2]', '[1, 2]', '[1.1, 2.2]', '[1.1, 2.2]', '["a", "b"]', '[true, false]');
    
    UPSERT INTO t2(id, intArr, integerArr, bigintArr, smallintArr, tinyintArr, floatArr, doubleArr, varcharArr, booleanArr) VALUES (1, '{1, 2}', '{1, 2}', '{1, 2}', '{1, 2}', '{1, 2}', '{1.1, 2.2}', '{1.1, 2.2}', '{"a", "b"}', '{true, false}');
  • Use the array constructor: ARRAY[element1, element2, ...].

    Note

    If an element is a string, enclose it in single quotation marks (''). For example, ARRAY['a', 'b', 'c'].

    The following is an example:

    UPSERT INTO t2(id, intArr, integerArr, bigintArr, smallintArr, tinyintArr, floatArr, doubleArr, varcharArr, booleanArr) VALUES (2, ARRAY[3, 4], ARRAY[3, 4], ARRAY[3, 4], ARRAY[3, 4], ARRAY[3, 4], ARRAY[3.3, 4.4], ARRAY[3.3, 4.4], ARRAY['c', 'd'], ARRAY[true, false]);

UPDATE

Use the array_append function to add elements to the end of an array. You can add up to four elements at a time.

  • Example 1: Use the array_append function to add one element to the end of an array.

    UPDATE t1 SET intArr = ARRAY_APPEND(intArr, 3), integerArr = ARRAY_APPEND(integerArr, 3), bigintArr = ARRAY_APPEND(bigintArr, 3), smallintArr = ARRAY_APPEND(smallintArr, 3), tinyintArr = ARRAY_APPEND(tinyintArr, 3), floatArr = ARRAY_APPEND(floatArr, 3.3), doubleArr = ARRAY_APPEND(doubleArr, 3.3), varcharArr = ARRAY_APPEND(varcharArr, 'c'), booleanArr = ARRAY_APPEND(booleanArr, true) WHERE id = 1;
  • Example 2: Use the array_append function to add four elements to the end of an array.

    UPDATE t1 SET intArr = ARRAY_APPEND(intArr, 4, 5, 6, 7), integerArr = ARRAY_APPEND(integerArr, 4, 5, 6, 7), bigintArr = ARRAY_APPEND(bigintArr, 4, 5, 6, 7), smallintArr = ARRAY_APPEND(smallintArr, 4, 5, 6, 7), tinyintArr = ARRAY_APPEND(tinyintArr, 4, 5, 6, 7), floatArr = ARRAY_APPEND(floatArr, 4.4, 5.5, 6.6, 7.7), doubleArr = ARRAY_APPEND(doubleArr, 4.4, 5.5, 6.6, 7.7), varcharArr = ARRAY_APPEND(varcharArr, 'd', 'e', 'f', 'g'), booleanArr = ARRAY_APPEND(booleanArr, false, false, false, false) WHERE id = 1;
Note

You can add null elements to an array. For example, ARRAY_APPEND(intArr, 1, null).

SELECT

Query all columns

The array is returned as a JSON array string.

  • Example 1: Query all data in table t1.

    SELECT * FROM t1;

    Result:

    +------+-----------------+-----------------+-----------------+-----------------+-----------------+-------------------------------+-------------------------------+-------------------------------+-------------------------------------------+
    | id   | intArr          | integerArr      | bigintArr       | smallintArr     | tinyintArr      | floatArr                      | doubleArr                     | varcharArr                    | booleanArr                                |
    +------+-----------------+-----------------+-----------------+-----------------+-----------------+-------------------------------+-------------------------------+-------------------------------+-------------------------------------------+
    |    1 | [1,2,3,4,5,6,7] | [1,2,3,4,5,6,7] | [1,2,3,4,5,6,7] | [1,2,3,4,5,6,7] | [1,2,3,4,5,6,7] | [1.1,2.2,3.3,4.4,5.5,6.6,7.7] | [1.1,2.2,3.3,4.4,5.5,6.6,7.7] | ["a","b","c","d","e","f","g"] | [true,false,true,false,false,false,false] |
    +------+-----------------+-----------------+-----------------+-----------------+-----------------+-------------------------------+-------------------------------+-------------------------------+-------------------------------------------+
  • Example 2: Query all data in table t2.

    SELECT * FROM t2;

    Result:

    +------+--------+------------+-----------+-------------+------------+-----------+-----------+------------+--------------+
    | id   | intArr | integerArr | bigintArr | smallintArr | tinyintArr | floatArr  | doubleArr | varcharArr | booleanArr   |
    +------+--------+------------+-----------+-------------+------------+-----------+-----------+------------+--------------+
    |    1 | [1,2]  | [1,2]      | [1,2]     | [1,2]       | [1,2]      | [1.1,2.2] | [1.1,2.2] | ["a","b"]  | [true,false] |
    |    2 | [3,4]  | [3,4]      | [3,4]     | [3,4]       | [3,4]      | [3.3,4.4] | [3.3,4.4] | ["c","d"]  | [true,false] |
    +------+--------+------------+-----------+-------------+------------+-----------+-----------+------------+--------------+

Query a specific element in an array

You can use arr to retrieve the i-th element from the arr array. The index is 1-based. If the index is less than 1 or greater than the array length, NULL is returned.

  • Example 1: Retrieve the i-th element of an array.

    SELECT intArr[1], integerArr[2], bigintArr[3], smallintArr[4], tinyintArr[5], floatArr[6], doubleArr[7], varcharArr[1], booleanArr[1], booleanArr[2] FROM t1 WHERE id = 1;

    Result:

    +-----------+---------------+--------------+----------------+---------------+-------------+--------------+---------------+---------------+---------------+
    | intArr[1] | integerArr[2] | bigintArr[3] | smallintArr[4] | tinyintArr[5] | floatArr[6] | doubleArr[7] | varcharArr[1] | booleanArr[1] | booleanArr[2] |
    +-----------+---------------+--------------+----------------+---------------+-------------+--------------+---------------+---------------+---------------+
    |         1 |             2 |            3 |              4 |             5 |         6.6 |          7.7 | a             |             1 |             0 |
    +-----------+---------------+--------------+----------------+---------------+-------------+--------------+---------------+---------------+---------------+
  • Example 2: (Abnormal use case) Retrieve the i-th element of an array where i is less than 1 or greater than the array length.

    SELECT intArr[0], integerArr[10], doubleArr[-1], varcharArr[100] FROM t1 WHERE id = 1;

    Result:

    +-----------+----------------+---------------+-----------------+
    | intArr[0] | integerArr[10] | doubleArr[-1] | varcharArr[100] |
    +-----------+----------------+---------------+-----------------+
    |      NULL |           NULL |          NULL | NULL            |
    +-----------+----------------+---------------+-----------------+

Equality query

Use the equivalent condition array = array_literal to retrieve data that meets the condition.

Note
  • Two arrays are considered equal only if they are identical, including the order of their elements.

  • The array constant array_literal must be represented using the array constructor: ARRAY[element1, element2, ...].

  • Example 1: Query for rows where intArr is equal to the array [1,2,3,4,5,6,7] and return their IDs.

    SELECT id FROM t1 WHERE id >= 0 AND id <= 10 AND intArr = ARRAY[1,2,3,4,5,6,7];

    Result:

    +------+
    | id   |
    +------+
    |    1 |
    +------+
  • Example 2: Query for rows where varcharArr is equal to the array ['c', 'd'] and return their IDs.

    SELECT id FROM t2 WHERE id >= 0 AND id <= 10 AND varcharArr = ARRAY['c', 'd'];

    Result:

    +------+
    | id   |
    +------+
    |    2 |
    +------+

Functions

Function overview

For array data, Lindorm provides the following functions to support more complex query requirements.

Function categorization

Function

Description

Containment check (array contains element)

element = ANY(arr)

Checks if an array contains a specified element.

  • arr: The name of the array column.

  • element: A constant of the same type as the array elements.

ARRAY_CONTAINS(arr, element)

Containment check (array contains array)

ARRAY_CONTAINS(arr, array_literal)

Checks if an array contains every element of a specified array array_literal.

  • arr: The name of the array column.

  • array_literal: An array constant that must be represented by an array constructor, that is, ARRAY[element1, element2, ...].

arr is the array column name, and array_literal is an array constant.

Array overlap check

ARRAYS_OVERLAP(arr, array_literal)

Checks if the array arr and the constant array_literal have common elements.

  • arr: The name of the array column.

  • array_literal: An array constant that must be represented by the array constructor: ARRAY[element1, element2, ...].

Count array elements

CARDINALITY(arr)

Returns the number of elements in the array. arr is the name of the array column.

ARRAY_SIZE(arr)

Containment check (array contains element)

element = ANY(arr)

Checks if the array column arr contains the element element. If element is null, the function returns null. If arr is empty, it returns false.

  • Example 1: Check if the INT array column intArr contains 1.

    SELECT id, intArr FROM t1 WHERE id >= 0 AND id <= 10 AND 1 = ANY(intArr);

    Result:

    +------+-----------------+
    | id   | intArr          |
    +------+-----------------+
    |    1 | [1,2,3,4,5,6,7] |
    +------+-----------------+
  • Example 2: Check if the DOUBLE array column doubleArr contains 1.1.

    SELECT id, doubleArr FROM t1 WHERE id >= 0 AND id <= 10 AND 1.1 = ANY(doubleArr);

    Result:

    +------+-------------------------------+
    | id   | doubleArr                     |
    +------+-------------------------------+
    |    1 | [1.1,2.2,3.3,4.4,5.5,6.6,7.7] |
    +------+-------------------------------+
  • Example 3: Check if the VARCHAR array column varcharArr contains c.

    SELECT id, varcharArr FROM t2 WHERE id >= 0 AND id <= 10 AND 'c' = ANY(varcharArr);

    Result:

    +------+------------+
    | id   | varcharArr |
    +------+------------+
    |    2 | ["c","d"]  |
    +------+------------+
  • Example 4: Check if the BOOLEAN array column contains the Boolean constant true.

    SELECT id, booleanArr FROM t2 WHERE id >= 0 AND id <= 10 AND true = ANY(booleanArr);

    Result:

    +------+--------------+
    | id   | booleanArr   |
    +------+--------------+
    |    1 | [true,false] |
    |    2 | [true,false] |
    +------+--------------+

ARRAY_CONTAINS(arr, element)

Checks if the array column arr contains the element element.

  • Example 1: Check if the INT array column intArr contains 1.

    SELECT id, intArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(intArr, 1);

    Result:

    +------+-----------------+
    | id   | intArr          |
    +------+-----------------+
    |    1 | [1,2,3,4,5,6,7] |
    +------+-----------------+
  • Example 2: Check if the DOUBLE array column doubleArr contains 1.1.

    SELECT id, doubleArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(doubleArr, 1.1);

    Result:

    +------+-------------------------------+
    | id   | doubleArr                     |
    +------+-------------------------------+
    |    1 | [1.1,2.2,3.3,4.4,5.5,6.6,7.7] |
    +------+-------------------------------+
  • Example 3: Check if the VARCHAR array column varcharArr contains c.

    SELECT id, varcharArr FROM t2 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(varcharArr, 'c');

    Result:

    +------+------------+
    | id   | varcharArr |
    +------+------------+
    |    2 | ["c","d"]  |
    +------+------------+
  • Example 4: Check if the BOOLEAN array column contains the Boolean constant true.

    SELECT id, booleanArr FROM t2 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(booleanArr, false);

    Result:

    +------+--------------+
    | id   | booleanArr   |
    +------+--------------+
    |    1 | [true,false] |
    |    2 | [true,false] |
    +------+--------------+

Containment check (array contains array)

Use the ARRAY_CONTAINS(arr, array_literal) function to check if the array column arr contains every element of the array array_literal.

Important

This function checks if arr contains every element in array_literal. If either arr or array_literal is null, the function returns null. If array_literal contains a null element, the function also returns null.

A return value of true does not mean that array_literal is a subset of arr. For example, if arr is ARRAY[1] and array_literal is ARRAY[1, 1, 1], the function returns true. However, array_literal has more elements than arr and is not a subset of arr.

  • Example 1: Check if the INT array column intArr contains every element of ARRAY[1,2].

    SELECT id, intArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(intArr, ARRAY[1,2]);

    Result:

    +------+-----------------+
    | id   | intArr          |
    +------+-----------------+
    |    1 | [1,2,3,4,5,6,7] |
    +------+-----------------+
  • Example 2: Check if the DOUBLE array column doubleArr contains every element of ARRAY[1.1,1.1].

    SELECT id, doubleArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(doubleArr, ARRAY[1.1, 1.1]);

    Result:

    +------+-------------------------------+
    | id   | doubleArr                     |
    +------+-------------------------------+
    |    1 | [1.1,2.2,3.3,4.4,5.5,6.6,7.7] |
    +------+-------------------------------+
  • Example 3: Check if the VARCHAR array column varcharArr contains every element of ARRAY['c', 'b', 'a'].

    SELECT id, varcharArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(varcharArr, ARRAY['c', 'b', 'a']);

    Result:

    +------+-------------------------------+
    | id   | varcharArr                    |
    +------+-------------------------------+
    |    1 | ["a","b","c","d","e","f","g"] |
    +------+-------------------------------+
  • Example 4: Check if the BOOLEAN array column booleanArr contains every element of ARRAY[true, true, true].

    SELECT id, booleanArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAY_CONTAINS(booleanArr, ARRAY[true, true, true]);

    Result:

    +------+-------------------------------------------+
    | id   | booleanArr                                |
    +------+-------------------------------------------+
    |    1 | [true,false,true,false,false,false,false] |
    +------+-------------------------------------------+

Array overlap check

Use the ARRAYS_OVERLAP(arr, array_literal) function to check if the array column arr and the array array_literal have any elements in common. If either parameter is null, the function returns null. If there are no common elements, or if the only common elements are null, the function returns false.

  • Example 1: Check if the INT array column intArr and the array ARRAY[1, 10, 100] have common elements.

    SELECT id, intArr FROM t1 WHERE id >= 0 AND id <= 10 AND ARRAYS_OVERLAP(intArr, ARRAY[1, 10, 100]);

    Result:

    +------+-----------------+
    | id   | intArr          |
    +------+-----------------+
    |    1 | [1,2,3,4,5,6,7] |
    +------+-----------------+
  • Example 2: Check if the DOUBLE array column doubleArr and the array ARRAY[1.1, 1.1] have common elements.

    SELECT id, doubleArr FROM t2 WHERE id >= 0 AND id <= 10 AND ARRAYS_OVERLAP(doubleArr, ARRAY[1.1, 1.1]);

    Result:

    +------+-----------+
    | id   | doubleArr |
    +------+-----------+
    |    1 | [1.1,2.2] |
    +------+-----------+
  • Example 3: Check if the VARCHAR array column varcharArr and the array ARRAY['a', 'c'] have common elements.

    SELECT id, varcharArr FROM t2 WHERE id >= 0 AND id <= 10 AND ARRAYS_OVERLAP(varcharArr, ARRAY['a', 'c']);

    Result:

    +------+------------+
    | id   | varcharArr |
    +------+------------+
    |    1 | ["a","b"]  |
    |    2 | ["c","d"]  |
    +------+------------+
  • Example 4: Check if the BOOLEAN array column booleanArr and the array ARRAY[true] have common elements.

    SELECT id, booleanArr FROM t2 WHERE id >= 0 AND id <= 10 AND ARRAYS_OVERLAP(booleanArr, ARRAY[true]);

    Result:

    +------+--------------+
    | id   | booleanArr   |
    +------+--------------+
    |    1 | [true,false] |
    |    2 | [true,false] |
    +------+--------------+

Count array elements

CARDINALITY(arr)

Returns the number of elements in the array column arr.

  • Example 1: Return the number of elements in the INT array column intArr.

    SELECT id, CARDINALITY(intArr) AS intArraySize FROM t1;

    Result:

    +------+--------------+
    | id   | intArraySize |
    +------+--------------+
    |    1 |            7 |
    +------+--------------+
  • Example 2: Count the rows in the sample table t2 where the ID is between 0 and 10, and the number of elements in intArr is greater than 5.

    SELECT COUNT(*) FROM t2 WHERE id >= 0 AND id <= 10 AND CARDINALITY(intArr) > 5;

    Result:

    +--------+
    | EXPR$0 |
    +--------+
    |      0 |
    +--------+
  • Example 3: Count the rows in the sample table t1 where the ID is between 0 and 10, and the number of elements in intArr is greater than 5.

    SELECT COUNT(*) FROM t1 WHERE id >= 0 AND id <= 10 AND CARDINALITY(intArr) > 5;

    Result:

    +--------+
    | EXPR$0 |
    +--------+
    |      1 |
    +--------+

ARRAY_SIZE(arr)

The ARRAY_SIZE(arr) function returns the number of elements in the array column arr.

The following example returns the number of elements in the VARCHAR array column varcharArr.

SELECT id, ARRAY_SIZE(varcharArr) AS varcharArraySize FROM t1;

Result:

+------+------------------+
| id   | varcharArraySize |
+------+------------------+
|    1 |                7 |
+------+------------------+