Data types

更新时间:
复制 MD 格式

Before you create a search index, you must understand the data types supported by search indexes. You must also understand the mappings between the data types supported by search indexes and the data types supported by the data tables for which the search indexes are created.

Supported data types

Search index provides primitive data types, such as Long, Double, Boolean, Keyword, Text, Date, IP, Geopoint, Vector, and JSON. It also provides special types, such as array types, nested types, and virtual columns.

Primitive data types

Search index provides primitive data types, such as Long, Double, Boolean, Keyword, Text, Date, IP, Geopoint, Vector, and JSON. The following table describes these data types.

Primitive data type

Description

Long

A 64-bit long integer.

Double

A 64-bit double-precision floating-point number.

Boolean

A Boolean value.

Keyword

A non-tokenized string.

FuzzyKeyword

A string that supports high-performance fuzzy queries.

Text

A tokenized string or text.

Date

A date and time type that supports custom date formats.

IP

An IP data type that supports IP addresses in IPv4 and IPv6 formats.

Geopoint

Coordinate information for a location point. The format is latitude,longitude. The latitude comes first, followed by the longitude. The latitude must be in the range of [-90,+90], and the longitude must be in the range of [-180,+180]. Example: 35.8,-45.91.

Vector

A vector type. The format is a string of a Float32 array. The length of the array is equal to the dimension of the field. For example, the vector string [1, 5.1, 4.7, 0.08 ] has four dimensions.

JSON type

A JSON type that supports the OBJECT and NESTED index types.

Array and nested types

In addition to primitive data types, such as Long, Double, Boolean, Keyword, Text, Date, Geopoint, and Vector, search index also provides two special types: array and nested. The array type is suitable for storing a series of data of the same type. The nested type is suitable for storing data with a hierarchical structure, similar to JSON. For more information, see Array and nested types.

Array type

Important
  • The Array data type can be used only in search indexes. Data tables do not support the Array data type.

  • You can use the Array data type in the same manner you use a non-Array data type. When you use an Array field to query data, a row of data is returned if at least one of the values in the array meets the condition.

  • The vector data type cannot be used in arrays.

The Array data type is a modifier that can be combined with primitive data types, such as Long, Double, Boolean, Keyword, Text, Date, IP, and GeoPoint. For example, if you combine the Array type with the Long type, the result is a long integer array that can store multiple long integers. The Array data type is suitable for storing a series of values of the same type.

If the data type of a field in a search index is a combination of the Array data type and a primitive data type, such as Long or Double, the field in the data table for which the search index is created must be of the String type and the field in the search index must be of the corresponding primitive data type.

The following table describes the combinations of the Array data type and primitive data types in search indexes.

Array type

Description

Long Array

An array of long integers. Example: "[1000, 4, 5555]".

Double Array

An array of floating-point numbers. Example: "[3.1415926, 0.99]".

Boolean Array

An array of Boolean values. Example: "[true, false]".

Keyword Array

An array of strings. The format is JSON-formatted data. Example: "[\"Hangzhou\", \"Xi'an\"]".

Text Array

An array of text. The format is a JSON array. Example: "[\"Hangzhou\", \"Xi'an\"]".

Text arrays are not commonly used.

Date Array

An array of dates. If the date type is an integer, the format is "[1218197720123, 1712850436000]". If the date type is a string, the format is "[\"2024-04-11 23:47:16.854775807\", \"2024-06-11 23:47:16.854775807\"]".

IP Array

An array of IP addresses. The format is a JSON array. Example: "[\"192,168.1.1\", \"255.255.255.255\"]".

Geopoint Array

An array of geographic points. Example: "[\"34.2, 43.0\", \"21.4, 45.2\"]".

Nested type

Data of the Nested type is nested documents. Nested documents are used when a row of data (document) contains multiple child rows (child documents). Multiple child rows are stored in a nested field. The Nested data type is suitable for storing data that has a hierarchical structure.

You must specify the schema of child rows in a nested field. The schema must include the fields of the child rows and the property of each field. The Nested data type can be used to store multiple values, which is similar to the JSON data type.

If a field in a search index is of the Nested type, the field in the data table for which the search index is created must be of the String type, and the field in the search index must be of the Nested type. You must perform nested queries to query fields of the Nested type.

When you write data to a data table, the data table field that corresponds to the nested field in the search index must be a JSON array of objects. For example, [{"tagName":"tag1", "score":0.8,"time": 1730690237000 }, {"tagName":"tag2", "score":0.2,"time": 1730691557000}].

Important

You must write strings of the JSON Array type to a nested field regardless of whether the field contains only one child row.

Nested fields are classified into single-level and multi-level nested fields. The following table describes the two types.

Nesting type

Description

Single-level nested type

A single-level nested type contains one layer of another data structure. This creates a simple structure with a basic hierarchy. This type is suitable for scenarios that require a simple hierarchy but not multiple levels. The following code is an example:

[
    {
        "tagName": "tag1",
        "score": 0.8
    },
    {
        "tagName": "tag2",
        "score": 0.2
    }
]

Multi-level nested type

A multi-level nested type contains multiple nested layers of other data structures. This results in a more complex hierarchy. Use this type for data models that require rich hierarchies and a high degree of modularity or organization. The following code is an example:

[
    {
        "name": "Zhang San",
        "age": 20,
        "phone": "1390000****",
        "address": [
            {
                "province": "Zhejiang Province",
                "city": "Hangzhou City",
                "street": "No. 1201, Xingfu Community, Sunshine Avenue"
            }
        ]
    }
]

Virtual columns

You can use the virtual column feature of search indexes to query new fields and the data of new field types without the need to modify the storage schema and data in the Tablestore tables. For more information, see Virtual columns.

The virtual column feature allows you to map a column in a table to a virtual column in a search index when you create the search index. The type of the virtual column can be different from that of the column in the table. This allows you to create a column without modifying the table schema and data. The new column can be used to accelerate queries or can be configured with different tokenization methods.

  • You can configure different tokenization methods for Text fields that are mapped to the same field in a table.

    A single String column can be mapped to multiple Text columns of a search index. Different Text columns use different tokenization methods to meet various business requirements.

  • Query acceleration

    You do not need to cleanse data or re-create a table schema. You need to only map required columns of a table to the columns in a search index. The column types can be different between the table and the search index. For example, map the numeric type to the Keyword type to improve the performance of a term query, and map the String type to the numeric type to improve the performance of a range query.

Data type mappings

The value of a field in a search index is derived from the value of the corresponding field in the data table. The data types of these fields must match. The following table describes the mappings between field data types in search indexes and data tables.

Data type of the search index field

Data type of the data table field

Long

Integer

Long Array

String

Double

Double

Double Array

String

Boolean

Boolean

Boolean Array

String

Keyword

String

Keyword Array

String

Text

String

Date

Integer, String

Date Array

String

IP

String

IP Array

String

Geopoint

String

Geopoint Array

String

Vector

String

Nested

String

JSON

String