Collapse (distinct)

Updated at:

The collapse feature deduplicates a query result set by a specified column so that each unique value appears only once in the returned results. Use collapse to keep query results diverse when a single category dominates the result set.

Prerequisites

Usage limits

Before you use the collapse feature, note the following constraints:

  • Supported column types — The collapse feature applies only to columns of the integer, floating-point number, and Keyword types. Columns of the array type are not supported.

  • Pagination — The collapse feature supports pagination only by using offset and limit. Token-based pagination is not supported.

  • Aggregation — When you use statistical aggregation together with the collapse feature, the aggregation applies only to the result set before the collapse operation.

  • Group count limit — After the collapse operation, the total number of groups returned depends on the maximum value of offset plus limit. A maximum of 100,000 groups can be returned.

  • Total count — The total number of rows in the response is the number of matched rows before the collapse operation. You cannot obtain the total number of groups after the collapse operation.

Parameters

Parameter

Description

query

The query type. You can set this parameter to any query type.

collapse

Collapses the result set based on the field that is specified by the fieldName parameter.

fieldName: the name of the field based on which the result set is collapsed. Only the fields whose values are of the INTEGER, FLOATING-POINT, or KEYWORD type are supported. The fields whose values are of the ARRAY type are not supported.

offset

The position from which the current query starts.

limit

The maximum number of rows that you want the current query to return.

To query only the number of rows that meet the query conditions without querying specific data of the rows, set the limit parameter to 0.

getTotalCount

Specifies whether to return the total number of rows that meet the query conditions. The default value of this parameter is false, which specifies that the total number of rows that meet the query conditions is not returned.

If you set this parameter to true, the query performance is compromised.

tableName

The name of the data table.

indexName

The name of the search index.

columnsToGet

Specifies whether to return all columns of each row that meets the query conditions.

  • If you set returnType to TableStore.ColumnReturnType.RETURN_SPECIFIED, you need to configure returnNames to specify the columns that you want to return.

  • If you set the returnType parameter to TableStore.ColumnReturnType.RETURN_ALL, all columns are returned.

  • If you set the returnType parameter to TableStore.ColumnReturnType.RETURN_ALL_FROM_INDEX, all columns in the search index are returned. .

  • If you set the returnType parameter to TableStore.ColumnReturnType.RETURN_NONE, only the primary key columns are returned.

Examples

The following sample code provides an example on how to use match all query to query the total number of rows in a table and collapse the result set based on the value of the col_keyword column:

let searchQuery = {
    offset: 0,
    limit: 100,
    query: {
        queryType: TableStore.QueryType.MATCH_ALL_QUERY,
    },
    collapse: {
        fieldName: "col_keyword",
    },
    getTotalCount: false,
};
let params = {
    tableName: tableName,
    indexName: indexName,
    searchQuery: searchQuery,
    columnToGet: { // Specify the columns that you want to return. You can set the parameter to RETURN_SPECIFIED to return the specified columns, RETURN_ALL to return all columns, RETURN_ALL_FROM_INDEX to return all columns in the search index, or RETURN_NONE to return only the primary key columns. 
        returnType: TableStore.ColumnReturnType.RETURN_ALL_FROM_INDEX
    },
    timeoutMs: 30000,
}
client.search(params, function (err, data) {
    if (err) {
        console.log('search error:', err.toString());
    } else {
        console.log('search success:', data);
    }
});

FAQ

References

  • The following query types are supported by search indexes: term query, terms query, match all query, match query, match phrase query, prefix query, range query, wildcard query, Boolean query, geo query, nested query, vector query, and exists query. You can select a query type to query data based on your business requirements.

    If you want to sort or paginate the rows that meet the query conditions, you can use the sorting and paging feature. For more information, see Sorting and paging.

    If you want to collapse the result set based on a specific column, you can use the collapse (distinct) feature. This way, data of the specified type appears only once in the query results. For more information, see Collapse (distinct).

  • If you want to analyze data in a data table, such as obtaining the extreme values, sum, and total number of rows, you can perform aggregation operations or execute SQL statements. For more information, see Aggregation and SQL query.

  • If you want to quickly obtain all rows that meet the query conditions without the need to sort the rows, you can call the ParallelScan and ComputeSplits operations to use the parallel scan feature. For more information, see Parallel scan.