Sorting and paging

Scenarios

Index presorting

By default, Tablestore sorts data in a search index based on a predefined sort order, called index presorting (IndexSort). When you query data, IndexSort determines the default order of the returned results.

When you create a search index, you can customize IndexSort. If you do not specify this setting, the index defaults to primary key sort.

Specify a sorting method when you query data

Sorting can be performed only for fields for which the enable_sort_and_agg parameter is set to true.

You can specify a sorting method for each query. A search index supports the following four types of sorters. You can also use multiple sorters to sort results based on a sequence of criteria.

ScoreSort

'sort' => array(
    array(
        'score_sort' => array(
            'order' => SortOrderConst::SORT_ORDER_DESC
        )
    ),
)

PrimaryKeySort

Sorts results by primary key.

'sort' => array(
    array(
        'pk_sort' => array(
            'order' => SortOrderConst::SORT_ORDER_ASC
        )
    ),
)

FieldSort

You can use FieldSort to sort the query results based on the values of one or more specific columns.

'sort' => array(
    array(
        'field_sort' => array(
            'field_name' => 'keyword',
            'order' => SortOrderConst::SORT_ORDER_ASC,
            'mode' => SortModeConst::SORT_MODE_AVG,
        )
    ),
)

GeoDistanceSort

Sorts results by the distance from a geographic point.

'sort' => array(
    array(
        'geo_distance_sort' => array(
            'field_name' => 'geo',
            'order' => SortOrderConst::SORT_ORDER_ASC,
            'distance_type' => GeoDistanceTypeConst::GEO_DISTANCE_PLANE,
            'points' => array('0.6,0.6')
        )
    ),
)

Combination of multiple sorting methods

You can sort query results based on the values of two columns in specific orders to determine the order in which the matched data is returned.

'sort' => array(
    array(
        'field_sort' => array(
            'field_name' => 'keyword',
            'order' => SortOrderConst::SORT_ORDER_ASC,
            'mode' => SortModeConst::SORT_MODE_AVG,
        )
    ),
    array(
        'pk_sort' => array(
            'order' => SortOrderConst::SORT_ORDER_ASC
        )
    ),
)

Specify a paging method

To page through query results, you can use limit and offset parameters or a token.

Configure the limit and offset parameters

If you do not set the limit and offset parameters, limit defaults to 10 and offset defaults to 0.

$request = array(
    'table_name' => 'php_sdk_test',
    'index_name' => 'php_sdk_test_search_index',
    'search_query' => array(
        'offset' => 10,
        'limit' => 10,
        'get_total_count' => true,
        'query' => array(
            'query_type' => QueryTypeConst::MATCH_ALL_QUERY
        ),
        'sort' => array(
            array(
                'field_sort' => array(
                    'field_name' => 'keyword',
                    'order' => SortOrderConst::SORT_ORDER_ASC
                )
            ),
        ),
        'token' => null,
    ),
    'columns_to_get' => array(
        'return_type' => ColumnReturnTypeConst::RETURN_SPECIFIED,
        'return_names' => array('col1', 'col2')
    )
);
$response = $otsClient->search($request);

Use a token

Token-based paging is recommended for deep paging because it has no depth limit.

If Tablestore cannot read full data that meets the query conditions, Tablestore returns next_token. You can use next_token to continue reading the subsequent data.

By default, token-based paging only allows you to move forward through the results. However, because a token remains valid throughout the query process, you can cache a previous token to page backward.

When you page with a token, the sort order is the same as in the previous request, whether it uses the default IndexSort or a custom sort. Therefore, you cannot set a Sort parameter when a token is used. You also cannot set an offset, as you can only read data sequentially.

$request = array(
    'table_name' => 'php_sdk_test',
    'index_name' => 'php_sdk_test_search_index',
    'search_query' => array(
        'offset' => 0,
        'limit' => 10,
        'get_total_count' => true,
        'query' => array(
            'query_type' => QueryTypeConst::FUNCTION_SCORE_QUERY,
            'query' => array(
                'query' => array(
                    'query_type' => QueryTypeConst::TERM_QUERY,
                    'query' => array(
                        'field_name' => 'keyword',
                        'term' => 'keyword'
                    )
                ),
                'field_value_factor' => array(
                    'field_name' => 'long'
                )
            )
        ),
        'sort' => array(
            array(
                'score_sort' => array(
                    'order' => SortOrderConst::SORT_ORDER_DESC
                )
            ),
        )
    ),
    'columns_to_get' => array(
        'return_type' => ColumnReturnTypeConst::RETURN_SPECIFIED,
        'return_names' => array('keyword', 'long')
    )
);

$response = $otsClient->search($request);
print "total_hits: " . $response['total_hits'] . "\n";
print json_encode($response['rows'], JSON_PRETTY_PRINT);

while($response['next_token'] != null) {
    $request['search_query']['token'] = $response['next_token'];
    $request['search_query']['sort'] = null;// If next_token is included in the response, you cannot specify the sorting method because the token includes the sorting information. 
    $response = $otsClient->search($request);
    print json_encode($response['rows'], JSON_PRETTY_PRINT);
}

FAQ

• What do I do if no data can be found by calling the Search operation?

• How do I increase the value of the limit parameter to 1000 when I call the Search operation of the search index feature to query data?

• Why does a token become invalid when pagination is used in a search index-based query?

References

• When you use a search index to query data, you can use the following query methods: term query, terms query, match all query, match query, match phrase query, prefix query, range query, wildcard query, Boolean query, geo query, nested query, and exists query. You can use different query methods to query data from multiple dimensions based on your business requirements.

  If you want to sort or paginate the rows that meet the query conditions, you can use the sorting and paging feature. For more information, see Sorting and paging.

  If you want to collapse the result set based on a specific column, you can use the collapse (distinct) feature. This way, data of the specified type appears only once in the query results. For more information, see Collapse (distinct).

• If you want to analyze data in a data table, such as obtaining the extreme values, sum, and total number of rows, you can perform aggregation operations or execute SQL statements. For more information, see Aggregation and SQL query.

• If you want to quickly obtain all rows that meet the query conditions without the need to sort the rows, you can call the ParallelScan and ComputeSplits operations to use the parallel scan feature. For more information, see Parallel scan.