Sorting and paging
When querying data with a search index, you can apply a predefined sort order or specify one at query time. To navigate large result sets, use limit/offset for offset-based paging, or use tokens for cursor-based paging.
Scenarios
Category | Method | Feature | Scenario |
Sorting | Predefine a sorting method when you create a search index | By default, results are returned in the order defined by the IndexSort parameter set during index creation. | |
Specify a sorting method when you query data | Sorting based on the BM25-based keyword relevance score (ScoreSort) | Sort results by BM25 relevance score. Best suited for full-text search. | |
Sort results by primary key value. Useful when you need results ordered by unique row identifiers. | |||
Sorting based on the values of one or more columns (FieldSort) | Sort results by one or more column values, such as sales volume or page views. Commonly used in e-commerce, social networking, and media asset management. | ||
Sort results by distance from a central point. Commonly used in mapping and logistics—for example, sorting nearby restaurants by distance. | |||
Paging | Specify a paging method when you query data | Jump to any page when the result set contains fewer than 100,000 rows. | |
Page through results sequentially using a cursor; you can only page forward. Tokens remain valid for the duration of the query, so you can cache a previous token to return to an earlier page. |
Index presorting
By default, Tablestore sorts data in a search index based on a predefined sort order, called index presorting (IndexSort). When you query data, IndexSort determines the default order of the returned results.
When you create a search index, you can customize IndexSort. If you do not specify this setting, the index defaults to primary key sort.
Index presorting supports only
PrimaryKeySort(primary key sort) andFieldSort(field sort).You cannot use index presorting for a search index that contains nested type fields.
After you create a search index, the dynamic schema modification feature enables you to modify the
IndexSortsettings.
Specify a sorting method
Sorting requires enableSortAndAgg to be set to true on the fields you want to sort by.
You can specify a sorting method for each query. A search index supports the following four types of sorters. You can also use multiple sorters to sort results based on a sequence of criteria.
ScoreSort
Sorts query results by relevance score, which is calculated using the BM25 algorithm. This method is suitable for relevance-based scenarios, such as full-text search.
If you want to sort by relevance score, you must explicitly specify
ScoreSort. Otherwise, Tablestore sorts the results according to theIndexSortsettings of the index.When you use
ScoreSort, fields of the FuzzyKeyword type are not included in the sorting, and theweightparameter has no effect on these fields.
Use ScoreSort to rank results by BM25 relevance score, in ascending or descending order.
sort: {
sorters: [
{
scoreSort: {
order: TableStore.SortOrder.SORT_ORDER_ASC
}
}
]
}PrimaryKeySort
Sorts results by primary key.
Use PrimaryKeySort to order results by primary key value.
sort: {
sorters: [
{
primaryKeySort: {
order: TableStore.SortOrder.SORT_ORDER_DESC // Sort in descending order.
//order: TableStore.SortOrder.SORT_ORDER_ASC // Sort in ascending order.
}
}
]
}FieldSort
Use FieldSort to sort results by one or more column values.
Sort by a single column
sort: {
sorters: [
{
fieldSort: {
fieldName: "Col_Keyword",
order: TableStore.SortOrder.SORT_ORDER_DESC
}
}
]
}Sort by multiple columns
Specify multiple sorters to sort by a primary column first, then break ties using a secondary column.
sort: {
sorters: [
{
fieldSort: {
fieldName: "Col_Keyword",
order: TableStore.SortOrder.SORT_ORDER_DESC
}
},
{
fieldSort: {
fieldName: "Col_Long",
order: TableStore.SortOrder.SORT_ORDER_DESC
}
}
]
}GeoDistanceSort
Sorts results by the distance from a geographic point.
Use GeoDistanceSort to rank results by distance from a central geographic point.
sort: {
sorters: [
{
geoDistanceSort: {
fieldName: "Col_Geo_Point",
points: ["0,0"],// Specify the coordinate pair of the central point.
order: TableStore.SortOrder.SORT_ORDER_ASC // Return results in ascending order of distance.
}
}
]
}For a complete example, see Search on GitHub.
Specify a paging method
To page through query results, you can use limit and offset parameters or a token.
Configure the limit and offset parameters
Offset-based paging
You can use offset-based paging when the total number of rows to retrieve is less than 100,000. The sum of limit and offset must be less than or equal to 100,000, and the maximum value for limit is 100.
To increase the limit threshold, see How do I increase the limit for the Search API to 1,000?.
If you do not set the limit and offset parameters, limit defaults to 10 and offset defaults to 0.
Use offset and limit to jump directly to any page. This method supports result sets of up to 100,000 rows.
/**
* Set offset to 90 and limit to 10 to retrieve rows 90–99.
*/
client.search({
tableName: TABLE_NAME,
indexName: INDEX_NAME,
searchQuery: {
offset: 90,
limit: 10,
query: {
queryType: TableStore.QueryType.MATCH_ALL_QUERY
},
getTotalCount: true // Return the total number of matching rows. Default: false.
},
columnToGet: {
// RETURN_ALL: return all columns.
// RETURN_SPECIFIED: return specified columns.
// RETURN_NONE: return primary key columns only.
returnType: TableStore.ColumnReturnType.RETURN_ALL
}
}, function (err, data) {
if (err) {
console.log('error:', err);
return;
}
console.log('success:', JSON.stringify(data, null, 2));
});Use a token
Token-based paging is recommended for deep paging because it has no depth limit.
If a response does not contain all matching data, the server returns a nextToken.
By default, token-based paging only allows you to move forward through the results. However, because a token remains valid throughout the query process, you can cache a previous token to page backward.
Token-based paging returns results page by page using a cursor (NextToken). Each response includes a token for the next page. Tokens are valid for the duration of the query, so cache a previous token to navigate back to an earlier page.
To persist or pass NextToken to a frontend page, encode it as a Base64 string. Tokens are binary data (byte stream), not strings—using string(NextToken) loses data. Convert using Buffer:
Encode:
data.nextToken.toString("base64")Decode:
Buffer.from(base64String, "base64")
When you page with a token, the sort order is the same as in the previous request, whether it uses the default IndexSort or a custom sort. Therefore, you cannot set a Sort parameter when a token is used. You also cannot set an offset, as you can only read data sequentially.
A search index that contains a nested type field does not support index presorting. If you need to page through results from such an index, you must specify a sort order in the query. Otherwise, the server does not return a nextToken even if more data is available.
The following examples show token-based paging in both synchronous and asynchronous modes. Both use the same initial params object.
var params = {
tableName: TABLE_NAME,
indexName: INDEX_NAME,
searchQuery: {
offset: 0,
limit: 10,
token: null, // Set to nextToken from the previous response to fetch the next page.
query: {
queryType: TableStore.QueryType.MATCH_ALL_QUERY
},
getTotalCount: true
},
columnToGet: {
returnType: TableStore.ColumnReturnType.RETURN_SPECIFIED,
returnNames: ["pic_tag", "pic_description", "time_stemp", "pos"]
}
};
/**
* Synchronous mode: await each page before fetching the next.
*/
(async () => {
try {
var data = await client.search(params);
console.log('success:', JSON.stringify(data, null, 2));
while (data.nextToken && data.nextToken.length) {
// Encode the binary token as Base64 for storage or transfer,
// then decode it back to binary before passing it as the next token.
var nextToken = data.nextToken.toString("base64");
var token = Buffer.from(nextToken, "base64");
params.searchQuery.token = token;
data = await client.search(params);
console.log('token success:', JSON.stringify(data, null, 2));
}
} catch (error) {
console.log(error);
}
})()
/**
* Asynchronous mode: use callbacks to fetch the next page.
*/
client.search(params, function (err, data) {
console.log('success:', JSON.stringify(data, null, 2));
if (data.nextToken && data.nextToken.length) {
// Encode and decode the token the same way as in synchronous mode.
var nextToken = data.nextToken.toString("base64");
var token = Buffer.from(nextToken, "base64");
params.searchQuery.token = token;
client.search(params, function (err, data) {
console.log('token success:', JSON.stringify(data, null, 2));
});
}
});