Highlighting search queries-Tablestore(Tablestore)-阿里云帮助中心

To use the summary and highlighting feature, you must enable highlighting for a Text field when you create a search index. Then, you can set highlight parameters in your query to return snippets that contain the matched search queries and highlight the matched terms.

Scenarios

You can use the summary and highlighting feature to extract snippets near matched search queries and highlight the queries. This feature is useful in full-text search scenarios, such as web searches, chat record retrieval, and document searches.

Function overview

The summary and highlighting feature highlights text in search results that matches or is relevant to your search queries. This helps users quickly find relevant content and improves information retrieval efficiency. This feature is also supported for complex data structures that contain nested types, such as JSON, to precisely locate information. By default, Tablestore uses the tags to mark matched search queries.

To use the summary and highlighting feature, you must perform the following configurations:

When you create a search index, set the enableHighlighting parameter to True for the Text field. For more information, see Create a search index.
Important
You can enable the summary and highlighting feature only for Text fields.
When you query data, you can customize the highlight style by specifying parameters, such as the encoding method for highlighted fragments, the maximum number of fragments to return, and the pre-tags and post-tags.

Assume that you have enabled the summary and highlighting feature for a Text field when you created a search index. If the value of the Text field is query highlight test, you can use a match phrase query for the keyword highlight. After you specify the highlight parameters, the highlighted fragment query highlight test is returned.

Notes

When you use the summary and highlighting feature with a MatchQuery or MatchPhraseQuery, the matched terms may be highlighted by multiple pre-tags and post-tags.
If the tokenizer type for the Text field is maximum semantic tokenization, the summary and highlighting feature is not supported when you use a MatchPhraseQuery to query data.
Fragment splitting may divide a matched term in the text. In this case, the term may not be highlighted.

API

The summary and highlighting feature uses the Search API. The supported query types are TermQuery, TermsQuery, MatchQuery, MatchPhraseQuery, PrefixQuery, WildcardQuery, and NestedQuery.

Parameters

When you use the summary and highlighting feature, you can set highlight parameters using the Highlight parameter. For sub-columns of a nested type, you can set the highlight parameters using the InnerHits parameter.

Highlight parameters

Parameter		Description
highlightEncoder		The encoding method for the original content of the highlighted fragment. Valid values: PLAIN (default): The original content is displayed without encoding. HTML: The original content of the highlighted fragment is HTML-escaped. For example, `<` is escaped to `<`, `>` to `>`, `"` to `"`, `'` to `'`, and `/` to `/`. Use the HTML format for web display.
fieldHighlightParams		The highlighting parameters for a field. You can set this parameter only for fields that are part of a keyword query in SearchQuery.
HighlightParameter	numberOfFragments	The maximum number of highlighted fragments to return. Set this parameter to 1.
	fragmentSize	The length of each fragment. The default value is 100. Important The actual length of the returned fragment may not be exactly equal to this value.
	preTag	The prefix tag for highlighting the search query, such as `<em>` or `<b>`. The default value is `<em>`. You can customize the prefix tag as needed. The supported characters for preTag include `< > " ' /`, `a-z`, `A-Z`, and `0-9`.
	postTag	The suffix tag for highlighting the search query, such as `</em>` or `</b>`. The default value is `</em>`. You can customize the suffix tag as needed. The supported characters for postTag include `< > " ' /`, `a-z`, `A-Z`, and `0-9`.
	highlightFragmentOrder	The sorting rule for fragments when a highlighted field returns multiple fragments. TEXT_SEQUENCE (default): The order in which the fragments appear in the text. SCORE: The fragments are sorted based on the scores of the search query hits.

InnerHits parameters

Parameter	Description
sort	The sorting rule for the returned nested sub-rows.
offset	The starting position from which to return sub-rows when a nested column contains multiple sub-rows.
limit	The number of sub-rows to return when a nested column contains multiple sub-rows. Default value: 3.
highlight	The highlight parameter configuration for nested sub-columns. For more information about the configuration, see Highlight parameters.

Usage

Important

You can use the summary and highlighting feature only through an SDK.

Before you can perform a highlight query, you must complete the following preparations:

You must have an Alibaba Cloud account or a RAM user that has the permissions to perform operations on Tablestore. To grant a RAM user the permissions to perform operations on Tablestore, see Grant permissions to a RAM user using a RAM policy.
When you use an SDK, if you do not have a valid AccessKey, you must create one for your Alibaba Cloud account or RAM user. For more information, see Create an AccessKey.
A data table has been created. For more information, see Data table operations.
Create a search index for the data table and enable highlighting for the specified field. For more information, see Create a search index.
When you use an SDK, you must initialize the Tablestore client. For more information, see Initialize Tablestore Client.

You can use the summary and highlighting feature with the Java SDK, Go SDK, Python SDK, and Node.js SDK. The following examples show how to use this feature with the Java SDK.

Using query summary and highlighting for non-nested fields

The following example shows how to use MatchQuery to find data where the Col_Text column matches hangzhou shanghai and how to highlight the search query in the results. The Col_Text column is a Text type.

/**
 * Use MatchQuery with summary and highlighting.
 */
public static void matchQueryWithHighlighting(SyncClient client) {
    SearchRequest searchRequest = SearchRequest.newBuilder()
            .tableName("<TABLE_NAME>")
            .indexName("<SEARCH_INDEX_NAME>")
            .returnAllColumnsFromIndex(true)
            .searchQuery(SearchQuery.newBuilder()
                    .limit(5)
                    .query(QueryBuilders.bool()
                            .should(QueryBuilders.match("Col_Text", "hangzhou shanghai")))
                    .highlight(Highlight.newBuilder()
                            .addFieldHighlightParam("Col_Text", HighlightParameter.newBuilder()
                                    .highlightFragmentOrder(HighlightFragmentOrder.TEXT_SEQUENCE)
                                    .preTag("")
                                    .postTag("")
                                    .build())
                            .build())
                    .build())
            .build();
    SearchResponse resp = client.search(searchRequest);

    // Print the query and highlighting results. Set the prefix to an empty string when you query non-nested fields.
    printSearchHit(resp.getSearchHits(), "");
}

/**
 * Prints the content of searchHit.
 * @param searchHits The search hits.
 * @param prefix The prefix to add for nested structures to display hierarchical information.
 */
private static void printSearchHit(List<SearchHit> searchHits, String prefix) {
    for (SearchHit searchHit : searchHits) {
        if (searchHit.getScore() != null) {
            System.out.printf("%s Score: %s\n", prefix, searchHit.getScore());
        }

        if (searchHit.getOffset() != null) {
            System.out.printf("%s Offset: %s\n", prefix, searchHit.getOffset());
        }

        if (searchHit.getRow() != null) {
            System.out.printf("%s Row: %s\n", prefix, searchHit.getRow().toString());
        }

        // Print the highlighted fragment results for each field.
        if (searchHit.getHighlightResultItem() != null) {
            System.out.printf("%s Highlight: \n", prefix);
            StringBuilder strBuilder = new StringBuilder();
            for (Map.Entry<String, HighlightField> entry : searchHit.getHighlightResultItem().getHighlightFields().entrySet()) {
                strBuilder.append(entry.getKey()).append(":").append("[");
                strBuilder.append(StringUtils.join(",", entry.getValue().getFragments())).append("]\n");
            }
            System.out.printf("%s   %s", prefix, strBuilder);
        }

        System.out.println();
    }
}

Use highlighting when you query nested fields

The following example shows how to use `NestedQuery` to query for data where the value of the `Level1_Col1_Nested` sub-field in the `Col_Nested` nested field matches `hangzhou shanghai. The search query is highlighted in the returned results.

/**
 * Use summary and highlighting in a NestedQuery. Set parameters using innerHits.
 */
public static void nestedQueryWithHighlighting(SyncClient client) {
        SearchRequest searchRequest = SearchRequest.newBuilder()
                .tableName("<TABLE_NAME>")
                .indexName("<SEARCH_INDEX_NAME>")
                .returnAllColumnsFromIndex(true)
                .searchQuery(SearchQuery.newBuilder()
                        .limit(5)
                        .query(QueryBuilders.nested()
                                .path("Col_Nested")
                                .scoreMode(ScoreMode.Min)
                                .query(QueryBuilders.match("Col_Nested.Level1_Col1_Nested", "hangzhou shanghai"))
                                .innerHits(InnerHits.newBuilder()
                                        .highlight(Highlight.newBuilder()
                                                .addFieldHighlightParam("Col_Nested.Level1_Col1_Nested", HighlightParameter.newBuilder().build())
                                                .build())
                                        .build()))
                        .build())
                .build();
        SearchResponse resp = client.search(searchRequest);

        // Print the highlighted results.
        printSearchHit(resp.getSearchHits(), "");
}

/**
 * Print the content of searchHit.
 * @param searchHits The search hits.
 * @param prefix The prefix to add when printing nested structures to show hierarchical information.
 */
private static void printSearchHit(List<SearchHit> searchHits, String prefix) {
    for (SearchHit searchHit : searchHits) {
        if (searchHit.getScore() != null) {
            System.out.printf("%s Score: %s\n", prefix, searchHit.getScore());
        }

        if (searchHit.getOffset() != null) {
            System.out.printf("%s Offset: %s\n", prefix, searchHit.getOffset());
        }

        if (searchHit.getRow() != null) {
            System.out.printf("%s Row: %s\n", prefix, searchHit.getRow().toString());
        }

        // Print the highlighted fragments for each field.
        if (searchHit.getHighlightResultItem() != null) {
            System.out.printf("%s Highlight: \n", prefix);
            StringBuilder strBuilder = new StringBuilder();
            for (Map.Entry<String, HighlightField> entry : searchHit.getHighlightResultItem().getHighlightFields().entrySet()) {
                strBuilder.append(entry.getKey()).append(":").append("[");
                strBuilder.append(StringUtils.join(",", entry.getValue().getFragments())).append("]\n");
            }
            System.out.printf("%s   %s", prefix, strBuilder);
        }

        // Highlighted results for the nested type.
        for (SearchInnerHit searchInnerHit : searchHit.getSearchInnerHits().values()) {
            System.out.printf("%s Path: %s\n", prefix, searchInnerHit.getPath());
            System.out.printf("%s InnerHit: \n", prefix);
            printSearchHit(searchInnerHit.getSubSearchHits(), prefix + "    ");
        }

        System.out.println();
    }
}

Assume that the multi-level nested field `Col_Nested` includes two sub-fields: `Level1_Col1_Text` (Text) and `Level1_Col2_Nested` (Nested). The `Level1_Col2_Nested` nested field includes the `Level2_Col1_Text` sub-field.

The following example shows how to add a `BoolQuery` to a `NestedQuery` to use the summary and highlighting features on both the `Level1_Col1_Text` sub-field in the `Col_Nested` field and the `Level2_Col1_Text` sub-field under `Level1_Col2_Nested`.

public static void nestedQueryWithHighlighting(SyncClient client) {
    SearchRequest searchRequest = SearchRequest.newBuilder()
            .tableName("<TABLE_NAME>")
            .indexName("<SEARCH_INDEX_NAME>")
            .returnAllColumnsFromIndex(true)
            .searchQuery(SearchQuery.newBuilder()
                    .limit(5)
                    .query(QueryBuilders.nested()
                            .path("Col_Nested")
                            .scoreMode(ScoreMode.Min)
                            .query(QueryBuilders.bool()
                                    .should(QueryBuilders.match("Col_Nested.Level1_Col1_Text", "hangzhou shanghai"))
                                    .should(QueryBuilders.nested()
                                            .path("Col_Nested.Level1_Col2_Nested")
                                            .scoreMode(ScoreMode.Min)
                                            .query(QueryBuilders.match("Col_Nested.Level1_Col2_Nested.Level2_Col1_Text", "hangzhou shanghai"))
                                            .innerHits(InnerHits.newBuilder()
                                                    .highlight(Highlight.newBuilder()
                                                            .addFieldHighlightParam("Col_Nested.Level1_Col2_Nested.Level2_Col1_Text", HighlightParameter.newBuilder().build())
                                                            .build())
                                                    .build())))
                            .innerHits(InnerHits.newBuilder()
                                    .sort(new Sort(Arrays.asList(
                                            new ScoreSort(),
                                            new DocSort()
                                    )))
                                    .highlight(Highlight.newBuilder()
                                            .addFieldHighlightParam("Col_Nested.Level1_Col1_Text", HighlightParameter.newBuilder().build())
                                            .build())
                                    .build()))
                    .build())
            .build();
    SearchResponse resp = client.search(searchRequest);
    // Print the highlighted results.
    printSearchHit(resp.getSearchHits(), "");
}

/**
 * Print the content of searchHit.
 * @param searchHits The search hits.
 * @param prefix The prefix to add when printing nested structures to show hierarchical information.
 */
private static void printSearchHit(List<SearchHit> searchHits, String prefix) {
    for (SearchHit searchHit : searchHits) {
        if (searchHit.getScore() != null) {
            System.out.printf("%s Score: %s\n", prefix, searchHit.getScore());
        }

        if (searchHit.getOffset() != null) {
            System.out.printf("%s Offset: %s\n", prefix, searchHit.getOffset());
        }

        if (searchHit.getRow() != null) {
            System.out.printf("%s Row: %s\n", prefix, searchHit.getRow().toString());
        }

        // Print the highlighted fragments for each field.
        if (searchHit.getHighlightResultItem() != null) {
            System.out.printf("%s Highlight: \n", prefix);
            StringBuilder strBuilder = new StringBuilder();
            for (Map.Entry<String, HighlightField> entry : searchHit.getHighlightResultItem().getHighlightFields().entrySet()) {
                strBuilder.append(entry.getKey()).append(":").append("[");
                strBuilder.append(StringUtils.join(",", entry.getValue().getFragments())).append("]\n");
            }
            System.out.printf("%s   %s", prefix, strBuilder);
        }

        // Highlighted results for the nested type.
        for (SearchInnerHit searchInnerHit : searchHit.getSearchInnerHits().values()) {
            System.out.printf("%s Path: %s\n", prefix, searchInnerHit.getPath());
            System.out.printf("%s InnerHit: \n", prefix);
            printSearchHit(searchInnerHit.getSubSearchHits(), prefix + "    ");
        }

        System.out.println();
    }
}

Billing

Using the summary and highlighting feature during data queries does not affect the existing billing rules.

In VCU mode (formerly reserved mode), querying data using a search index consumes the compute resources of the VCU. In CU mode (formerly pay-as-you-go mode), querying data using a search index consumes read throughput. For more information, see Search index metering and billing.

FAQ

References

When you use a search index to query data, you can use the following query methods: term query, terms query, match all query, match query, match phrase query, prefix query, range query, wildcard query, fuzzy query, Boolean query, geo query, nested query, KNN vector query, and exists query. You can select query methods based on your business requirements to query data from multiple dimensions.
You can sort or paginate rows that meet the query conditions by using the sorting and paging features. For more information, see Perform sorting and paging.
You can use the collapse (distinct) feature to collapse the result set based on a specific column. This way, data of the specified type appears only once in the query results. For more information, see Collapse (remove duplicates).
If you want to analyze data in a data table, you can use the aggregation feature of the Search operation or execute SQL statements. For example, you can obtain the minimum and maximum values, sum, and total number of rows. For more information, see Aggregation and SQL query.
If you want to obtain all rows that meet the query conditions without the need to sort the rows, you can call the ParallelScan and ComputeSplits operations to use the parallel scan feature. For more information, see Parallel scan.