Drop-down suggestion

Overview

The drop-down suggestion feature extracts queries from your document content. It can generate candidate queries based on various matching methods for Chinese, including prefix matching, full Pinyin spelling, Pinyin initial acronyms, a mix of Chinese characters and Pinyin, tokenized prefixes, and homophones.

For example, you can search for long dress as follows:

• Chinese prefix: 连, 连衣, …

• Pinyin prefix (full spelling): l, li, lian, lianyi, lianyiqun, …

• Pinyin prefix (initials): l, ly, lyq, …

• Chinese characters plus Pinyin: 连yi, 连衣qun, …

• Prefixes after tokenization: long style, long style dress, dress long, …

• Chinese homophones and similar-sounding incorrect characters: 连衣群, 联谊群, …

You can also use manual intervention to influence suggestion results. To view key performance metrics for your drop-down suggestions, see the Drop-down suggestion report.

Data sources

Drop-down suggestions use data from your application documents and end-user queries. You can apply filters to both documents and queries.

Candidate queries from documents

You can select up to three fields from an application as the data source for each drop-down suggestion model. During processing, the system selects a sample of documents (up to one million) and processes the selected fields according to specified rules to generate candidate queries. The system then retains a subset of them as final suggestions. Two generation rules are supported: extract from original value and retain original value.

• extract from original value: This method uses a proprietary Alibaba NLP analyzer trained on vast amounts of natural language data. It applies tokenization to the field content, extracts meaningful terms, and combines them to form candidate queries. This approach ensures that the generated suggestions can retrieve corresponding documents.

• retain original value: This method uses the raw content of a field as a candidate query without tokenization. If the content exceeds 30 characters, it is truncated to the first 30 characters. This method is suitable for fields that do not require tokenization, such as store names, usernames, or song titles, or when you provide your own pre-generated candidate queries. We recommend using fields with short and clear content for this method.

Candidate queries from user queries

The system analyzes user search histories over the past N days (default is 7) and considers metrics such as search frequency, average number of results, historical term weights, and recent query success rates. Based on this analysis, it selects representative queries as candidate suggestions.

Historical search query feature: You can also enable the historical search query option to prioritize suggestions based on the current user's past queries. This requires you to pass the raw_query and user_id parameters in your requests. The historical search query feature is an extension that incurs training fees based on the compute hours that each training job consumes.

Manual intervention

You can manually influence drop-down suggestions in the following ways:

• Manage candidate queries using a blacklist and a whitelist.

• Set a filter condition on the source application documents. When you apply a filter, the system uses only documents that meet the condition to generate candidate suggestions.

Parameter

Description

Filter condition

Specify a filter condition based on fields in your OpenSearch application schema. The filter applies to all documents in the application. Note: Supported operators: <, >, <=, >=, =, != Supported field types: Numeric and string. The array type is not supported. Connectors: Separate conditions with a comma (,), which acts as the AND operator. The OR operator is not supported. Example: If you set the filter condition to status=1,level=1, only documents that match both conditions are used.

On the Create Drop-down Suggestion Model page in the console, you can configure the following settings:

• Target Application: Select the application that contains the source data for candidate suggestions.

• Model Name: Enter a name that is 1 to 30 characters long. It must start with a letter and can contain uppercase letters, lowercase letters, digits, and underscores (_). The name must be unique among all your models.

• Training Fields: Select source fields from the target application and specify a processing method for each field: extract from original value or retain original value.

• Historical Search Queries: Use the toggle to enable or disable the historical search query feature.

• Doc Filter Condition and Query Filter Condition: These are filter conditions that apply to the application document data source and the end-user query data source, respectively (for example, status=1 or biz_type=phone). For syntax rules, see the filter condition description above.

Controlling suggestion results

blacklist: The blacklist supports contains-matching. Any query that contains a blacklisted keyword is excluded from the drop-down suggestion results. If you see undesirable suggestions, add relevant keywords to the blacklist to block them.

whitelist: If a query in the whitelist meets the recommendation criteria, it is prioritized in the drop-down suggestion results. If high-quality queries do not appear or are ranked too low, add them to the whitelist to boost their visibility. **For details on how to configure blacklists and whitelists, click here.

Best practices

• To improve the effectiveness of drop-down suggestions (for example, to increase suggestion-guided searches and click-through rates), associate suggestion requests with search requests. For instructions, see the "Associate a drop-down suggestion request with a search request" section at the end of this topic.

• Choose fields with concise content that is relevant to the document's main topic.

• Use the "extract from original value" and "retain original value" rules appropriately based on your needs.

• In the response, suggestions contains the search results, while errors indicates whether an error occurred. A non-empty errors field does not necessarily mean that the suggestions field is empty. Therefore, when parsing the response, check if suggestions is empty to determine whether to display data.

Procedure

1. In the console, navigate to Search Algorithm Center > Search Guidance > Drop-down Suggestions, and click Create.

2. Enter a Model Name, select Training Fields and an extraction method, enable Historical Search Queries (optional), enter a Filter Condition (optional), and click Submit.

3. On the Drop-down Suggestion list page, find the model you created and click Train to start training.

4. After you start training, the process typically takes 20 to 30 minutes to complete.

5. After the model is trained, you can test the suggestions. The following figures show the results for the "extract from original value" and "retain original value" methods.

Retain original value:

Extract from original value:

6. To query for candidate queries online, see the demo below. For detailed API information, see the drop-down suggestion development guide.

UI reference

Drop-down Suggestion list page

In the OpenSearch console, navigate to Search Algorithm Center > Search Guidance > Drop-down Suggestions to access the list page (as shown in the figure below).

The list page displays information for each drop-down suggestion model, including Model Name, Creation Time, Status, and Latest Version Status (which can be Pending Training, Training, Trained and Passed, or Abnormal Data). The Actions column allows you to view details, train, or delete a model.

Drop-down suggestion model details page

The Basic Information section shows the model's creation time, status, last training start time, and latest version status. If the status is Abnormal Data, a link to an exception report will be displayed.

The Configuration Information section shows the configured training fields, filter conditions, blacklists/whitelists, and the status of scheduled training and historical search queries.

The Data Verification section shows the data completeness and integrity level for model training.

The Training History section shows the training records for the model.

Core metrics data

You can select different time ranges to view core metrics for the drop-down suggestion model, displayed in a table and a line chart.

Note: For definitions of specific metrics, see the Drop-down suggestion report.

SDK demo

API:

GET v3/openapi/suggestions/{suggestion_name}/actions/search?hit=10&query={your_query}&re_search=homonym&user_id=xxx

Java SDK Maven dependency:

<dependency>
    <groupId>com.aliyun.opensearch</groupId>
    <artifactId>aliyun-sdk-opensearch</artifactId>
    <version>4.0.0</version>
</dependency>

Related link: Release Notes

Code sample:

package com.example.opensearch;

import com.aliyun.opensearch.OpenSearchClient;
import com.aliyun.opensearch.SuggestionClient;
import com.aliyun.opensearch.sdk.generated.OpenSearch;
import com.aliyun.opensearch.sdk.generated.commons.OpenSearchClientException;
import com.aliyun.opensearch.sdk.generated.commons.OpenSearchException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

import java.nio.charset.Charset;

public class SuggestDemo {
    static private final String accesskey = "YOUR_ACCESSKEY_ID";
    static private final String secret = "YOUR_ACCESSKEY_SECRET";
    static private final String host = "THE_ENDPOINT_OF_THE_REGION_WHERE_YOUR_APP_IS_LOCATED";
    OpenSearch openSearch;
    OpenSearchClient openSearchClient;
    static private final byte hits = 8; // The maximum number of suggestions to return.
    static private final String suggestionName = "YOUR_SUGGESTION_MODEL_NAME"; // The name of your drop-down suggestion model.

    @Before
    public void setUp() {
        // Initialize the OpenSearch object.
        openSearch = new OpenSearch(accesskey, secret, host);
        openSearchClient = new OpenSearchClient(openSearch);
    }

    @Test
    public void TestEnv() {
        // View the file and default encoding formats.
        System.out.println(String.format("file.encoding: %s", System.getProperty("file.encoding")));
        System.out.println(String.format("defaultCharset: %s", Charset.defaultCharset().name()));

        // Create a SuggestionClient object.
        SuggestionClient suggestionClient = new SuggestionClient("YOUR_APP_NAME", suggestionName, openSearchClient);
        String query = "YOUR_SEARCH_QUERY";
try {
            SuggestParams suggestParams = new SuggestParams();
            suggestParams.setQuery(query); // Set the query.
            suggestParams.setHits(10); // Set the maximum number of suggestions to return.
            suggestParams.setUserId("12345678"); // Set the user ID.
            // The Chinese homophone matching feature is enabled by default. You can adjust this by using the re_search parameter.
            // To disable this feature, use ReSearch.findByValue(1). It is enabled by default (equivalent to ReSearch.findByValue(0) or omitting the parameter).
            suggestParams.setReSearch(ReSearch.findByValue(1));
            SearchResult result = suggestionClient.execute(suggestParams); 
            System.out.println(result); // Print the results.
        } catch (OpenSearchException e) {
            e.printStackTrace();
        } catch (OpenSearchClientException e) {
            e.printStackTrace();
        }
    }

    @After
    public void clean() {
        openSearch.clear();
    }
}

For more details about the drop-down suggestion Java SDK, see the drop-down suggestion demo.

Sample response:

{
  "request_id": "159851481919726888064081",
  "searchtime": 0.006246,
  "suggestions": [
    {
      "suggestion": "trendy skirts"
    },
    {
      "suggestion": "dresses for petite women"
    },
    {
      "suggestion": "polka dot dresses"
    },
    {
      "suggestion": "youthful skirts"
    },
    {
      "suggestion": "polka dot skirt"
    },
    {
      "suggestion": "skirts for petite women"
    },
    {
      "suggestion": "polka dot skirts for petite women"
    }
  ]
}

Note: The request_id returned in the response can be used to associate the suggestion with a subsequent search request.

Associating suggestion and search requests

Associating suggestion requests with search requests provides the following benefits:

1. It allows you to collect metrics to measure the impact of drop-down suggestions on search performance. These metrics include suggestion-guided search PVs, click-through rates, and zero/low result rates. For more details, see the Drop-down suggestion report.

2. The associated request data provides insights, such as suggestion click data, which can be used to optimize the suggestion ranking model and improve the effectiveness of suggestion-guided searches.

   Association method:

   If a user starts a search by selecting a drop-down suggestion, include the from_request_id={from_request_id} parameter in the search request. The from_request_id parameter indicates the source of the search. If the current query comes from a recommendation list, such as drop-down suggestion, top search model, or hint model, you can assign the request_id of that recommendation request to this parameter. By associating these events, you can calculate key metrics for upstream features, measure their effectiveness, and gather data for optimization. This parameter is also described in the search processing documentation.

Example:

Assume a drop-down suggestion API call returns a request_id of 159851481919726888064081. You can associate it with a search request as shown below:

SearchParams searchParams = new SearchParams(config);
searchParams.setQuery("title:'skirts for petite women'"); // Query guided by a drop-down suggestion.

// Add the from_request_id parameter.
Map<String, String> customParam =new HashMap<>();
customParam.put("from_request_id","159851481919726888064081");
searchParams.setCustomParam(customParam);

// Execute the query and return a SearchResult object.
SearchResult execute = searcherClient.execute(searchParams);
// Return the query data as a string.
String result = execute.getResult();
System.out.println(result);