The distinct clause diversifies search results by spreading documents that share a common field value across the result set, preventing any single entity from dominating a page of results.
Example: In a job-listing search, the top-ranked results might all belong to one company. Add a distinct clause on company_id to ensure results include listings from multiple companies.
distinct=dist_key:company_id,dist_count:2,dist_times:10This example performs 10 rounds of extraction on company_id, selecting 2 documents per round. Documents not extracted in any round are ranked lower.
When reserved is set to false, the total_hit response value may be inaccurate because discarded documents are excluded from the count. Use update_total_hit or the distinct uniq plug-in to correct this.
Syntax
distinct=dist_key:field,dist_count:1,dist_times:1,reserved:falseParameters
| Parameter | Required | Default | Description |
|---|---|---|---|
dist_key | Yes | — | The attribute field to diversify on. Only INT and LITERAL types are supported; ARRAY type is not. Only one field can be specified. |
dist_count | No | 1 | Number of documents to extract per round. |
dist_times | No | 1 | Number of extraction rounds to perform. |
reserved | No | true | Specifies whether to retain documents not selected in any round. Set to false to discard them. When `false`, `total_hit` may be inaccurate. |
update_total_hit | No | false | Takes effect only when reserved is false. When true, the system calculates the difference between the number of discarded documents and the value of total_hit. The value of total_hit may be inaccurate. When false, total_hit includes discarded documents. |
dist_filter | No | — | A filter expression. Documents matching this filter are excluded from diversification but still participate in fine sorting alongside the extracted documents. By default, all retrieved documents are candidates for diversification. |
grade | No | — | Threshold values that classify documents into score bands. Diversification is applied within each band independently. Separate thresholds with |. The band order follows the rough-sorting order (descending or ascending). |
How dist_count and dist_times interact
The following example shows how dist_count and dist_times affect results with 6 documents across 3 key values:
doc 1: id:1 name:a
doc 2: id:2 name:a
doc 3: id:3 name:a
doc 4: id:4 name:b
doc 5: id:5 name:c
doc 6: id:6 name:cCase 1 — dist_count:2, dist_times:1:
distinct=dist_key:name,dist_count:2,dist_times:1 returns: doc 1, doc 2, doc 4, doc 5, doc 6
Case 2 — dist_count:1, dist_times:2:
distinct=dist_key:name,dist_count:1,dist_times:2 returns: doc 1, doc 4, doc 5, doc 2, doc 6
Case 2 interleaves documents across keys more evenly than Case 1 because it alternates between keys on each round rather than filling the quota from one key first.
Case 3 — dist_count:1, dist_times:1:
distinct=dist_key:name,dist_count:1,dist_times:1 returns: doc 1, doc 4, doc 5
grade parameter
Thresholds classify documents into score bands before diversification is applied. Documents in each band are diversified independently.
Single threshold — grade:3.0:
Band 1: score < 3.0
Band 2: score >= 3.0
Multiple thresholds — grade:3.0|5.0:
Band 1: score < 3.0
Band 2: 3.0 <= score < 5.0
Band 3: score >= 5.0
Band order matches the rough-sorting order. If rough sorting is descending, bands are processed from highest to lowest. If rough sorting is ascending, bands are processed from lowest to highest.
Disperse documents in multiple phases
The Searcher node in OpenSearch Retrieval Engine Edition runs two sorting phases: rough sorting and fine sorting. A distinct clause can contain two sub-distinct clauses separated by a semicolon — the first applies during rough sorting, the second applies after fine sorting completes.
distinct = sub_distinct_clause_when_sort;sub_distinct_clause_after_sortFour valid formats:
| Format | Behavior | Example |
|---|---|---|
sub_dist_clause;none_dist | Disperse during rough sorting only. | "distinct=dist_key:company_id,dist_count:1,dist_times:1;none_dist" |
sub_dist_clause1;sub_dist_clause2 | Disperse in both phases. Use a larger dist_count in the first clause to ensure enough documents reach fine sorting. | "distinct=dist_key:company_id,dist_count:2,dist_times:1;dist_key:company_id,dist_count:1,dist_times:1" |
none_dist;sub_dist_clause | Skip rough-sorting dispersal; disperse after fine sorting only. | "distinct=none_dist;dist_key:company_id,dist_count:1,dist_times:1" |
sub_dist_clause (single clause) | Apply the same dispersal in both phases. | "distinct=dist_key:company_id,dist_count:1,dist_times:1" |
distinct=none_dist;none_dist is invalid. To disable dispersal entirely, omit the distinct clause from the query.If after-sorting dispersal is enabled and start + hit exceeds the rank_size of scorers, pagination may become unstable.
distinct uniq plug-in
When reserved is false, the total and viewtotal response values can be inaccurate, which causes pagination errors. The distinct uniq plug-in corrects these values.
Conditions: The plug-in works only when all three of the following are true:
dist_timesis1dist_countis1reservedisfalse
Performance limit: The plug-in returns up to 5,000 results per query. If the result set exceeds 5,000, only 5,000 are returned.
Usage: Add duniqfield:field to a kvpairs clause. The duniqfield value must match the dist_key value in the distinct clause.
distinct=dist_key:company_id,dist_count:1,dist_times:1,reserved:false&&kvpairs=duniqfield:company_idFor kvpairs clause syntax, see kvpairs clause.
Usage notes
The distinct clause is optional.
dist_keymust reference an attribute field defined inschema.json.Only INT and LITERAL field types are supported. ARRAY type is not supported.
Only one field can be specified as
dist_key.