What are the document sorting cases in OpenSearch-OpenSearch(Open Search)-阿里云帮助中心

How sorting works

OpenSearch sorts documents in two sequential stages:

Stage	Internal name	Role	Score formula
Rough sort	FirstRank	Scores all retrieved documents and selects the top N	10,000 + rough sort expression result (max displayed: 20,000)
Fine sort	SecondRank	Re-scores the top N documents using a more precise expression	10,000 + fine sort expression result (no upper limit)

The final sort policy score (RANK) is the score from whichever stage a document reaches last:

Documents that pass through both stages: RANK = 10,000 + fine sort result
Documents eliminated after rough sort: RANK = 10,000 + rough sort result (max displayed: 20,000)

When a document transitions from rough sort to fine sort, the rough sort score is discarded and 10,000 points are added as the base for the fine sort score.

Example trace

Given sort clause sort=age;-RANK, a sort score of 13,10000.2259030193 means:

13 — value of the age field (ascending sort)
10000.2259030193 — RANK (final sort policy score)

The scoring detail for this document:

FirstRank:
expression[static_bm25()], result[0.496452].
SecondRank:
expression[text_relevance(name)], result[0.225903].

This document ranked in the top 200 of 1,000,000 retrieved documents based on rough sort, so it entered fine sort. The final RANK is 10,000 + 0.225903... = 10000.2259030193. The rough sort score (0.496452) is discarded.

Sort clause and sort policies

Sort clause

The sort clause controls the global sort order of documents, similar to the ORDER BY clause in SQL. Use it to sort by one or more fields and by the sort policy score (RANK).

Format: sort=<field1>;<field2>;...

Prefix with - for descending order, + for ascending order
RANK is a reserved keyword representing the final sort policy score

Default behavior

Sort clause configured?	Effective sorting
No	`-RANK` (sort by policy score, descending)
Yes, includes `-RANK`	Fields in the order specified, with policy score as one criterion
Yes, omits `RANK`	Fields only; policy score is not applied

Important

If you configure a sort clause, include -RANK explicitly to apply sort policy scoring. The system does not add it automatically.

Example: Sort by create_time descending, then by sort policy score descending:

sort=-create_time;-RANK

When to use the sort clause vs. sort policy expressions

Approach	Best for	Who controls it
Sort clause fields (for example, `-create_time`)	Strict ordering by an attribute value regardless of relevance	Business logic, fixed at configuration time
Sort policy expressions (RANK)	Relevance-based ranking with complex scoring rules	Business logic, encoded in the policy
Combining both	Attribute-first ordering with relevance as a tiebreaker	Business logic

Example: An e-commerce site always shows the newest products first, with the most relevant results among products of the same age ranked higher:

sort=-create_time;-RANK

Configure static_bm25() in the rough sort policy and text_relevance(name) in the fine sort policy. For details, see Sort policy configuration.

Sort policies

Configure a sort policy

A sort policy is a hierarchical scoring configuration that combines built-in functions and expressions. Configure a rough sort policy, a fine sort policy, or both.

Example schema

Field	Type	Index
id	int	Keyword
name	text	General index for Chinese
age	int	Keyword

For this schema, configure static_bm25() in the rough sort policy and text_relevance(name) in the fine sort policy. Set the sort clause to sort=age;-RANK to sort by age ascending, then by RANK descending within each age group.

Important

All fields referenced by fine sort functions must be configured as attribute fields. Otherwise, the Invalid formula error is reported.

Fine sort functions

The following utility functions are available in fine sort expressions:

Function	Description	Example
`i in (value1, value2, …, valuen)`	Returns 1 if the value of `i` is in the set; returns 0 otherwise	`age in (1,2,3,4,5)` returns 1; `age in (6,7,8,9)` returns 0
`if(cond, then_value, else_value)`	Returns `then_value` if `cond` is not 0; returns `else_value` if `cond` is 0	`if(2,3,5)` returns 3; `if(0,3,5)` returns 5
`random()`	Returns a random value in [0, 1]	—
`now()`	Returns the number of seconds elapsed since 00:00:00 January 1, 1970 (UTC)	—

For text relevance, geographical location, timeliness, algorithm relevance, and other functions, see Fine sort functions.

For mathematical functions and operators, see Sort policy configuration.

If built-in expressions cannot meet the requirements of complex scenarios, use the Cava plug-in to write a custom scoring script. See Sort plug-in development — Cava language.

Common sort policy configurations

The following examples cover the most common business scenarios.

1. Add score points based on field conditions

Add 10 points if age > 10, 20 points if age > 40, and 30 points if weight > 60:

# Option A: multiply boolean result by the point value
(age>10)*10 + (age>40)*20 + (weight>60)*30

# Option B: use if() for explicit readability
if(age>10,10,0) + if(age>40,20,0) + if(weight>60,30,0)

2. Rank exact entity matches before partial matches

Rank "xxx Company" before "xxx Hangzhou Branch":

field_match_ratio(title)

3. Rank shorter prefix matches before longer ones

Rank "dim_itm_tb" before "dim_itm_tb_dst_itm_relation_dd" for the query all:'dim_itm_tb':

field_match_ratio(detail)

4. Rank documents where all query terms appear close together

Rank results for item:"iphone 8" OR item:'iphone 8' by proximity of matched terms:

query_min_slide_window(title)

5. Rank exact phrase matches above documents with scattered terms

When the search keyword is "Republic of China", rank documents containing the exact phrase before those containing "Interesting news of the Republic of China — Republic of China" or similar:

query_min_slide_window(title)

6. Prevent repeated BM25 scoring for the same keyword

Avoid static_bm25() giving inflated scores when a keyword appears many times in a field:

query_match_ratio(title)

7. Demote keyword-stuffed documents

Move documents with stacked search keywords to the bottom of results:

# Penalize documents where the keyword appears more than 3 times in the title
if(field_term_match_count(title)>3, 1, 10)

8. Boost documents with a non-empty field

Add 500 points to the sort score when a specific field is non-empty. Add a mark field to the source data: set mark = 0 for empty strings and mark = 1 for non-empty strings. Then configure the fine sort expression:

if(mark==1, 500, 0)

Debug sort scores

Enable Show Sort Details on the query results page to see the score breakdown for each document.

The output format is:

FirstRank:
expression[<rough sort expression>], result[<rough sort score>].
SecondRank:
expression[<fine sort expression>], result[<fine sort score>].

Use this output to diagnose ranking behavior:

Question	What to check
Why is document A ranked above document B?	Compare the fine sort results for each document
Why is a document's score 10,000 + X and not higher?	The document passed through fine sort; the rough sort score was replaced by the fine sort score
Why is a document's score capped at 20,000?	The document was only roughly sorted and its rough sort score hit the upper limit

Case study: text relevance ranking

This case study demonstrates how static_bm25() and text_relevance() interact to rank search results.

Functions used

Function	Description	Valid range
`static_bm25()`	Static text relevance; measures the match between the query and the document text	[0, 1]
`text_relevance(field)`	Text match degree based on the keyword in a specific field	[0, 1]

Use static_bm25() in the rough sort policy for fast first-pass filtering, and text_relevance(field) in the fine sort policy for precise re-ranking of the top N candidates.

Test data

id    name
1     Black humor, also known as "black comedy", is a modernist literary genre that emerged in the United States in the 1960s.
2     "Black Humor" is a song sung by Jay Chou.
3     Jay Chou, a Mandopop male singer, musician, music arranger, record producer, and magician from Taiwan (China).
4     Night is falling, and everything around is dark. To ease the oppressive atmosphere, Jay Chou humorously told a joke.
5     Black Humor female version (Original singer: Jay Chou)
6     Jay Chou"Black Humor"-Official Music Video

Setup

Create a rough sort policy named test_first_rank_name with the expression static_bm25().
Create a fine sort policy named test_second_rank_name with the expression text_relevance(name).
Create a query analysis task named test_qp and set Search Query Rewriting to OR.
Upload the test data to the OpenSearch application.

Case 1: rank by text relevance

Search query: Black Humor Jay Chou

Analysis: The query intent is the song "Black Humor" by Jay Chou. Document 4 ("Night is falling...Jay Chou humorously told a joke") matches after keyword splitting but is not relevant. The sort policy should push it below the relevant results.

Steps: Apply the rough sort policy (test_first_rank_name) and fine sort policy (test_second_rank_name) from setup steps 1–2. The text_relevance(name) function scores documents by how closely the name field matches the full query.

Case 2: rank with query rewriting (OR mode)

Search query: Black Humor Jay Chou (with OR rewriting)

Analysis: Same relevance problem as Case 1. With OR rewriting enabled, documents that match only "Black Humor" or only "Jay Chou" are also retrieved. The fine sort policy ranks documents matching both terms (or matching the phrase more closely) above those matching only one term.

Steps: Apply both sort policies (setup steps 1–2) and the query analysis task with OR rewriting (setup step 3).

What's next

Sort policy configuration — configure expressions, mathematical functions, and basic operations
Fine sort functions — full reference for text relevance, geographical location, timeliness, and algorithm relevance functions
Sort plug-in development — Cava language — write custom scoring scripts for complex scenarios