OpenSearch Industry Algorithm Edition addresses the core challenges of education search: question libraries that grow to tens of millions of entries, peak-hour concurrency spikes, cross-discipline query errors, and queries that combine image and text input. This page describes the query processing pipeline, key capabilities, and verified performance results from a production K12 deployment.
Challenges in education search
Test-question search presents a distinct set of problems that general-purpose search engines struggle with:
| Challenge | Description |
|---|---|
| Scale and growth | A question library can contain tens of millions of test questions and keeps growing, putting sustained pressure on the database. |
| Peak-hour concurrency | Most searches happen at the same time—morning study sessions, exam periods—causing high concurrent load and high search latency. |
| Cross-discipline query errors | Questions span mathematics, language, science, and other subjects. Without subject-aware query analysis, a mathematics query can surface irrelevant results from other disciplines. |
| Multimodal input | Students photograph a question and search by image. The pipeline must handle both image and text as query inputs. |
| Multilingual content | Libraries include content in English and other languages, requiring multilingual processing at query time. |
| Accuracy requirements | Search precision directly affects learning outcomes, so relevance must be high. |
How it works
When a query arrives, OpenSearch runs it through a six-stage pipeline before retrieving and ranking documents:
| Stage | What happens |
|---|---|
| Spelling correction | Typos and OCR errors in the query are corrected before processing begins. |
| Discipline category prediction | The query is classified into a subject discipline (for example, Mathematics). The predicted discipline feeds into the sort expression to boost discipline-relevant results. |
| Tokenization | The query is split into meaningful tokens for retrieval. |
| Term weight analysis | Each token is assigned an importance score of 7 (high), 4 (medium), or 1 (low). Tokens with a score of 1 are excluded from retrieval to reduce noise. |
| Synonym rewriting | Tokens are expanded using synonym rules—for example, "square centimeters" is rewritten to include "cm²" as an equivalent term. |
| Text vectorization | The query is encoded as a dense vector, enabling semantic matching alongside keyword retrieval. This combines the precision of keyword search with the contextual relevance of vector search, so results align with query intent even when exact terms differ. |
Example: For the query "What is the area of the following triangle in square centimeters?", the pipeline produces:
| Stage | Output |
|---|---|
| Discipline category prediction | Mathematics |
| Term weight analysis | 1 7 1 7 1 4 7 7 1 |
| Synonym rewriting | square centimeters → (cm ^ 2) |
| Text vectorization | -0.100582, -0.0540699, -0.0417337, 0.0602, ... |
Key capabilities
Exclusive analyzer for test questions
OpenSearch includes an analyzer built specifically for test-question queries. It applies the full six-stage pipeline described above. Upload your own query terms to extend the analyzer with a custom vocabulary for your question library.
Category prediction
Category prediction classifies each query into a subject discipline and question type. When the query contains an image, the model incorporates image information and optical character recognition (OCR) results to determine the discipline and question type.
The predicted category score feeds directly into the sort expression: a document whose category matches the query's predicted discipline receives a higher sort score and ranks higher.
Category prediction also identifies field types within a question—distinguishing the question description from answer options.
Term weight analysis
Term weight analysis assigns an importance score to each token in the query:
| Score | Meaning | Effect on retrieval |
|---|---|---|
| 7 | High importance | Always used for document retrieval |
| 4 | Medium importance | Used for document retrieval |
| 1 | Low importance | Excluded from retrieval |
The model is trained on user behavior data using a sequence labeling model that learns which terms are semantically central to a question versus which are structural connectors.
Example: For the query "The factors of 35 are () and the multiples of 24 within 100 are ()":
| Token | Weight |
|---|---|
| factors | 7 |
| multiples | 7 |
| 35 | 4 |
| 24 | 4 |
| Other tokens (are, of, within, etc.) | 1 |
OpenSearch uses "factors" and "multiples" as the primary retrieval terms and "35" and "24" as secondary terms. The low-weight tokens are excluded, which increases the number of relevant documents retrieved and reduces noise.
OCR often captures non-question elements from an image (watermarks, page numbers, publisher names). Term weight analysis assigns these elements a score of 1, preventing them from interfering with retrieval.
Query rewriting and synonym expansion
Query rewriting applies multiple interventions in a single pass: intervention dictionaries, spelling correction, synonym expansion, and term weight adjustments. These interventions are composable—for example, configure a synonym dictionary so that "cubic meters" expands to include "tons" as an equivalent unit, and simultaneously apply term weight analysis to suppress low-relevance structural tokens in the same query.
Custom sort
OpenSearch uses a two-phase sort to balance recall breadth with ranking precision:
Rough sort: Retrieves all matching documents and selects the top-N candidates using a base relevance score that combines term weight scores.
Fine sort: Re-ranks the top-N candidates using a custom sort expression that incorporates term weight scores, category prediction scores, and additional business signals such as question quality or recency.
How to configure a sort expression:
Write a sort expression that combines the signals you want and assign it to your application. A typical expression for test-question search combines three components:
| Signal | Source | Effect |
|---|---|---|
| Term weight score | Term weight analysis output | Boosts results that match high-importance query tokens |
| Category prediction score | Category prediction output | Boosts results whose subject discipline matches the query |
| Business signal | Custom field (for example, question quality rating) | Applies domain-specific ranking logic |
This lets you tune the trade-off between retrieval breadth and ranking precision without changing the underlying index.
Performance results
An online education platform running K12 content migrated from an Elasticsearch-based search service to OpenSearch. The platform serves tens of millions of users with a question library of approximately 80 million test questions, composed of its own library and third-party content.
After switching to OpenSearch:
| Metric | Before | After |
|---|---|---|
| Search accuracy (absolute improvement) | Baseline | +5% |
| Search latency | 100–300 ms | 50 ms |
| Data synchronization throughput | — | >4,000 TPS |
Before and after: search result quality
Query: "Zhang Huiyan says that the style of Song poetry in the Song dynasty is probably similar to Yuefu."
| Rank | Before OpenSearch | After OpenSearch |
|---|---|---|
| Top 1 | Zhang Hui is a solo singer of a song and dance troupe. Her wage is CNY 5,800 per month. In June 2006, Zhang Hui participated in three performances of the troupe in Shanghai and received a reward of CNY 3,800... | Zhang Huiyan says that the style of Song poetry in the Song dynasty is probably similar to Yuefu. |
| Top 2 | Zhang Huiyan's love for music comes from... | Zhang Huiyan says that the style of Song poetry in the Song dynasty is probably similar to Yuefu. () |
| Top 3 | Among the following documents, which one is the document that is cited in an article published by Ms. Zhang Hui in music periodicals of China? | Among the following options, which one is probably similar to the style of Song poetry in the Song dynasty that Zhang Huiyan said? |
The pre-OpenSearch results match on "Zhang Hui" (a common name) rather than the full entity "Zhang Huiyan." After switching to OpenSearch, all three top results are semantically relevant to the query.