Scroll search

更新时间:
复制 MD 格式

Scroll search retrieves large result sets by iterating through batches of matched documents. Unlike a regular search, which returns at most 5,000 documents, scroll search is designed to retrieve more results beyond this limit.

Scroll search is designed for batch processing tasks such as data export, analytics, and machine learning jobs—not for real-time or interactive user queries. For result sets of fewer than 5,000 documents, use a regular search instead.

How scroll search works

A scroll search runs in two phases:

  1. Initiate the scroll — Send a search request with search_type=scan and a scroll parameter. The response returns a scroll ID but no documents.

  2. Retrieve batches — Pass the scroll ID from the previous response to fetch the next batch of documents. Repeat until the items array in the response is empty.

Each scroll ID is valid for the duration specified by the scroll parameter. If you don't fetch the next batch before the scroll ID expires, the scroll session ends and you must restart from step 1.

Request parameters

ParameterTypeRequiredDescription
scrollSTRINGYesThe validity period of the scroll ID, specified as a number followed by a unit: weeks (w), days (d), hours (h), minutes (m), or seconds (s). Example: 1m. Required for the first request only.
search_typeSTRINGYes (first request only)The scroll search type. Set to scan. Required only for the first request.
scroll_idSTRINGYes (subsequent requests)The scroll ID returned by the previous response. Not required for the first request.
fetch_fieldsSTRINGNoThe fields to return in search results.

Response parameters

ParameterTypeDescription
statusSTRINGThe request result. Valid values: OK (success) or FAIL (failure). If FAIL, check the errors field for the error code.
request_idSTRINGThe request ID, used for troubleshooting.
resultSTRINGThe search results, including searchtime, total, num, viewtotal, items, facet, and scroll_id.
errorsSTRINGThe error details. The error_message field contains the error message. See Error codes.
Scroll search results are returned in fullJSON or JSON format only.

Run a scroll search

Step 1: Initiate the scroll

Send the first request with search_type=scan and a scroll value. The response contains a scroll ID and an empty items array—no documents are returned at this stage.

Example response for the first request:

{
    "status": "OK",
    "request_id": "150150574119953661605242",
    "result": {
        "searchtime": 0.005029,
        "total": 1,
        "num": 0,
        "viewtotal": 1,
        "scroll_id": "eJxtUMtuhDAM/BrvOYQC5cABdulvRFFIirsm2TpBavv1Ndut1EMlS36NZ0Y2ZHMxbueceAjIuWCMnrPjRITLyfzZm83y9V QVGT8x80U3PxQNUqieVZV1/an4ItbTUBPSx5wgXqKdvOSbmuKR8ZYjGWWirB4tvToAiX7u3G2eCNK77vnz8GlGPAV6suKBeqxAn0OiTd7NGEnesspyoyFLF6hecn4JUKjVgp0K3FnkfMfIyPoDuYWegX9GeYOpicY9TG8gwOSuBL04X1 MMg3ROwCesLlG6X7a2o=",
        "items": [],
        "facet": []
    },
    "errors": [],
    "tracer": ""
}

Step 2: Retrieve document batches

Pass the scroll_id from the previous response to fetch the next batch. The scroll ID in the response is updated with each request—always use the most recent one.

Set the hit parameter on the first request to control the batch size. This value is fixed for the entire session; changing hit in later requests has no effect. Each batch can contain at most 500 documents.

Example response for a follow-up request:

{
    "status": "OK",
    "request_id": "150150574119952551519970",
    "result": {
        "searchtime": 0.006293,
        "total": 1,
        "num": 1,
        "viewtotal": 1,
        "scroll_id": "eJxNT9tugzAM/RrznIRC4YEHaNlvRFFIhteQtE6Qtn39TNdJk2z5dnx8rIPJRdudcqKhl60Uir2Vp06ISv8b6s3QbZCVzpaCdp93XXBzg2wEW9MJ2dWq8q7YVXt0YckDLlBP0WyOw31N8YgYizZEnAUsjkx4VT4k8zexpjiNS/XYHX0NNkWP71BfVyxQjxLUxSfazFH4PYSPnCL3iMniDZq3jN98aFRCgGrZniy8/itkBHWGuYVeQH+B+QzTCUZ1NJ9gj4FVMfrQPr8Y+Hk+dgU14fIDVhtfTw==",
        "items": [
            {
                "fields": {
                    "cate_id": "0",
                    "float_arr": "0",
                    "id": "1",
                    "int_arr": "0",
                    "litteral_arr": "Search",
                    "name": "Search",
                    "phone": "1381111****",
                    "index_name": "app_schema_demo"
                },
                "property": {},
                "attribute": {},
                "variableValue": {},
                "sortExprValues": [
                    "1"
                ]
            }
        ],
        "facet": []
    },
    "errors": [],
    "tracer": ""
}

Step 3: Detect end of results

When items is an empty array, all documents have been retrieved and the scroll session is complete. Stop sending requests at this point.

{
    "items": []
}

Limitations

LimitationDetails
Unsupported featuresAggregate clauses, distinct clauses, rough sort expressions, fine sort expressions, and query analysis are not supported.
SortingSorting is supported only on a single field of the INT type. Requires OpenSearch API and SDK version V3 or later.
PaginationThe start parameter in the config clause has no effect during a scroll search. The default value 0 is always used, and pages cannot be skipped.
Batch sizeEach batch can contain at most 500 documents. Set the hit parameter on the first request to control the number of documents per batch.
Cross-application scrollingScroll searches across applications are not supported.
Response formatResults are returned in fullJSON or JSON format only.

Troubleshooting

Error: "Scroll_id is expired"

The scroll ID expired before the next batch was fetched. Modify the scroll parameter and restart the scroll session from step 1.

Invalid scroll_id

If the value of the scroll_id parameter in a request is invalid, an error occurs.

Checking for errors

Use the error code and message in the errors field to diagnose failures—do not rely on the status field alone. See Error codes for the full list.

What's next