Data push-OpenSearch(Open Search)-阿里云帮助中心

Push documents to OpenSearch using the API or SDK. Pushes are processed asynchronously—an OK response means the system received the data, not that processing succeeded. Monitor the OpenSearch console for processing errors after each push.

Warning

Always check return values after a push. If error code 3007 is returned, retry the request. Failing to retry error codes can cause data loss.

Push commands

Three commands control how documents are written to the index. The key difference between add and update is field coverage: add overwrites all fields, while update only touches the fields you specify.

Command	What it does	Fields not in the request	Document already exists?
`add`	Adds or fully replaces a document	Default to blank	Deleted first, then re-added
`update`	Updates specific fields in a document	Retain original values	Deleted first, then re-added
`delete`	Deletes a document by primary key ID	N/A	Operation succeeds even if the document does not exist

When to use each command

Use add to replace a document completely or add a new one.
Use update to change a subset of fields without overwriting the rest.
Use delete to remove a document by its primary key ID.

How data push works

OpenSearch processes push requests asynchronously:

Submit a push request via the API or SDK using HTTP POST.
The system returns OK to confirm receipt.
The system processes the data in the background.
If processing fails, the error appears in the OpenSearch console.

Check the console regularly to catch processing errors early.

Limits and best practices

HTTP POST size limit

Each HTTP POST request has a maximum payload size of 2 MB. If the payload exceeds 2 MB, the server rejects the request and returns an error.

See Limits for the full list of quotas.

Batch your pushes

Pushing documents one at a time generates a large number of individual network calls and increases the chance of hitting rate limits. Package multiple documents into a single request to improve throughput and avoid throttling errors.

Keep each batch under 2 MB to stay within the per-request size limit.

Also perform URL encoding on both the URL and the body of each HTTP POST request. Without encoding, resolution and signature errors may occur.

Incremental data and duplicate keys

When pushing incremental data from a data source or via the API, documents with duplicate primary key values are overwritten.

Error handling

Error message	Cause	Action
`connection timed out`	Network interruption	Retry until successful
`1000 :system error`	System-side error	Retry until successful
`push rate exceeds app quota`	High push frequency, or payload > 2 MB per second	Retry until successful; reduce payload size if the error persists
`request too frequently`	Total push count exceeded the limit	Batch documents and push in fewer, larger requests

ApsaraDB RDS for MySQL synchronization

The rate and size of data synchronized from ApsaraDB RDS for MySQL are limited. See Limits.

A binary log is generated whenever all tables in the database are updated. If binary log volume exceeds the limit, data queues up and synchronization delays occur. To avoid this:

Move frequently updated tables into separate databases.
Enable automatic data synchronization for the primary table, and use API operations to sync secondary tables.