Push demo

Prerequisites

Before you begin, ensure that you have:

• An OpenSearch High-Performance Search Edition application with a configured table

• A RAM user with the required permissions — see AliyunServiceRoleForOpenSearch and Access authorization rules

Set up credentials

Store your AccessKey ID and AccessKey secret as environment variables. Never hardcode credentials in source files.

Linux and macOS

Replace <access_key_id> and <access_key_secret> with the RAM user's credentials:

export ALIBABA_CLOUD_ACCESS_KEY_ID=<access_key_id>
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=<access_key_secret>

Windows

1. Create an environment variable file and add ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET with your AccessKey ID and AccessKey secret as the values.

2. Restart Windows for the environment variables to take effect.

How it works

Pushing data with the Java SDK follows three steps:

1. Build each document as a Map object and assign a command (ADD or DELETE).

2. Serialize all documents into a JSONArray string and load it into the client buffer.

3. Call documentClient.push() to submit the entire batch to the application at once.

Each batch can only contain fields from a single table. Cross-table pushes are not supported.

Supported commands

Data format

Datasets must be in the valid format. To download a sample file as a template:

1. Log on to the OpenSearch console.

2. In the Actions column for your application, choose More > Upload File.

3. Download the sample file.

If the number of documents in a single push exceeds the limit, the push fails with an error.

Push and query documents

The following example pushes two documents (doc1 and doc2), updates doc2 using doc3, deletes doc1, then queries the results to verify. All operations use documentClient.push() to submit the batch.

package com.aliyun.opensearch;

import com.aliyun.opensearch.sdk.dependencies.com.google.common.collect.Lists;
import com.aliyun.opensearch.sdk.dependencies.com.google.common.collect.Maps;
import com.aliyun.opensearch.sdk.dependencies.org.json.JSONArray;
import com.aliyun.opensearch.sdk.dependencies.org.json.JSONObject;
import com.aliyun.opensearch.sdk.generated.OpenSearch;
import com.aliyun.opensearch.sdk.generated.commons.OpenSearchClientException;
import com.aliyun.opensearch.sdk.generated.commons.OpenSearchException;
import com.aliyun.opensearch.sdk.generated.commons.OpenSearchResult;
import com.aliyun.opensearch.sdk.generated.document.Command;
import com.aliyun.opensearch.sdk.generated.document.DocumentConstants;
import com.aliyun.opensearch.sdk.generated.search.*;
import com.aliyun.opensearch.sdk.generated.search.general.SearchResult;

import java.io.UnsupportedEncodingException;
import java.nio.charset.Charset;
import java.util.Map;
import java.util.Random;

public class testPushSearch2 {

    private String appName = "The name of the OpenSearch application to which you want to push data";
    private String tableName = "The name of the table to which you want to push data";
    private String host = "The endpoint of the OpenSearch API in your region";

    public static void main(String[] args) {
        // Read credentials from environment variables.
        // Set up the environment variables before running this code.
        // See the "Set up credentials" section for instructions.
        String accesskey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String secret = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");

        // Print the default encoding format.
        System.out.println(String.format("file.encoding: %s", System.getProperty("file.encoding")));
        System.out.println(String.format("defaultCharset: %s", Charset.defaultCharset().name()));

        // Generate random values to use as primary keys.
        Random rand = new Random();
        int value1 = rand.nextInt(Integer.MAX_VALUE);
        int value2 = rand.nextInt(Integer.MAX_VALUE);

        // Build doc1.
        Map<String, Object> doc1 = Maps.newLinkedHashMap();
        doc1.put("id", value1);

        String title_string = "Add doc1 in push mode"; // UTF-8 encoded
        byte[] bytes;
        try {
            bytes = title_string.getBytes("utf-8");
            String utf8_string = new String(bytes, "utf-8");
            doc1.put("name", utf8_string);
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }

        doc1.put("phone", "137****1111");

        int[] int_arr = {11, 11};
        doc1.put("int_arr", int_arr);

        String[] literal_arr1 = {"Add doc1 in push mode", "Test adding doc1 in push mode"};
        doc1.put("literal_arr", literal_arr1);

        float[] float_arr = {(float) 1.1, (float) 1.1};
        doc1.put("float_arr", float_arr);

        doc1.put("cate_id", 1);

        // Wrap doc1 in a JSONObject with the ADD command.
        // Standard applications do not support UPDATE or partial updates.
        // Use ADD to add a new document or fully replace an existing one.
        JSONObject json1 = new JSONObject();
        json1.put(DocumentConstants.DOC_KEY_CMD, Command.ADD.toString());
        json1.put(DocumentConstants.DOC_KEY_FIELDS, doc1);

        // Build doc2.
        Map<String, Object> doc2 = Maps.newLinkedHashMap();
        doc2.put("id", value2);

        String title_string2 = "Add doc2 in push mode"; // UTF-8 encoded
        byte[] bytes2;
        try {
            bytes2 = title_string2.getBytes("utf-8");
            String utf8_string2 = new String(bytes2, "utf-8");
            doc2.put("name", utf8_string2);
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }

        doc2.put("phone", "137****2222");

        int[] int_arr2 = {22, 22};
        doc2.put("int_arr", int_arr2);

        String[] literal_arr2 = {"Add doc2 in push mode", "Test adding doc2 in push mode"};
        doc2.put("literal_arr", literal_arr2);

        float[] float_arr2 = {(float) 2.2, (float) 2.2};
        doc2.put("float_arr", float_arr2);

        doc2.put("cate_id", 1);

        // ADD adds doc2. If a document with value2 as the primary key already exists, it is fully replaced.
        JSONObject json2 = new JSONObject();
        json2.put(DocumentConstants.DOC_KEY_CMD, Command.ADD.toString());
        json2.put(DocumentConstants.DOC_KEY_FIELDS, doc2);

        // Build doc3, which updates doc2 (same primary key: value2).
        Map<String, Object> doc3 = Maps.newLinkedHashMap();
        doc3.put("id", value2);

        String title_string3 = "Update doc2 to doc3 in push mode"; // UTF-8 encoded
        byte[] bytes3;
        try {
            bytes3 = title_string3.getBytes("utf-8");
            String utf8_string3 = new String(bytes3, "utf-8");
            doc3.put("name", utf8_string3);
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }

        doc3.put("phone", "137****3333");

        int[] int_arr3 = {33, 33};
        doc3.put("int_arr", int_arr3);

        String[] literal_arr3 = {"Update doc2 to doc3 in push mode", "Update doc2 to doc3 in push mode"};
        doc3.put("literal_arr", literal_arr3);

        float[] float_arr3 = {(float) 3.3, (float) 3.3};
        doc3.put("float_arr", float_arr3);

        doc3.put("cate_id", 1);

        // ADD with value2 as the primary key replaces doc2 with doc3.
        JSONObject json3 = new JSONObject();
        json3.put(DocumentConstants.DOC_KEY_CMD, Command.ADD.toString());
        json3.put(DocumentConstants.DOC_KEY_FIELDS, doc3);

        // Build a DELETE command for doc1. Only the primary key is required.
        Map<String, Object> doc4 = Maps.newLinkedHashMap();
        doc4.put("id", value1);

        JSONObject json4 = new JSONObject();
        json4.put(DocumentConstants.DOC_KEY_CMD, Command.DELETE.toString());
        json4.put(DocumentConstants.DOC_KEY_FIELDS, doc4);

        // Assemble the batch: add doc1, add doc2, update doc2 to doc3, delete doc1.
        JSONArray docsJsonArr = new JSONArray();
        docsJsonArr.put(json1); // Add doc1
        docsJsonArr.put(json2); // Add doc2
        docsJsonArr.put(json3); // Update doc2 to doc3
        docsJsonArr.put(json4); // Delete doc1
        String docsJson = docsJsonArr.toString();

        // Initialize the client.
        OpenSearch openSearch = new OpenSearch(accesskey, secret, host);
        OpenSearchClient serviceClient = new OpenSearchClient(openSearch);
        DocumentClient documentClient = new DocumentClient(serviceClient);

        try {
            // Push the batch.
            OpenSearchResult osr = documentClient.push(docsJson, appName, tableName);

            // A successful response means the push request was accepted without API-level errors.
            // The application may still encounter indexing errors after the push — for example,
            // if a field value cannot be converted to the expected type. Check the error logs
            // in the OpenSearch console to confirm that all documents were indexed successfully.
            if (osr.getResult().equalsIgnoreCase("true")) {
                System.out.println("Push accepted. Request ID: " + osr.getTraceInfo().getRequestId());
            } else {
                System.out.println("Push failed: " + osr.getTraceInfo());
            }

        } catch (OpenSearchException e) {
            e.printStackTrace();
        } catch (OpenSearchClientException e) {
            e.printStackTrace();
        }

        // Hibernate the thread for 1 second.
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        // Query the pushed documents.
        SearcherClient searcherClient = new SearcherClient(serviceClient);

        // Configure paging and the response format.
        Config config = new Config(Lists.newArrayList(appName));
        config.setStart(0);
        config.setHits(30);
        // Supported formats: XML and JSON. FULLJSON is not supported.
        config.setSearchFormat(SearchFormat.JSON);

        // Specify the fields to return.
        config.setFetchFields(Lists.newArrayList("id", "name", "phone", "int_arr", "literal_arr", "float_arr", "cate_id"));

        SearchParams searchParams = new SearchParams(config);

        // Query by primary key. Specify multiple values in a single setQuery call —
        // multiple setQuery calls overwrite each other.
        searchParams.setQuery("id:'" + value1 + "'|'" + value2 + "'");

        // Apply a filter condition.
        searchParams.setFilter("cate_id<=3");

        // Sort by id (descending), then by RANK (ascending) for documents with the same id.
        Sort sorter = new Sort();
        sorter.addToSortFields(new SortField("id", Order.DECREASE));
        sorter.addToSortFields(new SortField("RANK", Order.INCREASE));
        searchParams.setSort(sorter);

        SearchResult searchResult;
        try {
            searchResult = searcherClient.execute(searchParams);
            String result = searchResult.getResult();
            JSONObject obj = new JSONObject(result);
            System.out.println(obj.toString());

        } catch (OpenSearchException e) {
            e.printStackTrace();
        } catch (OpenSearchClientException e) {
            e.printStackTrace();
        }
    }
}

Understanding the push result

A successful push means the request was accepted without API-level errors. After acceptance, the application may still encounter indexing errors — for example, if a field value cannot be converted to the expected type. Check the error logs in the OpenSearch console to confirm that all documents were indexed successfully.

Next steps

After pushing data, configure relevance and ranking settings for your application to improve search result quality.