Import data by API or SDK

Field type mappings

OpenSearch field type	API push field type
INT	int or long
INT_ARRAY	int[] or long[]
FLOAT	float
FLOAT_ARRAY	float[]
DOUBLE	double
DOUBLE_ARRAY	double[]
LITERAL	string
LITERAL_ARRAY	string[]
SHORT_TEXT	string
TEXT	string
TIMESTAMP	long
GEO_POINT	A string in the lon lat format, where lon is longitude and lat is latitude. The values must be of the double type and separated by a space. The lon value must be in the [-180, 180] range, and the lat value must be in the [-90, 90] range.

Push data via API

1. The console's file upload feature is for testing purposes and is unsuitable for large-scale data uploads. It is inefficient and limited to 2 MB per file. For more information about the system limits for pushing data via an API or SDK, see System limits.

   On the instance management page, find your target instance. In the Actions column, click the dropdown menu and select Upload File.

2. The API and SDK do not support data source plug-ins, so you must process and structure your data before pushing it. For example, when pushing array-type data via the API, ensure you upload it as an array, not a string. The following code shows an example:

   // Define a Map object to store the document data to be uploaded. This is Document 2.
   Map<String,Object> doc2 =Maps.newLinkedHashMap();
        doc2.put("id",1);
   String[] literal_arr2 ={"Element 1","Element 2"};
        doc2.put("literal_arr", literal_arr2);

3. For more details about using an API or SDK, see the following resources:

   Development Guide

   Java SDK: Commit push demo and Push demo

   PHP SDK: Push demo and Push with search demo

   Go SDK: Behavioral data push demo and Push demo

   TypeScript/Node.js SDK: Install the @alicloud/opensearch-sdk package.

   npm install @alicloud/opensearch-sdk

   Use the pushDocument interface to push data:

   import Client, { Config, Document, PushDocumentRequestModel } from '@alicloud/opensearch-sdk';
   
   // Authenticate with an AccessKey.
   const config = new Config({
     endpoint: 'opensearch-cn-beijing.aliyuncs.com',  // The data API endpoint.
     accessKeyId: process.env.ALIBABA_CLOUD_ACCESS_KEY_ID,
     accessKeySecret: process.env.ALIBABA_CLOUD_ACCESS_KEY_SECRET,
   });
   const client = new Client(config);
   
   // Construct the documents to be pushed.
   const docs = [
     new Document({
       cmd: 'ADD',
       fields: { id: '1', title: 'Example document', body: 'Document content' },
     }),
   ];
   
   // Push the data to the specified table of your application.
   const request = new PushDocumentRequestModel({ body: docs });
   const response = await client.pushDocument('your_app_name', 'your_table_name', request);
   console.log(response.body);

   The endpoint format for different regions is opensearch-cn-{region}.aliyuncs.com. For example, the endpoint for the China (Hangzhou) region is opensearch-cn-hangzhou.aliyuncs.com. SDK authentication uses an Alibaba Cloud AccessKey (AK/SK) that is passed in through the Config object, not API keys.

   OpenSearch provides two npm packages: @alicloud/opensearch-sdk for the data API (search, data push, and query suggestion), and @alicloud/opensearch20171225 for the control API (application management). For data push scenarios, use @alicloud/opensearch-sdk.

Related topics

1. To import data by using an API or SDK, see Commit push demo.

2. To upload files from the console (limited to 2 MB per file), see the Tutorial.

3. To import data from a data source, see Configure an RDS data source.