This topic describes how to use OpenSearch Vector Search Edition SDKs for Java and Python to update table data. You can upload, update, and delete documents.
Dependencies
Java
<dependency>
<groupId>com.aliyun</groupId>
<artifactId>aliyun-sdk-ha3engine-vector</artifactId>
<version>1.1.18</version>
</dependency>
Python
# Requires: Python >=3.6
pip install alibabacloud_ha3engine_vector==1.1.18
Go
go get github.com/aliyun/alibabacloud-ha3-go-sdk@v1.1.18-vector
Asynchronous Java
<dependency>
<groupId>com.aliyun</groupId>
<artifactId>aliyun-sdk-ha3engine-async</artifactId>
<version>1.1.8</version>
</dependency>
Parameter description
You must specify the following parameters in the SDKs for Java and Python: endpoint, instance_id, access_user_name, access_pass_word, and data_source_name.
endpoint: The internal or public endpoint. Specify the domain name only, without thehttp://prefix, such asha-cn-xxxxxxxx.public.ha.aliyuncs.com.
This information is available on the Instance Details page in the Network Information and API Endpoint sections.
After you turn on Public Access, you can access the OpenSearch Vector Search Edition instance by using the public endpoint of the instance on your on-premises machine. A public endpoint contains the word "public". You can configure an IP address whitelist for access to the instance. For more information, see Network Information section in the "View instance details" topic. If you do not configure such an IP address whitelist, an access prohibition error will be returned. We recommend that you ping the instance before you use it to ensure that the instance is accessible.
If you use an Elastic Compute Service (ECS) instance to access the OpenSearch Vector Search Edition instance, you can specify the same vSwitch as that of the ECS instance to access the OpenSearch Vector Search Edition instance by using the API endpoint.
instance_id: the ID of the OpenSearch Vector Search Edition instance.
access_user_name: the username.
access_pass_word: the password.
You can view the username and password in the API Endpoint section of the Instance Details page. The password is specified when you purchase the instance and can be modified.
data_source_name: The name of the data source for API data submission. The default format is<instance_id>_<table_name>.
data_source_name=ha-cn-zpr3dgzxg04_test_image_vector
Data update demo
Upload a document
Java
import com.aliyun.ha3engine.vector.Client;
import com.aliyun.ha3engine.vector.models.Config;
import com.aliyun.ha3engine.vector.models.PushDocumentsRequest;
import com.aliyun.ha3engine.vector.models.PushDocumentsResponse;
import com.aliyun.tea.TeaException;
import org.junit.Before;
import org.junit.Test;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;
/**
* @author alibaba
*/
public class PushDoc {
/**
* The OpenSearch Vector Search Edition client.
*/
private Client client;
@Before
public void clientInit() throws Exception {
/*
Initialize the OpenSearch Vector Search Edition client.
*/
Config config = new Config();
// The instance name. You can find it in the upper-left corner of the Instance Details page. Example: ha-cn-i7*****605.
config.setInstanceId("ha-cn-i7*****605");
// The username. You can find it in the API Endpoint section on the Instance Details page.
config.setAccessUserName("username");
// The password. You can change it in the API Endpoint section on the Instance Details page.
config.setAccessPassWord("password");
// The API endpoint. You can find it in the API Endpoint section on the Instance Details page.
config.setEndpoint("ha-cn-i7*****605.public.ha.aliyuncs.com");
client = new Client(config);
}
@Test
public void add() throws Exception {
// The name of the table to which you want to push documents. The format is `_`.
String tableName = "<instance_id>_<table_name>";
// The primary key field for the documents to be pushed.
String pkField = "<field_pk>";
try {
// This is the outer structure for document pushes. It can contain one or more document operations.
ArrayList<Map<String, ?>> documents = new ArrayList<>();
// Add a document.
Map<String, Object> add2Document = new HashMap<>();
Map<String, Object> add2DocumentFields = new HashMap<>();
// Insert the document content. Key-value pairs must match.
// The field must be consistent with the pkField parameter.
add2DocumentFields.put("<field_pk>", "<field_pk_value>");
add2DocumentFields.put("<field_map_key_1>", "<field_map_value_1>");
add2DocumentFields.put("<field_map_key_2>", "<field_map_value_2>");
// OpenSearch Vector Search Edition supports multi-value attributes. In the index, set `"multi_value": true`.
ArrayList<Object> addDocumentMultiFields = new ArrayList<>();
addDocumentMultiFields.add("multi_value_1");
addDocumentMultiFields.add("multi_value_2");
add2DocumentFields.put("<multi_value_key>", addDocumentMultiFields);
// Add the document content to the add2Document structure.
add2Document.put("fields", add2DocumentFields);
// The command to add a new document is `add`.
add2Document.put("cmd", "add");
documents.add(add2Document);
// Push the data.
PushDocumentsRequest request = new PushDocumentsRequest();
// By default, the system checks whether the primary key field exists when data is pushed. To disable this check, set the request header `X-Opensearch-Validate-Data` to `false`.
// request.headers = new HashMap<>();
// request.headers.put("X-Opensearch-Validate-Data", "false");
request.setBody(documents);
PushDocumentsResponse response = client.pushDocuments(tableName, pkField, request);
String responseBody = response.getBody();
System.out.println("result: " + responseBody);
} catch (TeaException e) {
System.out.println(e.getCode());
System.out.println(e.getMessage());
Map<String, Object> exceptionData = e.getData();
System.out.println(com.aliyun.teautil.Common.toJSONString(exceptionData));
}
}
}Python
from alibabacloud_ha3engine_vector.client import Client
from alibabacloud_ha3engine_vector.models import Config
from alibabacloud_ha3engine_vector import models
from Tea.exceptions import TeaException, RetryError
config = Config(
# The API endpoint. You can find it in the API Endpoint section on the Instance Details page.
endpoint="ha-cn-i7*****605.public.ha.aliyuncs.com",
# The username. You can find it in the API Endpoint section on the Instance Details page.
access_user_name="username",
# The password. You can change it in the API Endpoint section on the Instance Details page.
access_pass_word="password")
# Initialize the engine client.
client = Client(config)
def push():
# The name of the table to which you want to push documents. The format is `_`.
tableName = "<instance_id>_<table_name>";
try:
# Add a document.
# If the document already exists, it is deleted and then re-added.
# =====================================================
# Update the document content.
add2DocumentFields = {
"id": 1, # Primary key ID of the INT single-value type.
"name": "search", # STRING single-value type.
"str_arr": "a\x1Db\x1Dc\x1Dd" # STRING multi-value type.
}
# Add the document content to the add2Document structure.
add2Document = {
"fields": add2DocumentFields,
"cmd": "add" # The command to add a new document is `add`.
}
optionsHeaders = {}
# This is the outer structure for document pushes. It can contain one or more document operations.
documentArrayList = []
documentArrayList.append(add2Document)
pushDocumentsRequest = models.PushDocumentsRequest(optionsHeaders, documentArrayList)
# By default, the system checks whether the primary key field exists when data is pushed. To disable this check, set the request header `X-Opensearch-Validate-Data` to `false`.
# pushDocumentsRequest.headers = {"X-Opensearch-Validate-Data": "false"}
# The primary key field for the documents to be pushed.
pkField = "id"
# Send the request by using the default runtime parameters.
response = client.push_documents(tableName, pkField, pushDocumentsRequest)
print(response.body)
except TeaException as e:
print(f"send request with TeaException : {e}")
except RetryError as e:
print(f"send request with Connection Exception : {e}")
if __name__ == "__main__":
push()Go
The demo dynamically encapsulates document data into Map objects and calls the add() method to add these Map objects to the cache. Then, the demo calls the pushDocuments() method to submit the document data in these Map objects at a time.
package main
import (
"fmt"
"github.com/alibabacloud-go/tea/tea"
ha3engine "github.com/aliyun/alibabacloud-ha3-go-sdk/client"
)
func main() {
// Create a client instance for requests.
config := &ha3engine.Config{
// The API endpoint. You can find it in the API Endpoint section on the Instance Details page.
Endpoint: tea.String("ha-cn-i7*****605.public.ha.aliyuncs.com"),
// The username. You can find it in the API Endpoint section on the Instance Details page.
AccessUserName: tea.String("username"),
// The password. You can change it in the API Endpoint section on the Instance Details page.
AccessPassWord: tea.String("password"),
}
// Initialize a client to send requests.
client, _clientErr := ha3engine.NewClient(config)
// If an error occurs during client initialization, _clientErr is returned and the error message is printed.
if _clientErr != nil {
fmt.Println(_clientErr)
return
}
docPush(client)
}
func docPush(client *ha3engine.Client) {
pushDocumentsRequestModel := &ha3engine.PushDocumentsRequest{}
// The name of the table to which you want to push documents. The format is `_`.
tableName := "<instance_id>_<table_name>"
// The primary key field for the documents to be pushed.
keyField := "<field_pk>"
// The vector field.
vector := [4]float32{1.2, 2.2, 3.2, 4.2}
filed := map[string]interface{}{
"fields": map[string]interface{}{
"id": 1,
"name": "test",
"vector": vector,
},
"cmd": tea.String("add"),
}
array := []map[string]interface{}{}
array = append(array, filed)
pushDocumentsRequestModel.SetBody(array)
// By default, the system checks whether the primary key field exists when data is pushed. To disable this check, set the request header `X-Opensearch-Validate-Data` to `false`.
//headers := map[string]*string{}
//headers["X-Opensearch-Validate-Data"] = tea.String("false")
//pushDocumentsRequestModel.SetHeaders(headers)
// Call the method to send the request.
response, _requestErr := client.PushDocuments(tea.String(tableName), tea.String(keyField), pushDocumentsRequestModel)
// If an error occurs while sending the request, _requestErr is returned and the error message is printed.
if _requestErr != nil {
fmt.Println(_requestErr)
return
}
// Print the content of the response.
fmt.Println(response)
}Java in asynchronous mode
The demo dynamically encapsulates document data into Map objects and calls the add() method to add these Map objects to the cache. Then, the demo calls the pushDocuments() method to submit the document data in these Map objects at a time.
import com.aliyun.ha3engine.async.AsyncClient;
import com.aliyun.ha3engine.async.models.PushDocumentsRequest;
import com.aliyun.ha3engine.async.models.PushDocumentsResponse;
import com.aliyun.sdk.ha3engine.async.core.AsyncConfigInfoProvider;
import com.aliyun.tea.TeaException;
import darabonba.core.client.ClientOverrideConfiguration;
import org.junit.Before;
import org.junit.Test;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
/**
* @author alibaba
*/
public class AddDoc {
/**
* The OpenSearch Vector Search Edition client.
*/
private AsyncClient client;
@Before
public void clientInit() {
// Configure the username and password for the instance. You can find them in the API Endpoint section on the Instance Details page.
AsyncConfigInfoProvider provider = AsyncConfigInfoProvider.create("username", "password");
// Initialize the asynchronous client.
client = AsyncClient.builder()
.credentialsProvider(provider)
.overrideConfiguration(
ClientOverrideConfiguration.create()
.setEndpointOverride("ha-cn-i7*****605.public.ha.aliyuncs.com")
.setProtocol("http")
).build();
}
@Test
public void add() {
try {
// The name of the table for document pushes.
String tableName = "<table_name>";
// The primary key field for the documents to be pushed.
String pkField = "<field_pk>";
// This is the outer structure for document pushes. It can contain one or more document operations.
ArrayList<Map<String, ?>> documents = new ArrayList<>();
// Add a document.
Map<String, Object> add2Document = new HashMap<>();
Map<String, Object> add2DocumentFields = new HashMap<>();
// Update the document content. Key-value pairs must match.
// The field must be consistent with the pkField parameter.
add2DocumentFields.put("<field_pk>", "<field_pk_value>");
add2DocumentFields.put("<field_map_key_1>", "<field_map_value_1>");
add2DocumentFields.put("<field_map_key_2>", "<field_map_value_2>");
// OpenSearch Vector Search Edition supports multi-value attributes. In the index, set `"multi_value": true`.
ArrayList<Object> addDocumentMultiFields = new ArrayList<>();
addDocumentMultiFields.add("multi_value_1");
addDocumentMultiFields.add("multi_value_2");
add2DocumentFields.put("<multi_value_key>", addDocumentMultiFields);
// Add the document content to the add2Document structure.
add2Document.put("fields", add2DocumentFields);
// The command to add a new document is `add`.
add2Document.put("cmd", "add");
documents.add(add2Document);
// Push the data.
PushDocumentsRequest request = PushDocumentsRequest.builder().body(documents).build();
// By default, the system checks whether the primary key field exists when data is pushed. To disable this check, set the request header `X-Opensearch-Validate-Data` to `false`.
// request.getHeaderParameters().put("X-Opensearch-Validate-Data", "false");
CompletableFuture<PushDocumentsResponse> responseCompletableFuture = client.pushDocuments(tableName, pkField, request);
String responseBody = responseCompletableFuture.get().getBody();
System.out.println("result: " + responseBody);
} catch (ExecutionException | InterruptedException e) {
System.out.println(e.getMessage());
} catch (TeaException e) {
System.out.println(e.getCode());
System.out.println(e.getMessage());
Map<String, Object> exceptionData = e.getData();
System.out.println(com.aliyun.teautil.Common.toJSONString(exceptionData));
}
}
}If you call the add() method to push data, new data that uses the same primary key as the old data overwrites the old data.
Delete a document
Java
import com.aliyun.ha3engine.vector.Client;
import com.aliyun.ha3engine.vector.models.*;
import com.aliyun.tea.TeaException;
import org.junit.Before;
import org.junit.Test;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;
/**
* @author alibaba
*/
public class PushDoc {
/**
* The OpenSearch Vector Search Edition client.
*/
private Client client;
@Before
public void clientInit() throws Exception {
/*
Initialize the OpenSearch Vector Search Edition client.
*/
Config config = new Config();
// The instance name. You can find it in the upper-left corner of the Instance Details page. Example: ha-cn-i7*****605.
config.setInstanceId("ha-cn-i7*****605");
// The username. You can find it in the API Endpoint section on the Instance Details page.
config.setAccessUserName("username");
// The password. You can change it in the Network Information section on the Instance Details page.
config.setAccessPassWord("password");
// The API endpoint. You can find it in the API Endpoint section on the Instance Details page.
config.setEndpoint("ha-cn-i7*****605.public.ha.aliyuncs.com");
client = new Client(config);
}
@Test
public void delete() throws Exception {
// The name of the table to which you want to push documents. The format is `_`.
String tableName = "<instance_id>_<table_name>";
// The primary key field for the documents to be pushed.
String pkField = "<field_pk>";
try {
// This is the outer structure for document pushes. It can contain one or more document operations.
ArrayList<Map<String, ?>> documents = new ArrayList<>();
// Delete a document.
Map<String, Object> delete2Document = new HashMap<>();
Map<String, Object> delete2DocumentFields = new HashMap<>();
// Insert the document content. Key-value pairs must match.
// The field must be consistent with the pkField parameter.
delete2DocumentFields.put("<field_pk>", "<field_pk_value>");
// Add the document content to the delete2Document structure.
delete2Document.put("fields", delete2DocumentFields);
// The command to delete a document is `delete`.
delete2Document.put("cmd", "delete");
documents.add(delete2Document);
// Push the data.
PushDocumentsRequest request = new PushDocumentsRequest();
request.setBody(documents);
PushDocumentsResponse response = client.pushDocuments(tableName, pkField, request);
String responseBody = response.getBody();
System.out.println("result: " + responseBody);
} catch (TeaException e) {
System.out.println(e.getCode());
System.out.println(e.getMessage());
Map<String, Object> exceptionData = e.getData();
System.out.println(com.aliyun.teautil.Common.toJSONString(exceptionData));
}
}
}Python
from alibabacloud_ha3engine_vector.client import Client
from alibabacloud_ha3engine_vector.models import Config
from alibabacloud_ha3engine_vector import models
from Tea.exceptions import TeaException, RetryError
config = Config(
# The API endpoint. You can find it in the API Endpoint section on the Instance Details page.
endpoint="ha-cn-i7*****605.public.ha.aliyuncs.com",
# The username. You can find it in the API Endpoint section on the Instance Details page.
access_user_name="username",
# The password. You can change it in the API Endpoint section on the Instance Details page.
access_pass_word="password")
# Initialize the engine client.
client = Client(config)
def push():
# The name of the table to which you want to push documents. The format is `_`.
tableName = "<instance_id>_<table_name>";
try:
# This is the outer structure for document pushes. It can contain one or more document operations.
documentArrayList = []
# Delete a document.
# To delete a document, you must specify its primary key.
# If the index was built with multi-level hashing, you must specify the primary key for each hash level.
delete2DocumentFields = {
"id": 1 # Primary key ID of the INT single-value type.
}
delete2Document = {
"fields": delete2DocumentFields, # Add the document content to the delete2Document structure.
"cmd": "delete" # The command to delete a document is `delete`.
}
optionsHeaders = {}
documentArrayList.append(delete2Document)
pushDocumentsRequest = models.PushDocumentsRequest(
optionsHeaders, documentArrayList
)
# The primary key field for the documents to be pushed.
pkField = "id"
# Send the request by using the default runtime parameters.
response = client.push_documents(
tableName, pkField, pushDocumentsRequest
)
print(response)
except TeaException as e:
print(f"send request with TeaException : {e}")
except RetryError as e:
print(f"send request with Connection Exception : {e}")
if __name__ == "__main__":
push()Go
package main
import (
"fmt"
"github.com/alibabacloud-go/tea/tea"
ha3engine "github.com/aliyun/alibabacloud-ha3-go-sdk/client"
)
func main() {
// Create a client instance for requests.
config := &ha3engine.Config{
// The API endpoint. You can find it in the API Endpoint section on the Instance Details page.
Endpoint: tea.String("ha-cn-i7*****605.public.ha.aliyuncs.com"),
// The username. You can find it in the API Endpoint section on the Instance Details page.
AccessUserName: tea.String("username"),
// The password. You can change it in the API Endpoint section on the Instance Details page.
AccessPassWord: tea.String("password"),
}
// Initialize a client to send requests.
client, _clientErr := ha3engine.NewClient(config)
// If an error occurs during client initialization, _clientErr is returned and the error message is printed.
if _clientErr != nil {
fmt.Println(_clientErr)
return
}
deleteDoc(client)
}
func deleteDoc(client *ha3engine.Client) {
pushDocumentsRequestModel := &ha3engine.PushDocumentsRequest{}
// The name of the table to which you want to push documents. The format is `_`.
tableName := "<instance_id>_<table_name>"
// The primary key field for the documents to be pushed.
keyField := "<field_pk>"
var array []map[string]interface{}
filed := map[string]interface{}{
"fields": map[string]interface{}{
"id": 2,
},
"cmd": tea.String("delete"),
}
array = append(array, filed)
pushDocumentsRequestModel.SetBody(array)
// Call the method to send the request.
response, _requestErr := client.PushDocuments(tea.String(tableName), tea.String(keyField), pushDocumentsRequestModel)
// If an error occurs while sending the request, _requestErr is returned and the error message is printed.
if _requestErr != nil {
fmt.Println(_requestErr)
return
}
// Print the content of the response.
fmt.Println(response)
}Java in asynchronous mode
import com.aliyun.ha3engine.async.AsyncClient;
import com.aliyun.ha3engine.async.models.PushDocumentsRequest;
import com.aliyun.ha3engine.async.models.PushDocumentsResponse;
import com.aliyun.sdk.ha3engine.async.core.AsyncConfigInfoProvider;
import com.aliyun.tea.TeaException;
import darabonba.core.client.ClientOverrideConfiguration;
import org.junit.Before;
import org.junit.Test;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
/**
* @author alibaba
*/
public class DeleteDoc {
/**
* The OpenSearch Vector Search Edition client.
*/
private AsyncClient client;
@Before
public void clientInit() {
// Configure the username and password for the instance. You can find them in the API Endpoint section on the Instance Details page.
AsyncConfigInfoProvider provider = AsyncConfigInfoProvider.create("username", "password");
// Initialize the asynchronous client.
client = AsyncClient.builder()
.credentialsProvider(provider)
.overrideConfiguration(
ClientOverrideConfiguration.create()
.setEndpointOverride("ha-cn-i7*****605.public.ha.aliyuncs.com")
.setProtocol("http")
).build();
}
@Test
public void delete() throws Exception {
try {
// The name of the table to which you want to push documents. The format is `_`.
String tableName = "<instance_id>_<table_name>";
// The primary key field for the documents to be pushed.
String pkField = "<field_pk>";
// This is the outer structure for document pushes. It can contain one or more document operations.
ArrayList<Map<String, ?>> documents = new ArrayList<>();
// Delete a document.
Map<String, Object> deleteDocument = new HashMap<>();
Map<String, Object> deleteDocumentFields = new HashMap<>();
// Update the document content. Key-value pairs must match.
// The field must be consistent with the pkField parameter.
deleteDocumentFields.put("<field_pk>", "<field_pk_value>");
// Add the document content to the deleteDocument structure.
deleteDocument.put("fields", deleteDocumentFields);
// The command to delete a document is `delete`.
deleteDocument.put("cmd", "delete");
documents.add(deleteDocument);
// Push the data.
PushDocumentsRequest request = PushDocumentsRequest.builder().body(documents).build();
CompletableFuture<PushDocumentsResponse> responseCompletableFuture = client.pushDocuments(tableName, pkField, request);
String responseBody = responseCompletableFuture.get().getBody();
System.out.println("result: " + responseBody);
} catch (ExecutionException | InterruptedException e) {
System.out.println(e.getMessage());
} catch (TeaException e) {
System.out.println(e.getCode());
System.out.println(e.getMessage());
Map<String, Object> exceptionData = e.getData();
System.out.println(com.aliyun.teautil.Common.toJSONString(exceptionData));
}
}
}Additional information
For information about the response to a request, see Update data.
Do not use the
go get github.com/aliyun/alibabacloud-ha3-go-sdkcommand to pull the Git dependency. You must specify a version. This is because the tags for the OpenSearch Vector Search Edition and the OpenSearch Retrieval Engine Edition are in the same GitHub repository. You must pull the dependency that corresponds to your instance version.