Demo code for pushing data

更新时间:
复制 MD 格式

This topic describes how to set environment variables and import dependencies. It also provides sample code for pushing documents using the Python SDK.

Configure environment variables

Configure the ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variables.

Important
  • The AccessKey pair of an Alibaba Cloud account can be used to access all API operations. We recommend that you use a Resource Access Management (RAM) user to call API operations or perform routine O&M. For information about how to use a RAM user, see Create a RAM user.

  • For information about how to create an AccessKey pair, see Create an AccessKey pair.

  • If you use the AccessKey pair of a RAM user, make sure that the required permissions are granted to the AliyunServiceRoleForOpenSearch role by using your Alibaba Cloud account. For more information, see AliyunServiceRoleForOpenSearch and Access authorization rules.

  • We recommend that you do not include your AccessKey pair in materials that are easily accessible to others, such as the project code. Otherwise, your AccessKey pair may be leaked and resources in your account become insecure.

  • Linux and macOS

    Run the following commands. Replace <access_key_id> and <access_key_secret> with the AccessKey ID and AccessKey secret of the RAM user that you use.

    export ALIBABA_CLOUD_ACCESS_KEY_ID=<access_key_id> 
    export ALIBABA_CLOUD_ACCESS_KEY_SECRET=<access_key_secret>
  • Windows

    1. Create an environment variable file, add the ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variables to the file, and then set the environment variables to your AccessKey ID and AccessKey secret.

    2. Restart Windows for the AccessKey pair to take effect.

Import dependencies

pip install alibabacloud_tea_util 
pip install alibabacloud_opensearch_util
pip install alibabacloud_credentials

For more information about BaseRequest, see Python client examples.

Push example

# -*- coding: utf-8 -*-
import time,os
from typing import Dict, Any
from Tea.exceptions import TeaException
from Tea.request import TeaRequest
from alibabacloud_tea_util import models as util_models
from BaseRequest import Config, Client
class opensearch:
    def __init__(self, config: Config):
        self.Clients = Client(config=config)
        self.runtime = util_models.RuntimeOptions(
            connect_timeout=10000,
            read_timeout=10000,
            autoretry=False,
            ignore_ssl=False,
            max_idle_conns=50,
            max_attempts=3
        )
        self.header = {}
    def docBulk(self, app_name: str, table_name: str, doc_content: list) -> Dict[str, Any]:
        try:
            response = self.Clients._request(method="POST",
                                             pathname=f'/v3/openapi/apps/{app_name}/{table_name}/actions/bulk',query={},headers = self.header,
                                             body=doc_content, runtime=self.runtime)
            return response
        except Exception as e:
            print(e)
if __name__ == "__main__":
    # Configure a unified request entry point. Note: Remove http:// from the host.
    endpoint = "<endpoint>"
    # The protocol can be set to HTTPS or HTTP.
    endpoint_protocol = "HTTP"
    # User identity information
    # Read the AccessKey ID and AccessKey secret from environment variables.
    # Before you run the sample code, you must configure environment variables. For more information, see the "Configure environment variables" section in this topic.
    access_key_id = os.environ.get("ALIBABA_CLOUD_ACCESS_KEY_ID")
    access_key_secret = os.environ.get("ALIBABA_CLOUD_ACCESS_KEY_SECRET")
    # The type parameter specifies the authentication method, which can be sts or access_key. The default value is access_key. Set type to sts to use RAM-STS for authentication.
    # Valid values: sts and access_key.
    auth_type = "sts"
    # If you use RAM-STS for authentication, configure the security_token parameter. You can call the AssumeRole operation of Alibaba Cloud to obtain the STS authentication structure.
    security_token = "<security_token>"
    # Configure common request information.
    # Note: If you are not using a RAM user, omit the security_token and type parameters.
    Configs = Config(endpoint=endpoint, access_key_id=access_key_id, access_key_secret=access_key_secret,
                     security_token=security_token, type=auth_type, protocol=endpoint_protocol)
    # Create an OpenSearch instance.
    ops = opensearch(Configs)
    app_name = "app_name"
    table_name = "table_name"
    
  # ---------------  Push documents ---------------
    
    # The timestamp information ensures the sequential processing of document operations. The system uses the timestamp to determine the update order for documents with the same primary key.
    # If the timestamp is not specified, the system uses the time when the document is sent to OpenSearch as the update time.
    document1 = {"cmd": "ADD", "timestamp": int(time.time() * 1000), "fields": {"id": "1", "title": "opensearch"}}
    document2 = {"cmd": "ADD", "fields": {"id": 2, "describe": "123456"}}
    # Delete a record.
    deletedoc={"cmd": "DELETE", "fields": {"id": 2}}
    # Update a record.
    updatedoc={"cmd": "UPDATE", "fields": {"id": 2, "describe": "6666","title": "OpenSearch"}}
    documents = [document1,document2]
    res5 = ops.docBulk(app_name=app_name, table_name=table_name, doc_content=documents)
    print(res5)
Note

For more information, see Data processing.