Prerequisites
-
Sign up for an Alibaba Cloud account and complete real-name registration.
-
When you sign in to the console with your Alibaba Cloud account for the first time, you are prompted to create an access key to proceed.
-
An access key is required to create and use applications. The access key for your Alibaba Cloud account cannot be empty.
-
After you create an access key for your Alibaba Cloud account, you can also create an access key for a RAM user to access the service. To grant the necessary permissions to the RAM user, see Authorization and authentication rules.
-
You must have a VPC. For more information, see the VPC documentation.
To manage a Vector Search Edition instance with a RAM user, grant the AliyunSearchEngineFullAccess and AliyunSearchEngineReadOnlyAccess permissions to the RAM user.
Purchase an instance
-
Go to the OpenSearch console, and in the upper-left corner, switch to OpenSearch - Vector Search Edition.
-
On the Instance Management page of the Vector Search Edition console, click Create Instance.
-
For Product edition, select Vector Search Edition. Select a region. Configure Number of query nodes, Query node specifications, Number of data nodes, Data node specifications, and Total storage per data node. Set the VPC and vSwitch. Finally, set a username and password for query authentication (these are separate from your Alibaba Cloud account credentials), and then click Buy Now.
-
Choose the number and specifications of query and data nodes based on your business needs. After you select the specifications, the cost is automatically calculated on the pricing page.
-
The VPC and vSwitch must match those of the ECS instance that will access your Vector Search Edition instance. Otherwise, you will receive a
{'errors':{'code':'403','message':'Forbidden'}}error when you try to access the instance. -
Each data node includes a free storage quota. You can purchase additional storage, which is billed in 50 GB increments.
-
On the Confirm Order page, review the service agreement. If everything is correct, click Activate Now.
-
After the purchase is successful, click Management Console to view your new Vector Search Edition instance on the Instance Management page.
-
The system creates a new instance with a default name. To change the name, click Manage in the Actions column to open the instance details page.
Click the edit icon, change the instance name in the dialog box, and click Confirm.
Configure the cluster
A newly purchased instance has a status of "Pending Configuration" on its details page. The system automatically deploys an empty cluster with the same number and specifications of query and data nodes that you purchased. To enable search, you must configure the cluster in the following order: Configure data source -> Configure index -> Rebuild index.
-
Configure a data source. Supported data sources include MaxCompute data source and API data source. This example uses a MaxCompute data source. Click Add Data Source, select MaxCompute as the Data source type, and then set the
project,accessKeyId,accessKeySecret,Table, andpartitionparameters. You can also enable automatic full indexingConfigure automatic reindexing.
After validation, click OK to add the data source.
-
After you configure the data source, click Next to configure the index schema.
2.1. Add an index table: On the Index schema page, click Add Index Table.
2.2. Configure the index table:
-
Index table: Enter a custom name.
-
Data source: Select the data source that you configured in the previous step.
-
Data shards: Configure the number of shards based on the number of data nodes you purchased.
2.3. Configure fields:
The fields table includes the following columns: Field name, Field type, Primary key, Attribute, Attribute compression, Summary compression, Summary range, analyzer, and Advanced configuration. Configure the properties for each field as needed.
Attribute and field content compression:
-
For attribute fields, compression is optional and disabled by default. Select
file_compressorto enable compression. -
For field content, compression is optional and disabled by default. By default, multi-value and STRING type fields use
uniq, and single-value numeric fields useequal.
-
If you enable attribute compression, go to Deployment Management > Data Node > Online Table Configuration and edit the index loading method to mitigate any performance impact.
-
You can duplicate a field. The new field, known as the DUP field, initially shares the same content as the original. If you attempt to index different content into each field, the original field's content overwrites the DUP field's content. To index different content into the DUP field, you must manually remove the 'copy from' setting in its advanced configuration.
2.4. Configure indexes:
The index settings table includes the following columns: Index name, Index type, Included fields, Data compression, Advanced configuration, and Actions. You can view and modify the settings for each index.
-
For index fields, compression is optional and disabled by default. Select
file_compressorto enable compression.
-
Primary key indexes do not support compression.
-
If you enable index compression, go to Deployment Management > Data Node > Online Table Configuration and edit the index loading method to mitigate any performance impact.
-
After you complete the configuration, click Save Version. In the subsequent dialog box, add any remarks and then click Publish.
After the index is published, click Next to start an index rebuild.
-
To rebuild the index, select the required parameters and click Next.
-
API data source:
Select the Data source name and Associated index table. Confirm that the Data source type is API data source, and then click Next.
-
MaxCompute data source:
Select the Data source name and Associated index table. Confirm that the Data source type is MaxCompute. Configure the Data partition and Timestamp parameters, and then click Next.
Check index rebuild progress
After you configure the data source and index schema, you can view the cluster topology in O&M Center > Deployment Management.
The topology diagram shows the hierarchical relationship between Query Access, the cluster (including data nodes and query nodes), the index table, and the data source.
After the configuration is updated, you can track the index rebuild progress in O&M Center > Change History > Data Source Change.
The index rebuild process consists of five stages: init, trigger, scan, build, and switch. A green check mark appears next to each stage as it is completed.
On the instance details page, you can also check the status of the cluster's query nodes and data nodes. You can start running search tests after all statuses are normal.
Query nodes display Service Status (In Service) and Configuration Status (Configuration Updated). Data nodes additionally display Data Status (Data Updated). A normal state is indicated when all three statuses are at 100%.
Search test
You can run simple search tests in Extensions > Search Test. The Vector Search Edition supports two query methods: HA3 clause and SQL clause.
Select a query method, such as SQL clause. Enter a query in the input box, for example, select * from index_a, and click Search. The results include information such as the time taken (USE_TIME) and the number of matched rows (ROW_COUNT).
-
[Documentation] HA3 syntax
-
[Documentation] SQL syntax
Usage notes
-
Currently, only the pay-as-you-go billing method is supported.
-
You can change the username and password that you set during the instance purchase on the instance details page.
-
The system assigns a cluster name when you purchase the instance. This name currently cannot be changed.
-
Index rebuilds are billed based on the resources consumed during the process. For more information, see billing overview.