Create table statements for MaxCompute data sources

Table creation statements

For this example, the data table is named test_maxcompute_table and its corresponding done table is test_maxcompute_table_done.

#1. Create a data table.
CREATE TABLE IF NOT EXISTS test_maxcompute_table
(
  pk_id Bigint ,
  cate_id Bigint ,
  vector_array string 
)
PARTITIONED by 
(
  ds STRING
);
#2. Create a partition in the data table.
alter table test_maxcompute_table  add if not exists partition (ds="20230423");
#3. Create a done table.
create table test_maxcompute_table_done (attribute string) partitioned by (ds string);
#4. Create a partition in the done table.
alter table test_maxcompute_table_done  add if not exists partition (ds="20230423");
#5. Insert data into the data table.
insert into test_maxcompute_table partition (ds='20230423') select 1,'12345','xxx,xxx,xxx,xx....';
#6. Insert data into the done table.
insert into table test_maxcompute_table_done partition (ds="20230423") select '{"swift_start_timestamp":1682179200}';

Authorization statements

When configuring a MaxCompute data table, your OpenSearch account must have the DESCRIBE, SELECT, and DOWNLOAD permissions on the table, and the LABEL permission on its fields.

-- Add the user account for the engine instance.
add user ****@aliyun.com;
-- Grant the required table permissions to the account.
GRANT describe,select,download ON TABLE table_xxx TO USER ****@aliyun.com
GRANT describe,select,download ON TABLE table_xxx_done TO USER ****@aliyun.com
-- If field-level permission verification is enabled in MaxCompute, data pull may fail for fields with high-level security labels, which prevents index creation. To resolve this, you must grant the account field-level access permissions.
-- Grant permissions on the entire project.
SET LABEL 3 to USER ****@aliyun.com
-- Grant permissions on a single table.
GRANT LABEL 3 ON TABLE table_xxx(col1, col2) TO ****@aliyun.com

MaxCompute data source parameters

Navigate to Data Source Configuration and click Add Data Source. On the configuration page, specify the following parameters. By default, automatic index rebuilding is set to Yes.

Parameter	Required	Description
Data source type	Yes	Select MaxCompute.
Data source name	Yes	Up to 30 characters, containing only letters, digits, and underscores (_).
Project	Yes	The project name can be viewed on the MaxCompute Data Development page. In DataStudio, the current workspace name is displayed in the Workspace drop-down list in the top navigation bar. Note To check if your project is in standard mode, go to the Table Management page in DataStudio and search for the target table. A project is in standard mode if the standard mode tag is displayed next to the table name. For projects in standard mode, you must append _dev to the project name. For example, zy_ts_test becomes zy_ts_test_dev.
AccessKey ID	Yes	To find your AccessKey, hover over your profile avatar in the upper-right corner of the console and click AccessKey Management.
AccessKey Secret	Yes
Table	Yes	The name of the data table.
Partition Key	Yes	The partition key of the data table. For example: ds=20230423. In this example, ds is the partition key.