Data integration
Data Integration for ApsaraDB for SelectDB is a visual tool for importing external data into an instance database. This topic describes how to use Data Integration with ApsaraDB for SelectDB.
Prerequisites
You have created an ApsaraDB for SelectDB instance. For more information, see Create an instance.
The instance version is 3.0.7 or later.
Procedure
Log on to the Alibaba Cloud SelectDB console.
In the upper-left corner of the page, select the region where your instance is located.
On the Instances page, click the target Instance ID to open the Instance Details page.
In the left-side navigation pane, click Data Development and Management (Studio) > Data Integration.
NoteWhen you first use Data Development and Management (Studio), a prompt asks you to add your local public IP address to the webui_whitelist whitelist group. Review the prompt and click OK.
When you use Data Integration for the first time, if you are not already logged on to the WebUI, the system redirects you to the logon page.
You can use the admin account to log on to the WebUI.
If you forget the password for the admin account, you can reset it. For more information, see Reset account password.
On the Integration page, you can perform the following operations.
If you have not yet created a data integration task, the Stage Integration page appears, where you can select a task type and create a new task.
Create an integration task.
Sample data
Sample data is benchmark data used to test the performance of data analytics databases. You can create new sample data by following the steps below.
Select a sample data type.
Click New in the upper-right corner to go to the Stage Integration page, and then select a sample data type.
Sample data
Description
ClickBench
ClickBench dataset.
TPC-H
TPC-H dataset.
Github Demo
GitHub Event dataset.
SSB-FLAT
SSB-FLAT dataset.
On the New Integration page, specify the following parameters and click Confirm and Import.
Parameter
Description
Example
Integration name
The name of the Data Integration task.
test
Description
The comment for the Data Integration task.
test comment
Cluster
The cluster used to run the Data Integration task.
new_cluster
Sample data size
The size of the sample data.
1 GB
Object Storage Service (OSS)
Select an integration type.
Click New in the upper-right corner to go to the Stage Integration page, and then click Object Storage Service (OSS).
Configure parameters.
On the New Integration page, specify the following parameters and click OK.
Parameter
Description
Example
Integration name
The name of the Data Integration task.
test
Description
The comment for the Data Integration task.
test comment
Bucket
The name of the OSS bucket.
test_bucket_name
Default file path
The default path of files in OSS.
None
Authorization
The authentication method to access OSS.
Access Key
Access Key
The Access Key of your Alibaba Cloud account.
akdemo
Secret Key
The Secret Key of your Alibaba Cloud account.
skdemo
Advanced
The default properties for file integration and file import.
None
File settings
The properties of the files to integrate.
None
File Type
The file type of objects in OSS.
Valid values: JSON, ORC, CSV, Parquet, and Auto.
JSON
Compression type
The compression method of files in OSS.
gz
Column separator
The column separator for data in the OSS files.
\t
Line delimiter
The line delimiter for data in the OSS files.
\n
File size
The file size limit for files to be integrated from OSS.
Unlimited
Import settings
The default operation for file import.
None
On error
Continue: The data import continues when an error occurs.
Stop: The data import stops if an error occurs.
Customized: A custom policy is used to handle errors.
Stop
Strict mode
Open: This mode filters out rows that cause errors during column type conversion. Error data in this context refers to:
Source data that is not null but becomes null after type conversion. This excludes null values generated by functions.
Data that is converted correctly but falls outside the destination column's value range. strict mode does not filter rows for this type of range error.
Close: This mode does not filter out rows that cause errors during column type conversion.
Enabled
Search for an integration task: On the Integration page, click the search icon in the upper-right corner. Then enter the integration task name in the search box.
Delete an integration task: On the Integration page, find the integration task in the task list and click the delete icon in the Actions column.
NoteDeleting an integration task does not affect previously imported data, but it may affect in-progress imports.
You cannot restore a deleted integration task.