This topic describes how to create a Lindorm (Compute Engine) data source, which allows Dataphin to read data from and write data to Lindorm (Compute Engine).
The Lindorm (Compute Engine) data source is deprecated. You can no longer create new data sources of this type. Instead, use the Lindorm data source (Compute Engine-ETL resource group (Spark) connection type). For more information, see . Any Lindorm (Compute Engine) data sources created before version 6.1 remain functional.
Background
Lindorm (Compute Engine) is a proprietary database service from Alibaba Cloud. It provides various data models, including wide table, time series, file, and search, to support millisecond-level online data processing, as well as low-cost storage and analysis for massive datasets. For more information about Alibaba Cloud Lindorm (Compute Engine), see Cloud Native Multimodel Database Lindorm (Compute Engine).
Permissions
You can create a data source if you have a custom global role with the Create data source permission, or if you have a system role such as super administrator, data source administrator, business segment architect, or project administrator.
Procedure
On the Dataphin homepage, click Administration Center > Data Source Management in the top menu bar.
On the Data Source page, click + New Data Source.
On the New Data Source page, in the Big Data Storage section, select Lindorm (Compute Engine).
If you have used this data source type recently, you can also select Lindorm (Compute Engine) from the Recently Used section. Alternatively, enter Lindorm (Compute Engine) in the search box to find it quickly.
On the Create Lindorm (Compute Engine) Data Source page, configure the connection parameters.
Configure the basic information for the data source.
Parameter
Description
Data source name
Enter a name for the data source. The name must meet the following requirements:
Can contain only Chinese characters, uppercase or lowercase letters, digits, underscores (_), and hyphens (-).
Cannot exceed 64 characters in length.
Data source code
After you configure the data source code, you can directly access Dataphin data source tables in Flink SQL jobs or by using the Dataphin JDBC client. Use the format
data_source_code.table_nameordata_source_code.schema.table_name. To automatically switch data sources based on the job execution environment, use the variable format${data_source_code}.tableor${data_source_code}.schema.table. For more information, see and Develop Flink SQL jobs.ImportantThe data source code cannot be modified after it is configured.
You can preview data on the object details page in the asset directory and asset checklist only after the data source code is configured.
In Flink SQL, only MySQL, Hologres, MaxCompute, Oracle, StarRocks, Hive, SelectDB, and GaussDB data warehouse service (DWS) data sources are currently supported.
Data source description
Enter a brief description of the data source. The description cannot exceed 128 characters.
Data source configuration
Select the data source to configure:
If your business data source distinguishes between production and development data sources, select Production + Development Data Source.
If your business data source does not distinguish between production and development data sources, select Production Data Source.
Tag
You can use tags to categorize your data sources. For more information about how to create a tag, see or Manage Data Source Tags.
Configure the connection parameters.
If you select Production + Development Data Source for Data source configuration, you must configure connection information for both the production data source and the development data source. If you select Production Data Source, you only need to configure connection information for the production data source.
NoteTypically, the production and development data sources should be configured as separate instances to ensure environment isolation and reduce the impact of development activities on the production environment. However, Dataphin allows you to use the same instance for both if you provide identical parameter values.
Parameter
Description
core-site.xml
Upload the core-site.xml configuration file for Lindorm (Compute Engine).
hdfs-site.xml
Upload the hdfs-site.xml configuration file for Lindorm (Compute Engine).
hive-site.xml
Upload the hive-site.xml configuration file for Lindorm (Compute Engine).
JDBC URL
Enter the JDBC URL for Lindorm (Compute Engine). The format is
jdbc:hive2://host:port/;?token=xxx. For more information, see View Connection String.
Select a Default Resource Group. This resource group is used to run tasks related to this data source, such as database SQL execution, offline full-database migration, and data preview.
Click Test Connection or click OK to save the configuration and create the Lindorm (Compute Engine) data source.
Click Test Connection to verify that Dataphin can connect to the data source. If you click OK directly, the system automatically tests the connection to all selected clusters. You can create the data source even if these connection tests fail.
ImportantIf the connection test fails, you can troubleshoot the issue based on common network connectivity problems. For more information, see Network Connectivity Solutions.