Create a Lindorm data source

Permissions

Only custom global roles with the Add Data Source permission and system roles such as super admin, data source admin, workspace architect, and project admin can create data sources.

Procedure

1. In the top navigation bar of the Dataphin homepage, choose Management Center > Data Source Management.

2. On the Data Source page, click + Add Data Source.

3. On the Add Data Source page, in the NoSQL section, select Lindorm.

   If you have recently used Lindorm, you can also select it from the Recently Used section. Or, you can enter "Lindorm" in the search box to find it quickly.

4. On the New Lindorm Data Source page, configure the connection parameters for the data source.

   1. Configure basic information

      Parameter	Description
      data source name	Enter a name for the data source. The name must meet the following requirements: It can contain only Chinese characters, uppercase and lowercase letters, digits, underscores (_), and hyphens (-). It cannot exceed 64 characters in length.
      data source ID	After you configure the data source ID, you can directly access Dataphin data source tables in Flink SQL tasks or by using a Dataphin JDBC client. Use the format dataSourceID.tableName or dataSourceID.schema.tableName. To automatically switch data sources based on the task's execution environment, use the variable format ${dataSourceID}.table or ${dataSourceID}.schema.table. For more information, see Flink SQL Task Development Method. Important The data source code cannot be modified after it is configured. You can preview data on the object details page in the asset directory and asset checklist only after the data source code is configured. In Flink SQL, only MySQL, Hologres, MaxCompute, Oracle, StarRocks, Hive, SelectDB, and GaussDB data warehouse service (DWS) data sources are currently supported.
      data source description	Enter a brief description of the data source. The description cannot exceed 128 characters.
      Data source configuration	Select the data source to configure: If your business data source distinguishes between production and development data sources, select Production + Development Data Source. If your business data source does not distinguish between production and development data sources, select Production Data Source.
      tag	You can add tags to categorize the data source. To create a tag, see Manage Data Source Tags.

   2. Configure connection parameters

      If you selected Production + Development Data Source, you must configure connection information for both the production and development data sources. If you selected Production Data Source, you only need to configure the connection information for the production data source. The required parameters vary based on the Connection Type, as detailed in the following sections.

      Note

      Typically, you should configure the production and development data sources as separate instances to isolate environments and prevent development activities from impacting production. However, Dataphin allows you to use the same instance for both by entering the same parameter values.

      ETL resource group (Spark)

      Parameter	Description
      core-site.xml	Upload the core-site.xml, hdfs-site.xml, and hive-site.xml configuration files from your Lindorm instance.
      hdfs-site.xml
      hive-site.xml
      JDBC URL	Enter the JDBC URL for Lindorm. The format is jdbc:hive2://host:port/;?token=xxx. For more information, see View Connection String.
      Username, password	The username and password to access the Lindorm instance.

      OLAP / Wide-column / Column-store (MySQL)

      Parameter	Description
      JDBC URL	Enter the JDBC URL for Lindorm, for example, jdbc:mysql://host:port.
      Username, password	The username and password to access the Lindorm instance.

      Wide-column (HBase)

      Parameter	Description
      endpoint	The connection endpoint for Lindorm. To obtain the endpoint, see the "Connect to and use an instance" topic in the Lindorm (Wide Table) documentation.
      namespace	The namespace in Lindorm. The default value is default.
      Username, password	The username and password to access the Lindorm instance.
      Connection parameters (Optional)	Additional parameters for connecting to Lindorm. Configure these parameters in JSON format.

      Graph engine

      Parameter	Description
      endpoint	The connection endpoint for the Lindorm graph engine. The format is server_host:gremlin_port.
      DB	The name of the database (subgraph). The default value is default.
      Username, password	The username and password for authentication.

5. Select a Default Resource Group. This resource group runs tasks related to the data source, such as database SQL queries, offline full-database migration, and data preview.

6. Click test connection to verify connectivity, or click OK to save the configuration and create the Lindorm data source.

   When you click test connection, Dataphin verifies the connection to the data source. If you click OK directly, the system automatically tests the connection for all selected clusters. The data source can be created even if the connection tests fail for all selected clusters.

   Important

   If the connection test fails, refer to the troubleshooting guide for common network connectivity issues. For more information, see Network Connectivity Solution.