Solution 4: Network connectivity for data sources deployed on ECS

This solution is recommended if your data source meets the following condition:

• The data source is deployed on an Alibaba Cloud ECS instance.

Same account and region

If the ECS instance hosting the data source and DataWorks are under the same account and in the same region, we recommend that you use a VPC (private network) connection. Deploy the resource group of your DataWorks workspace and the ECS instance in the same VPC to enable network connectivity.

Different accounts or regions

If the ECS instance hosting the data source and DataWorks are under different accounts or under the same account but in different regions, we recommend that you use a VPC (private network) connection. Use a network connectivity tool (Cloud Enterprise Network or VPC Peering Connection) to connect the VPC of your DataWorks workspace resource group with the VPC where the ECS instance resides.

Step 1: Obtain basic information

Step 2: Establish network connectivity

Step 3: Add a route in the DataWorks resource group

For cross-region scenarios under the same account or cross-account scenarios, you also need to add a route in the DataWorks resource group that points to the vSwitch CIDR block of the ECS instance.

1. Go to the DataWorks Resource Group List page. Find the target resource group and click Network Settings in the Operation column.

2. Under the corresponding module, find the bound VPC and click Custom Route in the Operation column.

3. Click Add Route. Set the connection method to CIDR Block, and set Destination CIDR Block to the vSwitch CIDR block of the ECS instance.

Step 4: (Optional) Enable remote access for the database

Some databases require remote access to be enabled in their configuration files to allow specified users to access the database from external networks by using an IP address and port. The configuration method varies by database. Refer to the official documentation for your database.

For example: Enable remote access for MySQL.

Step 5: Configure ECS security group rules

Alibaba Cloud ECS uses security groups to implement firewall capabilities. You need to open the database port in the ECS security group for the VPC of the DataWorks resource group, so that the DataWorks resource group can access the service deployed on the ECS instance.

1. Go to the ECS console and switch to the region of your ECS instance at the top of the page.

2. In the left-side navigation pane, click Instances & Images > Instance. Find the ECS instance on which the MySQL database is deployed, and click the instance name to go to the Instance Details page.

3. At the top of the page, switch to the Security Group tab. Click the security group name to go to the Security Group Details page.

4. In the Access Rules section, click Add Rule and configure the following key parameters. You can use the default values for other parameters.

   • Source: Enter the vSwitch CIDR block bound to the DataWorks resource group.

   • Port: Select the port of the database deployed on the ECS instance. For example, open port 3306 for MySQL.

1. Basic information

2. Establish network connectivity

This example uses a VPC peering connection to configure network connectivity between the ECS instance and DataWorks.

1. Log on with Account A. Go to the VPC Peering Connection console. Switch the region to China (Hangzhou) at the top of the page, and then click Create Peering Connection to configure the parameters.

   The following table describes the key parameters for this example. You can use the default values for other parameters.

   Parameter	Description and example
   Peering Connection Name	Specify a custom name. In this example, enter Account_A to Account_B.
   Requester VPC	The VPC of the ECS instance under Account A. In this example, select Account_A_hangzhou_VPC.
   Accepter Account Type	In this example, select Cross-account.
   Accepter Account UID	Enter the UID of the Alibaba Cloud account for Account B.
   Accepter Region Type	In this example, select Cross-region.
   Accepter Region	The region of the DataWorks workspace and resource group under Account B. Select China (Shanghai).
   Accepter VPC	Manually enter the VPC ID of the VPC (Account_B_shanghai_VPC) that contains the DataWorks resource group under Account B.

2. Click Determine to complete the peering connection configuration. The system automatically redirects to the peering connection details page, where the Status is Accepting.

3. Log on with Account B. Go to the VPC Peering Connection console. Switch the region to China (Shanghai) at the top of the page. You can see a peering connection record that matches the one created under Account A. Click Receiver in the Operation column. After acceptance, the Status changes to Activated.

4. Click Configure Route Entry under Accepter VPC. In the Configure Route Entry dialog, specify a custom Name for the route entry and set Destination CIDR Block to the VPC CIDR block of the requester (ECS). In this example, enter 192.168.0.0/16.

5. Log on with Account A. Go to the VPC Peering Connection console. Switch the region to China (Hangzhou) at the top of the page and find the peering connection that you created.

6. Click Configure Route Entry under Requester VPC. In the Configure Route Entry dialog, specify a custom Name for the route entry and set Destination CIDR Block to the VPC CIDR block of the accepter (DataWorks resource group). In this example, enter 172.16.0.0/12.

3. Add a route in the DataWorks resource group

1. Log on with Account B. Go to the DataWorks Resource Group List page. Find the target resource group and click Network Settings in the Operation column.

2. Under the corresponding module, find the bound VPC and click Custom Route in the Operation column.

3. Click Add Route. Set the connection method to CIDR Block, and set Destination CIDR Block to the vSwitch CIDR block of the ECS instance (in this example, 192.168.6.0/24).

4. Enable remote access for the MySQL database

Connect to the ECS instance on which the MySQL database is deployed and enable remote access for the database.

1. Locate the my.cnf configuration file. By default, the file is located at /etc/my.cnf.

   find / -name my.cnf

2. Run the vim /etc/my.cnf command to edit the configuration file. Replace the my.cnf path based on the actual result from the previous step.

3. At the end of the configuration file, press i and add the following configuration under the [mysqld] section:

   bind-address=0.0.0.0

4. Press Esc, type :wq!, and then press Enter to save and exit.

5. Run the systemctl restart mysqld command to restart the service.

6. Create a user that will be used when configuring the data source in DataWorks to remotely connect to the MySQL database.

   1. Run the mysql -u root -p command to log on to the database as the administrator.

   2. Create a user and set a password.

      -- "dataworks_user" is the username, which you can customize.
      -- "%" means access is allowed from any IP. You can also specify an IP for fine-grained control.
      -- "StrongPassword123!" is the user password, which you can customize.
      CREATE USER 'dataworks_user'@'%' IDENTIFIED BY 'StrongPassword123!';

   3. Grant database permissions to the user.

      -- Execute any one of the following commands.
      
      -- Grant all privileges to the user (use with caution).
      GRANT ALL PRIVILEGES ON *.* TO 'dataworks_user'@'%' WITH GRANT OPTION;
      
      -- Grant privileges on a specific database (e.g., mydatabase) to the user.
      GRANT ALL PRIVILEGES ON mydatabase.* TO 'dataworks_user'@'%' WITH GRANT OPTION;

   4. Run the FLUSH PRIVILEGES; command to flush the privileges, and then exit the database (exit).

   5. Verify the remote connection.

      mysql -u dataworks_user -h <Primary private IP address of the ECS instance> -p

5. Configure ECS security group rules

1. Log on with Account A. Go to the ECS console and switch to China (Hangzhou) at the top of the page.

2. In the left-side navigation pane, click Instances & Images > Instance. Find the ECS instance on which the MySQL database is deployed, and click the instance name to go to the Instance Details page.

3. At the top of the page, switch to the Security Group tab. Click the security group name to go to the Security Group Details page.

4. In the Access Rules section, click Add Rule and configure the following key parameters. You can use the default values for other parameters.

   • Source: Enter the vSwitch CIDR block of the DataWorks resource group (in this example, 172.16.66.0/24).

   • Port: Select the port of the database deployed on the ECS instance (in this example, 3306).

6. Test connectivity

1. Log on with Account B.

2. Log on to the DataWorks console. In the target region, click Data Integration > Data Integration in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Integration.

3. In the left-side navigation pane, click Data Sources to go to the Data Sources page. Click Add Connection.

4. Select the MySQL data source type and configure the connection parameters.

   • Set Configuration Mode to User-created Data Store with Public IP Addresses.

   • Set Host Address to the private IP address of the ECS instance (in this example, 192.168.6.172).

   • Set Port Number to 3306.

   • Set Database Name to an existing database name.

   • Set Username and Password to the dataworks_user credentials created in the Enable remote access for MySQL step.

5. In the Connection Configuration section, click the test connectivity button next to the resource group bound to the workspace to check whether the result is Connectable.

   In the Connection Configuration section of the data source, select a resource group and click Test Connectivity. Verify that the connectivity status is Connectable.

   Note

   If the test result shows Failed, use the Network Connectivity Diagnostic Tool to troubleshoot. If connectivity still fails, submit a ticket for assistance.