Configure a script node that uses a Database Gateway in DMS-Data Management-阿里云帮助中心

A script node in task orchestration runs a script at a scheduled time or on a periodic basis. This topic describes how to configure a script node that uses a Database Gateway.

Background information

Many tasks exist as scripts on servers. A script node, combined with a Database Gateway, lets you schedule these scripts in a unified manner. For example:

Process data using advanced tools, such as the NumPy and scikit-learn libraries for Python, and the MLlib library for Spark. The resulting business models can be applied to the fine-sort and recommendation systems of a search engine.
Data consumption scenarios, such as running scripts to generate Excel files from data or to send emails that contain data.

Limits

Only Shell script files are supported.

Note

You can also use Shell scripts to schedule other programs, which in turn can schedule their own script files.
A script file name can contain only letters, numbers, underscores (_), and periods (.).

Step 1: Create a Database Gateway

Create a Database Gateway on the device that contains the script file, and then place the script file in the dg_scripts directory.

Note

Each Database Gateway must correspond to a single target device.

For example, to call script code on three Alibaba Cloud ECS instances, you must create three Database Gateways. Do not create three nodes under a single Database Gateway.

Log on to the Database Gateway console.
Click Create Gateway.
In the Create Gateway dialog box, enter a name and description, and then click Next.
Install the Database Gateway on the device that contains the script file.
1. Select Linux/macOS(x86_64). Script code tasks are currently supported only on the Linux operating system. Do not install or start the gateway as the root user.
2. If the target device is an Alibaba Cloud ECS instance, select Access from Alibaba Cloud VPC (Self-hosted Database on ECS/Leased Line/CEN/VPN Gateway).
3. Click Copy Command, paste the command on the device for the gateway installation, and press Enter.
  After the local gateway starts, proceed to the next step.
In the user directory on the Linux device where the gateway is installed, create a directory named dg_scripts. This directory is usually created automatically.
For example, if the current user is xiaoming, run the mkdir dg_scripts command in the /home/xiaoming directory to create the dg_scripts directory.
Move the Shell script file for the task to the dg_scripts directory.
For example, if the script file is named demo.sh, run the mv demo.sh /home/xiaoming/dg_scripts command to move the file to the dg_scripts directory.

Step 2: Configure a script code node

Log in to DMS 5.0.
In the top navigation bar, choose Data+AI > Data+AI > Task Orchestration.

Note
If you use the DMS console in simple mode, move the pointer over the icon in the upper-left corner and choose All Features > Data+AI > Data Development > Task Orchestration.
Click the name of the target task stream to go to its details page.
Note
To add a task stream, see Add a task stream.
Optional: From the Task Type list on the left of the canvas, drag the Script Code node to the canvas.
Double-click the script code node.
Set variables on the script code configuration page according to your business needs. You can click the icon in the upper-right corner of the Variable Settings area to view tips for configuring variables.
- Click the Node Variable tab to configure the variables. For more information, see Configure Time Variables.
- To configure task flow variables, click the Task Flow Variable tab. For more information, see Configure time variables.
- Click the Input Variables tab to view input variables, which include upstream variables, running status, and system variables.
- To use script output variables in descendant nodes, on the Output Variables tab, click Increase Variable.
  In the Variable text box, enter a name for the script output variable. For more information about script output variables, see Variable introduction.

In the Script Configuration area, you can configure the script parameters.

Parameter	Description
Region	Select the region where the target Database Gateway is located.
Gateway ID	Select the name of the target Database Gateway. Note View the gateway name on the Gateway List page in the console.
Gateway Instance ID	Select the gateway node ID of the target Database Gateway. Note View the gateway node ID on the gateway details page of the target gateway.
File name	Enter the name of the script file in the dg_scripts directory of the Database Gateway installation. For example, if the full path of the script file in the Database Gateway is /home/xiaoming/dg_scripts/demo.sh, enter demo.sh.
Runtime Parameter	Select or enter a variable. The variable is used as a runtime parameter in the script. Three types of variables are supported: System variables: A system variable is referenced in the format `${var_name}`, for example, `${sys.flow.start.year}`. For more information about system variables, see System variables. Time variables: The reference format is ${var_name}. To create custom time variables, see Configure time variables. Script output variables: You do not need to enter these variables manually. The script task automatically retrieves them from the script. For more information about script output variables, see Variable Introduction.

Click Try Run.
- If status SUCCEEDED appears in the last line of the execution log, the task ran successfully.
- If status FAILED appears in the last line of the execution log, the task failed.
  Note
  If the test run fails, check the execution log to identify the failed node and the cause of the failure. Then, modify the configuration and run the task again.
After a successful run, you can view the standard output of the script in the execution log.