SSH node-DataWorks(DataWorks)-阿里云帮助中心

In DataWorks, you can use an SSH node to remotely access a host via a specified SSH data source and trigger scripts to run on it. This topic describes the development workflow for an SSH node.

Introduction

An SSH node enables DataWorks to remotely access a host through an SSH data source. For example, you can remotely access an Elastic Compute Service (ECS) instance from DataWorks and periodically run scripts on it.

Prerequisites

The RAM user that you want to use is added to your workspace.
If you want to use a RAM user to develop tasks, you must add the RAM user to your workspace as a member and assign the Develop or Workspace Administrator role to the RAM user. The Workspace Administrator role has more permissions than necessary. Exercise caution when you assign the Workspace Administrator role. For more information about how to add a member and assign roles to the member, see Add members to a workspace.
A serverless resource group is associated with your workspace. For more information, see the topics in the Use serverless resource groups directory.
You have created an SSH node for development. For more information, see Create a node in a workflow.
You have created an SSH data source.

You must first create an SSH data source to remotely access your SSH server. Then, you can develop and periodically schedule SSH tasks in an SSH node. For more information about how to create a data source, see SSH data source.

Note
SSH nodes support only SSH data sources that are created with a JDBC connection string. To prevent task failures, ensure the data source can connect to the resource group.

Limitations

The maximum code length for an SSH node is 128KB.

Precautions

If an SSH node task that started a remote process exits unexpectedly (for example, due to a timeout), the remote process is not affected. DataWorks does not send a command to the remote host to terminate the process.
SSH nodes support standard shell syntax but do not support interactive features.
When you use an SSH node to run a script on an ECS instance, temporary files are generated on the ECS instance. Ensure the instance has sufficient disk space and can create the necessary files.
Do not run multiple tasks that operate on the same file simultaneously, as this may cause the SSH node to fail.

Step 1: Develop an SSH node

(Optional) Select an SSH data source

If multiple SSH data sources exist in your workspace, select the appropriate data source on the SSH node edit page. If only one exists, DataWorks uses it by default.

Note

SSH nodes support only SSH data sources that are created with a JDBC connection string. To prevent task failures, ensure the data source can connect to the resource group.

Code development: Simple example

Write the script that you want to run in the code editor of the SSH node. The following code provides an example.

# 1. Prepare the environment.
# Find the file that you want to execute on the remote host. For example, a file named hello.sh exists in the /tmp directory of the remote host.
# For ease of testing, you can run the following command in the SSH node to create the hello.sh file.
echo "echo hello,dataworks" >/tmp/hello.sh
# 2. Run the script on the remote host.
# Use the DataWorks SSH node to run the /tmp/hello.sh script.
sh /tmp/hello.sh

Code development: Using scheduling parameters

Use DataWorks Scheduling Parameters to dynamically pass arguments to your code in periodic scheduling scenarios. You can define variables in your task code by using the ${variable_name} format. On the node editing page, navigate to Scheduling Settings > Scheduling Parameters in the right-side panel to assign a value to the variable. For more information about supported formats and configuration details for scheduling parameters, see Configure scheduling properties for a node.

The following example shows how to use scheduling parameters in an SSH node:

# This example uses the ${myDate} variable to write the node's execution time to a log file. To do this, assign the value $[yyyy-mm-dd hh24:mi:ss] to the myDate variable in the node's properties.
echo ${myDate} >/tmp/sshnode.log
cat /tmp/sshnode.log

After you develop the script for the SSH node, you must configure its scheduling properties to run the task periodically. For more information, see Configure scheduling properties for a node.

Step 2: Deploy and manage the node

After you configure the scheduling properties, deploy the SSH node to the production environment. For more information, see Deploy a node or workflow.
After a node is deployed, it runs periodically based on your scheduling configuration. You can go to Operation and Maintenance Center > Node O&M > Auto Triggered Task O&M > Auto Triggered Task to view and manage the periodic task. For more information, see Introduction to Operation Center.