Assignment node: Pass parameters between nodes-DataWorks(DataWorks)-阿里云帮助中心

Usage notes

Version requirements: Available only in DataWorks Standard Edition and later.
Permissions: Your RAM account must be added to the target workspace and assigned the developer or workspace administrator role. For more information, see Add members to a workspace.

Core concepts: Parameter passing and referencing

The core function of an assignment node is parameter passing, which transfers data produced by an upstream node to downstream nodes.

Upstream assignment node: Produces data. It automatically assigns the last output or query result to a node output parameter named outputs.
Downstream business node: Receives and uses data. By adding a node input parameter (for example, param) to the node configuration and mapping it to the outputs parameter of the upstream assignment node, you can reference the data in your code.

Output format by language

The output format of the assignment node varies by language:

Language	Output source	Output format
ODPS SQL	Output of the last `SELECT` statement.	The output is passed as a two-dimensional array to downstream nodes.
Python 2	Output of the last `print` statement.	The output is converted to a string and split by comma (`,`) into a one-dimensional array. For example, if the last line of the assignment node outputs `'Electronics,Clothing,Books'`, the value passed to downstream nodes is `['Electronics','Clothing','Books']`.
Shell	Output of the last `echo` statement.

Procedure

The following example passes the output of an ODPS SQL assignment node to a Shell node. In practice, any node can serve as the downstream node.

Configure the upstream assignment node

In a workflow, create and edit an assignment node. On the code editor page, select ODPS SQL, Python, or Shell as needed, and write the code that produces the result to pass downstream.
Configure the downstream Shell node

Create a Shell node. On the Shell node editor page, reference the upstream result:
1. On the Shell node editor page, click Scheduling configuration on the right and select the Node context tab.
2. In the Input parameters section, click Add.
3. In the dialog box that appears, select the outputs parameter of the upstream assignment node and specify a parameter name for the current node's input parameter (for example, param).
  
  After the configuration is complete, the downstream node automatically establishes a dependency on the upstream assignment node.
4. After the parameter configuration is complete, reference the value passed from the upstream node in the downstream Shell node code by using the ${param} format.
Run and verify

Double-click the workflow name. On the workflow editor page, click the Run icon in the toolbar to run the workflow and check whether the referenced result is correct. You can also commit the node to the production environment, go to Operation Center, and perform a smoke test to validate the execution.

Note

DataWorks provides an ETL workflow template (Assignment Node Application) for you to quickly explore the capabilities of assignment nodes. For more information, see ETL workflow quick start.

Limitations

Scope: Assignment node parameters can only be passed to immediate downstream nodes. Cross-level passing is not supported.
Size limit: The output value cannot exceed 2 MB. If the output exceeds this limit, the assignment node fails.
Syntax restrictions:
- Comments are not supported in assignment node code. Adding comments may cause abnormal execution results.
- The WITH clause is not supported in ODPS SQL mode.

Examples by language

The output format (outputs) and the reference method for downstream nodes vary by language.

Example 1: Pass an ODPS SQL query result

SQL query results are passed to downstream nodes as a two-dimensional array.

Upstream node (assignment node - SQL) configuration
Assume the SQL code is as follows and the query returns two rows and two columns:
```
SELECT 'beijing', '1001'
UNION ALL 
SELECT 'hangzhou', '1002';
```

Downstream node (Shell node) configuration and output

In the Shell node, add an input parameter named region and reference the outputs parameter of the upstream SQL node.

Write the following code to read the data:

echo "Entire result set: ${region}"
echo "First row: ${region[0]}"
echo "First row, second field: ${region[0][1]}"

DataWorks directly parses the parameter and performs static substitution. The output is as follows:

Entire result set: beijing,1001
hangzhou,1002
First row: beijing,1001
First row, second field: 1001

Example 2: Pass Python 2 output results

The output of the Python 2 print statement is split by commas (,) and passed to downstream nodes as a one-dimensional array.

Upstream node (assignment node - Python 2) configuration
The Python 2 code is as follows:
```
print 'Electronics, Clothing, Books';
```