DocsICP FilingConsole
Documentation
Home Page DataWorks User Guide Data Studio (new version) Node development Node types General nodes Set Variable node

Set Variable node

Updated at:
Copy as MD
Product
My Favorites

A Set Variable node is used to overwrite the values of scheduling variables during a workflow run. An upstream node writes its computation results to a scheduling variable, and any subsequent node in the workflow (including indirect downstream nodes) can read the new value without passing output parameters layer by layer between nodes.

Note

The Set Variable node is a general-purpose node in the new Data Studio and can only be used within workflows.

The Set Variable node differs from the assignment node: the assignment node passes values through node context parameters (outputs) and can only pass values to the first layer of direct downstream nodes, whereas the Set Variable node passes values through scheduling variables and takes effect across the entire workflow. For a detailed comparison, see the "Differences from assignment nodes" section in this topic.

Limitation

Limit

Description

Edition requirement

Standard Edition or higher.

Resource group

A Set Variable node can run only on a serverless resource group.

Scope

Can only be created in workflows (including child workflows). Regular tasks do not involve scheduling variables and cannot contain this node.

Supported languages

Shell, Python3.

Variables that can be overwritten

Only scheduling variables that are already defined in the workflow in which the node resides can be overwritten. Undefined variable names are not created.

Data volume

The total data size of variables in a single run cannot exceed 2 MB.

How it works

Within a single workflow run, variables follow a read-write-propagate pattern, and each run is independent of the others.

  • Initialization: At the beginning of each run, all variables are loaded from the initial values that you specified in the schedule settings.

  • Read variables: In the code of a node, you can read the current value of a variable by using the ${workflow.variable.variable_name} format.

  • Write variables: A Set Variable node updates a variable value by printing output in the script. Once the node runs successfully, the new value takes effect in the current run.

  • Scope: Variable updates are propagated only to downstream nodes within the current run instance. Subsequent nodes receive the new value, but the initial workflow configuration and other run instances are not affected.

Note

Assignment markers are parsed only after a node runs successfully. If a node fails, the variable is not written back, and subsequent nodes still read the value before the overwrite.

Assignment marker syntax

Format

Syntax

Applicable scenario

Single-line

##WORKFLOW_VARIABLE::variable_name=value

Common scenarios where the value does not contain line breaks.

Multi-line

Starts with ##WORKFLOW_VARIABLE_BEGIN::variable_name, followed by the content on new lines, and ends with ##WORKFLOW_VARIABLE_END::variable_name.

Scenarios where the value contains line breaks, such as JSON or multi-line alert messages.

Shell example:

#!/bin/bash
# Read the current value of a variable
echo "current env: ${workflow.variable.env}"
# Single-line write. ds_flag is the variable_name.
echo "##WORKFLOW_VARIABLE::ds_flag=20260903"
# To include line breaks in a parameter, use multi-line write. table1 is the variable_name.
echo "##WORKFLOW_VARIABLE_BEGIN::table1"
echo 'age'
echo 'name'
echo "##WORKFLOW_VARIABLE_END::table1"

Python3 example:

print("current ds_flag: " + "${workflow.variable.ds_flag}")
# Single-line write. ds_flag and chain_flag are variable names.
print("##WORKFLOW_VARIABLE::ds_flag=20260904")
print("##WORKFLOW_VARIABLE::chain_flag=py-touched")
# To include line breaks in a parameter, use multi-line write. table1 is the variable_name.
print("##WORKFLOW_VARIABLE_BEGIN::table1")
print('age')
print('name')
print("##WORKFLOW_VARIABLE_END::table1")

A single Set Variable node can print multiple assignment markers to overwrite multiple variables at the same time.

Procedure

Step 1: Create a Set Variable node

  1. Go to the Data Studio page, double-click the target workflow, and open the workflow editing canvas.

  2. Click Add Node, select Assign Value to Variable from the general category, enter a node name, and confirm the creation.

  3. Configure upstream and downstream dependencies to determine where the node runs within the workflow.

    Note

    Variable overwrites take effect only on nodes that run after the Set Variable node. Make sure that the nodes consuming the new values are placed downstream of the Set Variable node.

Step 2: Write the overwrite logic

  1. Double-click the node to open the code editing page, and switch the compute language (Shell or Python3) at the bottom of the right panel.

  2. Write a script: Use ${workflow.variable.variable_name} to read the variables that participate in the calculation, and use assignment markers to write new values. For the assignment marker format, see Assignment marker format.

  3. Save the node.

Note

You can assign values only to scheduling variables that are already defined in the workflow. Assigning values to undefined variables has no effect. If none of the assignment markers match a defined variable, the node fails. When you edit the code, the UI displays the variable names available in the current workflow.

When the value contains quotation marks, such as JSON content, the replacement result is directly substituted into the code. You must wrap it according to the syntax rules of the host language (in Python, use """..."""). Otherwise, a script syntax error is triggered.

Step 3: Configure schedule settings

A Set Variable node cannot run independently and must be used together with other nodes. Therefore, you must properly configure schedule dependencies. For more information about schedule dependency configuration, see Configure schedule dependencies.

Step 4: Run and verify

Important

A Set Variable node can run only within a workflow. It does not support standalone execution.

  1. On the workflow editing canvas, click Run to run the entire pipeline.

  2. Check the runtime log of the Set Variable node to confirm that the assignment markers have been parsed.

  3. Check the runtime log of the downstream node to confirm that the reference ${workflow.variable.variable_name} returns the overwritten value.

Step 5: Deploy

After you verify that everything works correctly, submit and deploy the workflow to the production environment. The Set Variable node is deployed together with the workflow, and the variables are overwritten by using the same rules during production scheduling.

Step 6: View scheduling variables in Operation Center

  • Workflow variable task properties: Displays the variable definitions and output variables. You can view which variables this node overwrites.

  • Workflow variable task instance properties: Displays the specific values of execution variables and output variables.

  • Runtime log of tasks within the workflow: Prints the execution variable values read during the current run. If a variable has been overwritten, the log records the overwrite time and the Set Variable node that performed the overwrite. If a variable has not been overwritten, the initial value is displayed.

Usage examples

For complete usage examples, see Set Variable node usage examples.

Differences from the assignment node

Comparison item

Set Variable node

Assignment node

Value carrier

Scheduling variables of the workflow.

Node context parameters (outputs).

Value propagation scope

All nodes that run after this node within the same workflow, including indirect downstream nodes.

Only the first layer of direct downstream nodes.

How downstream nodes retrieve values

Reference ${workflow.variable.variable_name} in the code. No parameter configuration is required on downstream nodes.

Downstream nodes must add an input parameter in the node context and reference outputs, and then use ${parameter_name} to retrieve the value.

Supported languages

Shell and Python 3.

ODPS SQL, Python 2, and Shell.

How values are written

Print an assignment marker.

Automatically retrieves the last query or output result.

Usage notes

Within workflows only.

Both workflows and standalone tasks.

Use a Set Variable node when you need to pass results to multiple layers or multiple branches of downstream nodes. Use an assignment node when you only need to pass values to the immediately adjacent downstream node and want to leverage the context parameter extraction capability.

Notes

  • Scope: Both scheduling variables and Set Variable nodes are scoped to the workflow boundary. They are not visible across different workflows.

  • Execution order determines values: Variable overwrites take effect in the actual execution order of nodes. There is no guaranteed order between parallel branches. If two Set Variable nodes assign values to the same variable concurrently, the node that runs later overwrites the value set by the node that runs earlier. To ensure deterministic values, use dependencies to serialize the nodes into a sequential order. To avoid timing conflicts during variable assignment, do not assign values to the same variable in two branches of a workflow.

  • Read-only semantics: References to scheduling variables in regular node code are read-only. You cannot modify variable values through regular nodes. Runtime overwrites can only be performed through Set Variable nodes.

  • No write-back on failure: If a node fails, the assignment marker is not parsed, and the variable retains its value from before the attempted overwrite.

  • Undefined variables: Referencing an undefined variable does not cause an error. The placeholder is kept as-is in the code and participates in execution, which may cause script syntax errors. Overwriting an undefined variable has no effect. If none of the variables match, the node fails.

FAQ

  • Q: Can a Set Variable node pass values to a child workflow?

    A: Yes. Both workflows and child workflows support Set Variable nodes. The scheduling variable values of a parent workflow can be passed to the scheduling variables of a child workflow by resetting the values. For more information, see Pass scheduling variable values to a child workflow.

  • Q: Can I reference a scheduling variable in the value of a scheduling parameter?

    A: No. Scheduling variables can only be referenced in node code by using ${workflow.variable.variable_name}. Writing this format in the value of a scheduling parameter (script parameter) does not retrieve the variable value.

  • Q: Can I retrieve part of a variable value?

    A: No. Scheduling variables do not support extraction or sub-object access syntax such as ${workflow.variable.variable_name[0][1]} or ${workflow.variable.variable_name.name}. If you need to split values, use Shell or Python 3 in a Set Variable node to compute the desired parts and write them to separate variables for downstream nodes.

  • Q: Does an overwritten variable value affect the next scheduling run?

    A: No. The overwrite applies only to the current run. The next scheduling run starts with the initial value saved in the workflow schedule settings.

  • Q: Can a single node overwrite multiple variables at the same time?

    A: Yes. Print multiple assignment markers in the same node. You can mix single-line and multi-line formats.

Previous：HTTP node (Beta)Next：Algorithms