Scheduling variables

Step 1: Define scheduling variables

1. Go to the Data Studio page, double-click the target workflow, and open the workflow editing canvas.

2. Click Schedule Settings on the right side and find the Scheduling Variable section.

3. Click Add Variable and enter the variable information.

   Configuration item	Description
   Variable name	The unique identifier of the variable, which must be unique within the workflow. We recommend that you use only letters, digits, and underscores.
   Variable value	The initial value of the variable. Each time a task starts, the variable takes effect starting from this value. You can specify a constant, workspace parameter, or time expression such as $[yyyy-mm-dd]. For more information, see Scheduling parameter sources and expressions. Referencing workflow parameters (such as ${workflow.parameter_name}) or the output values of Set Variable nodes in the value is not supported.

   Examples: env=prod, ds_flag=$[yyyymmdd].

4. Save the workflow.

Step 2: Reference the variable in node code

In the code of a node within a workflow, use the ${workflow.variable.variable_name} format to reference scheduling variables. Variable names support letters, digits, underscores, and hyphens.

Take a SHELL node as an example. Configure env and ds_flag on the workflow, enter the following code in the code editor, and save.

#!/bin/bash
echo "env=${workflow.variable.env}"
echo "ds_flag=${workflow.variable.ds_flag}"

At runtime, the system replaces placeholders with the current values of the variables for this run (either initial values or values overwritten during this run) before the node executes, and then passes the code to the engine for execution.

• Scheduling variables and scheduling parameters can be used together: ${workflow.variable.env} is replaced according to scheduling variable rules, while ${bizdate} is still replaced with the business date according to scheduling parameter rules.

• An unknown variable name does not cause a replacement error. The placeholder is kept as-is in the code. Make sure that the referenced variables are defined on the workflow.

• Scheduling variables can only be referenced in the code area. Writing ${workflow.variable.variable_name} in a parameter value in the schedule settings script parameters does not retrieve the variable value.

• Extracting a partial value from a variable is not supported. Notations such as ${workflow.variable.variable_name[0][1]} and ${workflow.variable.variable_name.name} are invalid. To split a value, use a Set Variable node to compute the result with Shell or Python3 and write it to another variable.

Step 3: Test run

1. On the workflow editing canvas, click Run at the top of the page.

2. In the Run Parameters form, modify the value for this run in the workflow variables column as needed. The value assigned here is a temporary value for the debug run, takes effect only for this run, and does not change the initial value of the variable.

3. Execute the run and check the variable replacement results in the node runtime log to verify that they match expectations.

Step 4: Use a Set Variable node to modify variables at runtime

To modify variable values during a run, add a Set Variable node to the workflow. This node is a general-purpose node that supports Shell and Python3. It writes new values by printing assignment markers to standard output in the script. The following rules apply.

1. Each time a workflow runs, the system creates a copy of the scheduling variables for that run. When a node in the workflow executes, the ${workflow.variable.variable_name} placeholders in the code are replaced with values from this copy.

2. Before a Set Variable node executes, the ${workflow.variable.variable_name} placeholders in its script are also replaced with the current values of the variables. This allows the script to read existing variable values before computing new ones.

3. During script execution, new values are written by printing assignment markers to standard output. Two formats are supported:

   • Single-line format: ##WORKFLOW_VARIABLE::variable_name=value.

   • Multi-line format: Starts with ##WORKFLOW_VARIABLE_BEGIN::variable_name and ends with ##WORKFLOW_VARIABLE_END::variable_name. This format is suitable for values that contain line breaks, such as multi-line data.

   # Read variable env, compute and overwrite variable ds_flag
   echo "current env: ${workflow.variable.env}"
   echo "##WORKFLOW_VARIABLE::ds_flag=20260903"
   # Multi-line assignment
   echo "##WORKFLOW_VARIABLE_BEGIN::result_json"
   echo '{"name":"shirt","count":50}'
   echo '{"name":"coat","count":120}'
   echo "##WORKFLOW_VARIABLE_END::result_json"

4. After the node executes successfully, the system parses the assignment markers in the standard output and writes the new values back to the variable copy for the current run. Nodes that run after this node will obtain the new values when referencing ${workflow.variable.variable_name}.

Step 5: Deploy and verify

1. After testing is complete, submit and deploy the workflow to the production environment. Scheduling variables are deployed together with the workflow.

2. After successful deployment, go to Operation Center and verify that the variable replacement results in the scheduled tasks match expectations.

Step 1: Define variables

Add 7 variables in the workflow schedule settings.

Step 2: Build the node pipeline

Create the following six nodes and configure their dependencies as described in the table below: set_ds_flag branches into three paths, and summary_report merges the three paths.

The code of the three Set Variable nodes is as follows.

1. Set Variable node set_ds_flag. Set the language type of the Set Variable node to SHELL.

   # set_ds_flag (variable assignment node, Shell)
   #!/bin/bash
   echo "env=${workflow.variable.env} tbl=${workflow.variable.tbl_prefix} ds_flag(before)=${workflow.variable.ds_flag}"
   echo "##WORKFLOW_VARIABLE::ds_flag=20260903"
   echo "##WORKFLOW_VARIABLE_BEGIN::stats_json"
   echo "{\"table\":\"${workflow.variable.tbl_prefix}user_trade_${workflow.variable.env}\",\"rows\":120,\"ok\":true}"
   echo "##WORKFLOW_VARIABLE_END::stats_json"

2. Set Variable node assign_next_day. Set the language type of the node to python3.

   # assign_next_day (variable assignment node, Python3)
   print("[b_py] ds_flag(a)=" + "${workflow.variable.ds_flag}")
   stats = """${workflow.variable.stats_json}"""
   print("[b_py] stats_json=" + stats)
   print("##WORKFLOW_VARIABLE::ds_flag=20260904")
   print("##WORKFLOW_VARIABLE::chain_flag=py-touched")

3. Set Variable node assign_alert_msg. Set the language type of the node to python3.

   # assign_alert_msg (variable assignment node, Python3)
   msg_table = "${workflow.variable.tbl_prefix}" + "user_trade_" + "${workflow.variable.env}"
   print("[alert_assign] build alert for " + msg_table)
   print("##WORKFLOW_VARIABLE_BEGIN::alert_msg")
   print("[DAILY-ALERT] table: " + msg_table)
   print("[DAILY-ALERT] ds: ${workflow.variable.ds_flag}")
   print("please contact: dw-group-01")
   print("##WORKFLOW_VARIABLE_END::alert_msg")

The code of the three consumer nodes is as follows.

1. Python node quality_check.

   # quality_check (Python node)
   rows = __import__("json").loads("""${workflow.variable.stats_json}""").get("rows", 0)
   ts = int("${workflow.variable.quality_threshold}")
   print("[quality] rows=%s threshold=%s -> %s" % (rows, ts, "PASS" if rows >= ts else "WARN"))

2. Python node prepare_report.

   # prepare_report (Python node)
   print("[prepare] ds_flag=" + "${workflow.variable.ds_flag}" + " chain=" + "${workflow.variable.chain_flag}")
   stats = """${workflow.variable.stats_json}"""
   print("[prepare] report table rows from stats_json=" + stats)

3. Python node summary_report.

   # summary_report (Python node)
   print("[summary] ds_flag(final)=" + "${workflow.variable.ds_flag}")
   print("[summary] chain=" + "${workflow.variable.chain_flag}")
   stats = """${workflow.variable.stats_json}"""
   print("[summary] stats_json=" + stats)
   msg = """${workflow.variable.alert_msg}"""
   print("[summary] alert_msg=" + msg)

Step 3: Run the workflow and verify results

Go back to the workflow canvas and click the run button on the toolbar. After the workflow runs, the key output in the logs of each node is as follows. Variable values are passed sequentially in the order in which the nodes run.

Three passing scenarios

Configuration steps

1. In the editing canvas of the parent workflow, add a SUBPROCESS node.

2. In the Workflow Variables section, find the child workflow variable to which you want to pass a value. You can also click the edit icon in the upper-right corner of the node on the canvas to modify it later.

3. In the Reset Value field, enter a reference to the parent workflow variable, such as ${workflow.variable.env}. Leave it empty to skip resetting, in which case the child workflow uses its own initial value.

4. Save and deploy the parent workflow. If the variable definitions of the child workflow change, you must also redeploy the child workflow.