Scheduling parameters

You can use two custom variables to construct the value:

• Date part: Use a parameter such as datetime=$[yyyy-mm-dd].

• Time part: Use a parameter such as hour=$[hh24:mi:ss].

After obtaining the date and time separately, concatenate them with a space in your code, such as pt=${datetime} ${hour}. For example, if your SQL code is select * from xc_demo_partition where dt='${key1} ${key2}';, navigate to Scheduling > Parameters on the right and configure key1=$[yyyymmdd] key2=$[hh24miss].

Scheduling parameters use the instance's fixed scheduling time. The parameter's value is substituted when the instance is generated and does not change even if the actual run time is delayed. To verify this, go to Operation Center > Cycle Instance. In the Properties panel, compare the Scheduling Time (for example, 2021-07-13 00:00:00) with the Start Time (for example, 00:00:27). In DataStudio, you can go to Scheduling > Parameters and configure var1=$bizdate var2=$cyctime var3=$[yyyymmdd] var4=$[yyyymmddhh24miss]. Set the Scheduling Cycle to Daily and the Scheduled Time to 00:00. Then, use select '${var1}' '${var2}'; in your code to verify the values of the data timestamp and scheduling time.

After defining variables in your code, click Scheduling in the right-side pane and assign values to these variables in the Parameters panel. For details about parameter usage, see Supported formats of scheduling parameters. For example, in the Parameters field of an ODPS Spark node, enter $(var). Then, in the parameters table, add a new row with Parameter Name as var and Value as $bizdate. Click Preview Scheduling Parameters to verify the substitution result.

Some nodes, such as a batch synchronization node, cannot use the value of a scheduling parameter directly because it requires processing, like a format conversion. For this type of node, configure an assignment node as an upstream node. Use the assignment node to reference the scheduling parameter and process its return value. Then, use the node context feature to pass the final result to the target downstream node.

• To configure an assignment node, see assignment node.

• To configure the node context, see Configure node context.

Scheduling parameters are substituted automatically only by the scheduling system. Therefore, to test parameter substitution in DataStudio, you must use Perform Smoke Testing in Development Environment.

Solutions:

1. Check whether a value is assigned to the scheduling parameter.

2. Check whether the configured scheduling parameter meets the format requirements. For more information, see Supported formats of scheduling parameters.

Solutions:

1. Check whether a value is assigned to the variable.

2. Check for extra spaces in the scheduling parameter definition.

3. Node names cannot contain both periods (.) and Chinese characters.

Example: Consider the string time①=②$[yyyymmdd③hh24:mi:ss] time1=$[yyyymmdd]④time2=$[hh24:mi:ss], where ①, ②, ③, and ④ indicate potential locations for spaces.

• Do not add spaces around the equals sign (=). In the example, this applies to locations ① and ②.

• Do not add spaces within the parameter expression. In the example, this applies to location ③.

• Use a single space to separate multiple parameter definitions. In the example, this applies to location ④.

If you modify scheduling parameters in DataStudio and then commit and deploy the changes, go to the Properties panel of the Cycle Task in Operation Center to check if the parameters are as expected. If not, check the task deployment and deployment package pages to confirm if the deployment was successful. To check parameter substitution for a single instance, view the Properties panel for the Cycle Instance.

Example scenario:

• When the scheduling parameter time1 for Cycle Instance A1 of Node A is set to $bizdate for that day and the instance has run successfully, the parameter value in the code is the bizdate data timestamp (a specific value).

• If the scheduling parameter time1 is later changed from $bizdate to $cyctime and deployed, the instance's execution parameters immediately update to reflect the new value, resolved from the scheduling time (cyctime).

• If you re-run the instance, it will use the latest parameter configuration: time1=$cyctime.

• To view the parameter values that an instance used before a configuration change, you must check its logs. For more information, see the FAQ: How do I check parameter substitution for a running instance using logs?.

On the Operation Center > Cycle Instance page, select an instance and check the Execution Parameters field in the Properties panel below. You can see the actual substituted values (for example, var1=20210708233000 var2=20210709000000) to confirm if the parameters were updated as expected.

In the logs, find the line containing the string SKYNET_PARAVALUE.

07-09 00:00:19  INFO SKYNET_RERUN_MODE=1:
07-09 00:00:19  INFO SKYNET_FLOW_PARAVALUE=:
07-09 00:00:19  INFO SKYNET_ONDUTY=xxx:
07-09 00:00:19  INFO SKYNET_SYSTEMID=:
07-09 00:00:19  INFO CALC_ENGINE_IDENTIFIER=xxx:
07-09 00:00:19  INFO SKYNET_SOURCEID=xxx:
07-09 00:00:19  INFO SKYNET_PARAVALUE=var1=20210708233000 var2=20210709000000:
07-09 00:00:19  INFO SKYNET_TASKID=xxx:
07-09 00:00:19  INFO SKYNET_TENANT_ID=283789484710656:
07-09 00:00:19  INFO SKYNET_ID=700003341088:
07-09 00:00:19  INFO SKYNET_JOBID=700200619122:
07-09 00:00:19  INFO SKYNET_NODENAME=xc_partition_copy:
07-09 00:00:19  INFO SKYNET_CYCTYPE=0:
07-09 00:00:19  INFO SKYNET_DSC_JOB_VERSION=1:
07-09 00:00:19  INFO SKYNET_TASK_INPUT={}:
07-09 00:00:19  INFO SKYNET_FLOWNAME=ATCLOUD_FLOW:
0-09 00:00:19  INFO FILE_ID=700003341088:

The system supports daylight saving time for instance generation and parameter-based partition calculation, ensuring that nodes run correctly on days when daylight saving time begins and ends. The following examples use the US Pacific Time zone:

• When daylight saving time begins (spring forward), the clock jumps from 02:00 to 03:00, creating a 23-hour day. Consequently, a task scheduled between 02:00 and 02:59 on that day will be skipped.

• When daylight saving time ends (fall back), the clock moves from 02:59 back to 02:00, creating a 25-hour day.