Data push node

How it works

A data push node uses the Input and Output Parameters to retrieve output parameters from upstream nodes. You can then use these parameters as placeholders in the message body and send them to a destination. The data push node can retrieve output parameters from upstream SQL query nodes and assignment nodes.

• After an SQL query node runs, it generates an outputs parameter that contains the query results and passes them to downstream nodes. For more information, see Configure node context.

• When you configure the message body in a data push node, you can use the ${Parameter name} format in a Markdown component to reference and display output parameters from an upstream node.

• To display data in a table component, you can select the field names from the upstream SQL query in the component's associated parameter setting to retrieve data.

Prerequisites

• The DataWorks service is activated. For more information, see Activate DataWorks.

• A DataWorks workspace is created. For more information, see Create a workspace.

• A workflow is created in your DataWorks workspace.

• A serverless resource group is required, as this node can run only on a serverless resource group. To purchase and use a serverless resource group, see Use serverless resource groups.

• Public network access is enabled for the resource group. For more information, see Overview of network connectivity solutions.

Limitations

• The following data size limits apply when pushing data to different destinations:

  • DingTalk: The message size cannot exceed 20 KB.

  • Lark: The message size cannot exceed 20 KB, and images must be smaller than 10 MB.

  • WeCom: Each chatbot cannot send more than 20 messages per minute.

  • Teams: The message size cannot exceed 28 KB.

  • Email: Each data push task supports only one email body. For other limitations, refer to the SMTP limits of your email service provider.

• The data push node is available only in workspaces in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong), Singapore, Malaysia (Kuala Lumpur), US (Silicon Valley), US (Virginia), and Germany (Frankfurt).

1. Create an upstream node

A data push node retrieves Context Parameters from an upstream node's context to push them to a destination. Therefore, before you create a data push node, you must first create an upstream SQL query node or assignment node.

2. Create a data push node

1. On the workflow canvas, click to create a data push node.

   Parameter	Description
   Node Type	Select Data Push from the drop-down list.
   Path	Select the path that contains the upstream node from Step 1.
   Name	Enter a name for the node.

2. Double-click the new data push node to open its configuration page.

3. In the right-side pane, click Properties. Under Scheduling Dependency > Parent Nodes, click Add to set the node you created in Step 1 as an upstream dependency.

4. On the Properties panel, under Resource Group > Resource Group for Scheduling, select a serverless resource group created on or after June 28, 2024. If your resource group was created before this date, you must submit a ticket to upgrade it.

5. On the Properties panel, under Input and Output Parameters > Input Parameters, click Add to create an input parameter. For its value, select the output parameter of the query node from Step 1. Then, close the Scheduling panel.

6. On the node's configuration page, configure the destination, title, and body of the message.

   1. From the Destination drop-down list, select a destination. If your desired destination is not listed, click Create Data Push Destination to create it.

      Parameter	Description
      Type	Supported types: DingTalk, Lark, WeCom, Teams, and Email.
      Object	The name for the destination.
      Webhook	The webhook URL for a DingTalk, Lark, or WeCom chatbot; the incoming webhook for Teams; or the SMTP details for email. Obtain this information from the target platform. Note To obtain a DingTalk robot webhook, see "How do I obtain the access_token for an internal application robot webhook?" in FAQ. To obtain a Lark robot webhook, see Configure a Lark webhook trigger. To obtain a WeCom robot webhook, see Group Robot Configuration Guide. To obtain a Teams webhook, see Create Incoming Webhooks.

      To manage push destinations:

      1. Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose Data Analysis and Service > DataService Studio. On the page that appears, select the desired workspace from the drop-down list and click Go to DataService Studio.

      2. On the Service Development page, click the Settings icon in the lower-left corner and go to the Destination Management tab. On this tab, you can manage existing destinations or click Create Data Push Destination to add a new one. For more information, see Create a webhook data push destination.

   2. Name: Enter a title for the message.

   3. Body: You can add Markdown and Table components. For more information, see Configure push content.

      Note

      • If the upstream node is an SQL query node:

        • In the Markdown body of a data push, use field names from an SQL query as placeholders in the ${field_name} format to retrieve upstream output parameters.

        • For tables in a data push body, use field names from an upstream SQL query as Parameters to retrieve data.

      • If the upstream node is an assignment node, you must use a placeholder in the format ${Input parameter name} in the message body. ${Input parameter name} is the name of the Input Parameters you configured on the data push node.

7. Click the Save icon to save the data push node.

3. Run and deploy the task

After you configure the data push workflow, test it to ensure it runs correctly. After the test succeeds, commit and deploy the task.

1. On the workflow canvas, click the Run icon to run the workflow.

2. After the success icon appears on all nodes, click the Commit icon to commit the workflow.

3. In the Submission dialog box, select the nodes you want to commit and select the Ignore I/O Inconsistency Alerts checkbox.

4. Click Submission.

5. After the nodes are successfully committed, deploy them. For more information, see Publish tasks.

Best practices

The data push node supports various use cases within a workflow, including simple push, combined push, script-based push, conditional push, and MaxCompute push. For more information, see Best practices: Configure a data push node in a workflow.

Next steps

After the data push task is deployed, you can manage it in Operation Center. For more information, see Perform basic O&M operations on auto triggered nodes.