Data Push node-DataWorks(DataWorks)-阿里云帮助中心

Background

A data push node uses Node Context Parameters to retrieve output parameters from upstream SQL query nodes, such as assignment nodes, Hologres SQL nodes, or ClickHouse SQL nodes. You can reference these parameters as placeholders in the push body and send the content to the destination.

After an upstream SQL query node finishes running, it generates an output parameter named outputs that contains the query results.
In the downstream data push node, use Node Context Parameters to retrieve the outputs parameter from the upstream node and bind it as an input parameter.
In the data push node configuration, reference the input parameter as the push content, set the destination, and send the content.

Supported upstream SQL query nodes:

Assignment node: MaxCompute SQL query nodes are not directly supported. If needed, you can create an assignment node to query MaxCompute data.
Other SQL nodes: Hologres SQL node, ClickHouse SQL node, AnalyticDB for MySQL SQL node, AnalyticDB for PostgreSQL SQL node, StarRocks SQL node, PostgreSQL node, and MySQL node.

Prerequisites

You have created a workspace.
You have created a serverless resource group and associated it with the target DataWorks workspace.

Limitations

Data size limits for different push destinations are as follows.

Push destination	Push limits
DingTalk	The push data size cannot exceed 20 KB.
Lark	The push data size cannot exceed 20 KB, and images must be smaller than 10 MB.
WeCom	Each bot can send no more than 20 messages per minute.
Teams	The push data size cannot exceed 28 KB.
Email	Each data push task supports only one email body. If an email body has already been added, you cannot add another one. For more limits, refer to the SMTP limits of your email service.

Supported regions for Data Push:

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong), Singapore, Japan (Tokyo), US (Silicon Valley), US (Virginia), and Germany (Frankfurt).

Go to Data Studio

Go to the Workspaces page in the DataWorks console. In the top navigation bar, select a desired region. Find the desired workspace and choose Shortcuts > Data Studio in the Actions column.
In the left-side navigation pane, click the icon to go to the Data Studio page.

Create a push pipeline

Create and configure a pipeline consisting of an SQL query node and a data push node.

Create a workflow, and then create an SQL query node and a data push node in the workflow.
Create an SQL query node and add the SQL code for the data you want to query and push.
Note
- The data push node does not support directly retrieving query data from ODPS SQL. You can create an assignment node and query the data to push in the assignment node.
- Other SQL query nodes can be created as usual.

Configure the SQL query node.

You must configure the Compute Resource, Resource Group for Scheduling, and Same-cycle Dependency in the Scheduling Settings panel on the right side of the node. If no upstream node exists, select Add Root Node. The output parameter configuration differs as follows:

Node type	Configuration notes
Assignment node	An assignment node has a default output parameter. No additional configuration is required.
Other SQL query nodes	Other SQL query nodes do not have default output parameters. Go to Node Context Parameters in the Scheduling Settings panel on the right side of the node, find Node output parameters, and click Add Assignment Parameter to pass the query results to the downstream data push node.

You have created a data push node and configured the SQL query node as the upstream node.
1. Click Scheduling Settings. In the Scheduling Dependency > Same-cycle Dependency section, click Add and search by node name to add the SQL query node you created as the upstream node of the data push node.
2. In the Scheduling Policy > Resource Group for Scheduling section, select the serverless resource group you created.
3. In the Node Context Parameters > Input Parameters section, click Add parameters to add the output parameter of the SQL query node as the value source and bind it as the input parameter of the data push node.

Configure push destinations and content

On the data push node editing page, customize the push title and configure the destination and body content.

Configure the push destination

In the Destination section, select a destination.

If the destination does not exist, click Create Destination. You can also set a data push destination in Data Service.

Note
To modify a data push destination, go to the Data Push destination management page in Data Service.

On the Create Destination page, create a data push destination by configuring the following parameters.

Create a DingTalk, Lark, WeCom, or Teams push destination

Parameter	Description
Destination	Select DingTalk, Flying Book, WeCom, or Teams.
Destination Name	Customize the name based on your business needs.
Webhook	Obtain and configure the webhook URL for the selected push destination. For information about how to obtain a DingTalk bot Webhook, see FAQ, specifically the section How to obtain the access_token of an internal application bot Webhook. For information about how to obtain a Lark bot Webhook, see Lark Webhook trigger configuration. For information about how to obtain a WeCom bot Webhook, see Group bot configuration guide. For information about how to obtain a Teams Webhook, see Create Incoming Webhooks with Workflows for Microsoft Teams.

Create an email push destination

Parameter	Description
Destination	Select Mail.
SMTP Host	The address of the SMTP server used to send emails.
SMTP Port	The port number of the SMTP server. Default value: 465.
Sender Address	The sender email address.
Sender Nickname	Optional. A custom display name for the sender.
SMTP Account	The account used to log on to the SMTP server.
SMTP Password	The password for SMTP authentication.
Receiver Address	The email addresses of the recipients. Separate multiple addresses with commas (,).

Configure the push body

You can configure the body content in three formats: Markdown, table, and email.

In the Markdown body, use the Input Parameters parameter name as a placeholder in the format ${input_parameter_name} to retrieve the output parameter from the upstream node.
In the table body, use the column names of the upstream SQL query node as Parameters to retrieve data.
DataWorks supports adding an email body to the push content. Note the following when editing the email body:
- Each data push task supports only one email body.
- The email body is rendered only when the push destination is email. For other destinations, the email body is hidden in the webhook message.

Debug the data push

After the configuration is complete, save the SQL query node and data push node. Then, click the icon in the toolbar above the workflow to debug the task.

Next steps

Configure schedule settings: To run nodes on a periodic schedule, configure the Scheduling Policy in the Scheduling Settings panel on the right side of the node.
Deploy a node: To deploy tasks to the production environment, click the icon to start the deployment. Nodes are periodically scheduled only after they are deployed to production.