The data push node sends query results from other nodes in your Data Studio workflow to destinations such as DingTalk, Lark, WeCom, Teams, or email by creating a data push destination. This allows your team members to receive and stay informed of the latest data in a timely manner.
Background
A data push node uses Node Context Parameters to retrieve the output parameters from upstream SQL query nodes, such as an assignment node, Hologres SQL node, or ClickHouse SQL node. You can then edit these parameters in the push body as placeholders and send the content to the destination.
After an upstream SQL query node finishes running, it generates a node output parameter named
outputscontaining the query results.In the downstream data push node, use Node Context Parameters to retrieve the
outputsparameter from the upstream node and bind it as an input parameter of the data push node.In the data push node configuration, reference the input parameter as the push content, set the destination, and push the content.
Supported upstream SQL query nodes:
Assignment node: MaxCompute SQL query nodes are not directly supported. If needed, you can create an assignment node to query MaxCompute data.
Other SQL nodes: Hologres SQL node, ClickHouse SQL node, AnalyticDB for MySQL SQL node, AnalyticDB for PostgreSQL SQL node, StarRocks SQL node, PostgreSQL node, and MySQL node.
Prerequisites
You have created a workspace.
You have created a serverless resource group and associated it with the target DataWorks workspace.
Limitations
Data size limits for different push destinations are as follows.
Push destination
Push limits
DingTalk
The push data size cannot exceed 20 KB.
Lark
The push data size cannot exceed 20 KB, and images must be smaller than 10 MB.
WeCom
Each bot can send no more than 20 messages per minute.
Teams
The push data size cannot exceed 28 KB.
Email
Each data push task supports only one email body. If an email body has already been added, you cannot add another one. For more limits, refer to the SMTP limits of your email service.
Supported regions for Data Push:
China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong), Singapore, Japan (Tokyo), US (Silicon Valley), US (Virginia), and Germany (Frankfurt).
Go to Data Studio
Go to the Workspaces page in the DataWorks console. In the top navigation bar, select a desired region. Find the desired workspace and choose in the Actions column.
In the left-side navigation pane, click the
icon to go to the Data Studio page.
Create a push pipeline
Create and configure a pipeline that consists of an SQL query node and a data push node.
Create a workflow, and then create an SQL query node and a data push node in the workflow.
Create an SQL query node and add the SQL code for the data you want to query and push.
NoteThe data push node does not support directly retrieving query data from ODPS SQL. You can create an assignment node and query the data to be pushed in the assignment node.
Other SQL query nodes can be created as usual.
Configure the SQL query node.
You must configure the Compute Resource, Resource Group for Scheduling, and Same-cycle Dependency in the Scheduling Settings panel on the right side of the node. If no upstream node exists, you can select Add Root Node. The output parameter configuration differs as follows:
Node type
Configuration notes
Assignment node
An assignment node has a default node output parameter. No additional output parameter configuration is required.
Other SQL query nodes
Other SQL query nodes do not have default node output parameters. You must go to Node Context Parameters in the Scheduling Settings panel on the right side of the node, find Node output parameters, and click Add Assignment Parameter to pass the current query results to the downstream data push node through parameters.
You have created a data push node and configured the SQL query node as the upstream node.
Click Scheduling Settings. In the section, click Add and search by node name to add the SQL query node you created as the upstream node of the data push node.
In the section, select the serverless resource group you created.
In the section, click Add parameters to add the output parameter of the SQL query node as the value source and bind it as the input parameter of the data push node.
Configure push destinations and content
On the data push node editing page, customize the push title and configure the push destination and body content.
Configure the push destination
In the Destination section, select a destination.
If the destination does not exist, click Create Destination. You can also set a data push destination in Data Service.
NoteTo modify a data push destination, go to the Data Push destination management page in Data Service.
On the Create Destination page, create a data push destination by configuring the following parameters.
Create a DingTalk, Lark, WeCom, or Teams push destination
Parameter
Description
Destination
Select DingTalk, Flying Book, WeCom, or Teams.
Destination Name
You can customize the name based on your business needs.
Webhook
Obtain and configure the Webhook URL based on the selected push destination.
For information about how to obtain a DingTalk bot Webhook, see FAQ, specifically the section How to obtain the access_token of an internal application bot Webhook.
For information about how to obtain a Lark bot Webhook, see Lark Webhook trigger configuration.
For information about how to obtain a WeCom bot Webhook, see Group bot configuration guide.
For information about how to obtain a Teams Webhook, see Create Incoming Webhooks with Workflows for Microsoft Teams.
Create an email push destination
Parameter
Description
Destination
Select Mail.
SMTP Host
The address of the SMTP server used to send emails.
SMTP Port
The port number of the SMTP server used to connect to the mail server. Default value: 465. You can manually change this value.
Sender Address
The email address from which emails are sent.
Sender Nickname
Optional. A custom display name for the sender.
SMTP Account
The email account used to log on to the SMTP server.
SMTP Password
The password used to authenticate the SMTP account.
Receiver Address
The email addresses of the recipients. Separate multiple addresses with commas (,).
Configure the push body
You can configure the body content in three formats: Markdown, table, and email body.
In the Markdown body of a data push, you can directly use the parameter name of the Input Parameters of the data push node as a placeholder in the format
${input_parameter_name}to retrieve the output parameter from the upstream node.In the table body of a data push, you can use the column names of the upstream SQL query node as Parameters to retrieve data.
DataWorks Data Push supports adding an email body to the push content. When editing the email body, note the following:
Each data push task supports only one email body.
The email body is rendered only when the push destination is email. When the push destination is not email, the email body is hidden in the message pushed via Webhook.
Debug the data push
After the configuration is complete, save the SQL query node and data push node. Then, click the
icon in the toolbar above the workflow to debug the data push task.
Next steps
Configure schedule settings: If you want nodes in the project directory to be executed on a periodic schedule, configure the Scheduling Policy in the Scheduling Settings panel on the right side of the node.
Deploy a node: If you want to deploy tasks to the production environment, click the
icon on the page to initiate the deployment process. Nodes in the project directory are periodically scheduled only after they are deployed to the production environment.