Configure a DRDS node
DataWorks provides DRDS nodes that you can use to develop and periodically schedule DRDS tasks and integrate DRDS tasks with other types of tasks. This topic describes how to use a DRDS node to develop tasks.
Background information
PolarDB for Xscale (PolarDB-X), formerly called DRDS, is a distributed database service. It allows you to perform a scale-out operation to upgrade a relational database to a distributed system that supports storage and access of large amounts of data. PolarDB-X maintains the original features of relational databases, such as MySQL. For more information, see Overview.
Prerequisites
-
Create a workflow.
In DataStudio, you develop tasks for different engines within workflows. Therefore, you must create a workflow before you create a node. For more information, see Create a workflow.
-
A DRDS data source is added.
You must add your PolarDB-X database to DataWorks as a DRDS data source before you can use the data source to access data in the database. For information about how to add a data source, see Add and manage data sources. For information about how to use a DRDS data source in DataWorks, see DRDS (PolarDB-X 1.0) data source.
NoteYou can create DRDS nodes based on only DRDS data sources that are added to DataWorks in Java Database Connectivity (JDBC) connection string mode.
-
A network connection is established between your data source and a resource group.
You must make sure that the desired data source is connected to the resource group that you want to use. For more information about how to configure network connectivity, see Establish a network connection between a resource group and a data source.
-
(Optional, for RAM users) Add the RAM user for task development to the workspace and grant them the Development or Workspace Administrator role. The Workspace Administrator role has extensive permissions, so assign it with caution. For more information about how to add members and grant roles, see Add members to a workspace.
Limits
This type of node is supported in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou)3(Zhangjiakou)、China (Shenzhen), China (Chengdu), China (Hong Kong), Singapore, Malaysia (Kuala Lumpur), Germany (Frankfurt), US (Silicon Valley), and US (Virginia).
Step 1: Create a DRDS node
Log on to the DataWorks console. In the target region, click in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Development.
-
Right-click the target workflow and choose .
-
In the Create Node dialog box, enter a Name for the node and click OK. After the node is created, you can develop and configure the task in the node editor.
Step 2: Develop a DRDS task
(Optional) Select a DRDS data source
If multiple DRDS data sources are added to your workspace, you must select one from the Select Data Source drop-down list in the upper part of the configuration tab of the node based on your business requirements. If only one DRDS data source is added to your workspace, the DRDS data source is used to develop tasks.
You can create DRDS nodes based on only DRDS data sources that are added to DataWorks in Java Database Connectivity (JDBC) connection string mode.
Develop SQL code: Simple example
In the code editor on the configuration tab of the DRDS node, write task code. Sample code:
SELECT * FROM usertablename;
Develop SQL code: Use scheduling parameters
DataWorks provides Scheduling Parameter that allow you to dynamically pass parameters to code in periodic scheduling scenarios. In a node task, you can define a variable in your code by using the ${variable_name} format. Then, assign a value to the variable in the scheduling properties > Scheduling Parameter section in the right-side navigation bar of the node editing page. For more information about the supported formats and configuration details, see Supported formats for scheduling parameters and Configure and use scheduling parameters.
Sample code:
SELECT '${var}'; -- You can assign a specific scheduling parameter to the var variable.
Step 3: Configure task scheduling
If you need to run the task on a schedule, click Scheduling in the right-side navigation pane of the node editor. Configure the scheduling information for the node based on your requirements. For more information, see Configure scheduling properties for a task.
You must set the Rerun attribute and Parent Nodes properties for the node before you can submit it.
Step 4: Debug the task
Debug the task to verify that it runs as expected:
-
Select a resource group to run the code and assign values to custom parameters.
-
Click the
icon in the toolbar. In the Parameter dialog box, select the scheduling resource group to use for debugging. -
If you use scheduling parameter variables in your code, assign debugging values to them here. For more information about the parameter assignment logic, see Debug a task.
-
-
Save and run the code.
Click the
icon in the toolbar to save the code. Then, click the
icon to run the task. -
(Optional) Perform smoke testing.
To verify that a scheduled task runs as expected, you can perform smoke testing in the development environment when you submit the node or afterward. For more information, see Perform smoke testing.
Step 5: Submit and deploy the task
After you configure the node, you must submit and deploy it. The node will then run periodically based on its scheduling configuration.
-
Click the
icon in the toolbar to save the node. -
Click the
icon in the toolbar to submit the node task.In the Submission dialog box, enter a Change Description, and select whether to request a code review after submitting the node.
Note-
You must set the Rerun attribute and Parent Nodes of a node to submit it.
-
The code review feature ensures code quality and prevents errors that can occur when unverified code is deployed to the production environment. If you enable this feature, a reviewer must approve the submitted code before deployment. For more information, see Code review.
-
If your workspace is in standard mode, submit the task and then click Deploy in the upper-right corner of the node editor page to deploy it to the production environment. For more information, see Deploy tasks.
Next steps
Task O&M: After the task is committed and deployed, it runs periodically based on its scheduling configuration. You can click Operation Center in the upper-right corner of the node editor to monitor its scheduling and operational status. For more information, see Manage auto-triggered tasks.