How to create a CDH Impala node-DataWorks(DataWorks)-阿里云帮助中心

Prerequisites

You have created a CDH cluster and registered it with DataWorks.

Before you create CDH-related nodes and develop CDH tasks, you must register your CDH cluster with a DataWorks workspace. For more information, see Bind a CDH compute resource in the old version of DataStudio.

Limitations

This task type can run on a serverless resource group (recommended) or an older exclusive resource group for scheduling.

Step 1: Create a CDH Impala node

Log on to the DataWorks console. In the target region, click Data Development and O&M > Data Development in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Development.
Right-click a workflow and choose Create Node > CDH > CDH Impala.

Note
You can also hover over the New button at the top and follow the on-screen instructions to create a CDH node.
In the Create Node dialog box, enter a Name for the node and click OK.

Step 2: Develop an Impala task

Double-click the created node to open the task development page.

(Optional) Select a CDH engine instance

If your workspace is registered with multiple CDH clusters, select the appropriate one from the Engine Instance CDH drop-down list at the top of the page. If only one cluster is bound, no selection is needed. For example, select CDH production + test environment. To access a domain that has an IP address allowlist, you must use an exclusive resource group for scheduling.

Simple example

Enter the task code in the SQL editor. Example:

SHOW tables;
SELECT * FROM userinfo ;

Using scheduling parameters

DataWorks provides Scheduling Parameter to dynamically pass values to your code during scheduled runs. You can define variables in your code by using the ${Variable name} format and then assign values to these variables at Scheduling Settings > Parameter. For information about the supported formats for scheduling parameters, see Supported formats of scheduling parameters.

SELECT '${var}'; -- Use with scheduling parameters.

Step 3: Configure task scheduling

To run the node task on a schedule, click Scheduling on the right side of the node configuration tab and configure the scheduling properties:

Configure the basic properties of the task. For more information, see Configure basic properties.
Configure the scheduling cycle, rerun properties, and dependencies. For more information, see Configure time properties and Configure same-cycle scheduling dependencies.

Note
You must set the Rerun attribute and Parent Nodes properties for the node before you can commit it.
Configure resource properties. For more information, see Configure resource properties. When accessing the internet or a virtual private cloud (VPC), select a resource group for scheduling that provides network connectivity to the target node. For more information, see Network connectivity solutions.

Step 4: Debug the task code

(Optional) Select a resource group and assign values to custom parameters.
- On the toolbar, click the icon. In the Parameter dialog box, select the resource group to use for debugging.
- If your code uses scheduling parameters, assign test values to them here for debugging. For more information about the parameter assignment logic, see What are the differences in value assignment logic for Run, Advanced Run, and smoke testing in the development environment?.
Save and run the SQL statements.

In the toolbar, click the (Save) icon to save the statements. Then, click the (Run) icon to run the task.
(Optional) Perform smoke testing.

You can perform smoke testing in the development environment either during or after committing the node. For more information, see Perform smoke testing.

Next steps

Submit and deploy the node task.
1. Click the (Save) icon in the toolbar to save the node.
2. Click the (Submit) icon in the toolbar to submit the node task.
3. In the Commit Node dialog box, enter a Change Description.
4. Click Determine.
If you use a standard-mode workspace, you must publish the submitted task to the production environment. To do so, click Task Publishing on the left side of the top menu bar. For more information, see Publish tasks.
View the periodically scheduled task.
1. In the upper-right corner of the editor, click O&M Personnel to go to Operation Center in the production environment.
2. View the running periodically scheduled task. For more information, see Manage periodic tasks.
To view more details about periodically scheduled tasks, click Operation Center in the top menu bar. For more information, see Operation Center overview.