DocsICP FilingConsole
官方文档
首页

Create and use a Data Lake Analytics node

更新时间：
复制 MD 格式
产品详情
我的收藏

DataWorks allows you to create a Data Lake Analytics node to build an online extract, transform, and load (ETL) process.

Background information

Data Lake Analytics nodes are used to connect to Alibaba Cloud Data Lake Analytics (DLA). For more information about DLA, see What isData Lake Analytics

Note

Tasks on Data Lake Analytics nodes can be run on serverless resource groups or old-version exclusive resource groups for scheduling. We recommend that you run tasks on serverless resource groups. For more information about how to purchase and use a serverless resource group, see Use serverless resource groups.

Limits

Data Lake Analytics nodes are supported in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), China (Hong Kong), Japan (Tokyo), Singapore, Germany (Frankfurt), UK (London), US (Silicon Valley), and US (Virginia).

Procedure

  1. Go to the DataStudio page.

    Log on to the DataWorks console. In the target region, click Data Development and O&M > Data Development in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Development.

  2. Hover over the Create icon and choose Create Node > Custom Rule > Data Lake Analytics.

    Alternatively, open the target workflow, right-click Custom Rule, and choose Create Node > Data Lake Analytics.

  3. In the Create Node dialog box, configure the Name and Path parameters.
    Note The node name must be 1 to 128 characters in length and can contain letters, digits, underscores (_), and periods (.).
  4. Click Confirm.

  5. Configure the Data Lake Analytics node.

    1. Select a data source.

      Select the target data source. If the data source you need is not in the drop-down list, click New data source on the right to go to the Data source management page and create one. For more information, see Configure a Data Lake Analytics (DLA) data source.

    2. Write SQL statements for the node.

      After you select a data source, write SQL statements based on the syntax that is supported by DLA. You can write data manipulation language (DML) or data definition language (DDL) statements.

    3. Click the Save icon in the top toolbar.

    4. Click the Run icon in the top toolbar to execute SQL statements.

    To change the resources for a test run on the Data Studio page, click the Advanced Run icon in the toolbar and select the target serverless resource group.

    Note

    A serverless resource group is required to access a data source that is deployed in a virtual private cloud (VPC). In this case, you must select a serverless resource group that is connected to the data source.

  6. In the right-side pane, click Scheduling to configure the node's scheduling properties. For more information, see Configure basic properties.

    You must select a serverless resource group that is connected to the Data Lake Analytics node to periodically schedule tasks on the Data Lake Analytics node.

    7. Save and commit the node.

    Note

    You must configure the Rerun attribute and Parent Nodes properties for the node before you can commit it.

    1. Click the Save icon in the top toolbar to save the node.

    2. Click the Submit icon in the top toolbar.

    3. In the Commit Node dialog box, enter a Change Description.

    4. Click OK.

    If you use a workspace in standard mode, click Deploy in the upper-right corner after you commit the node. For more information, see Deploy tasks.

  7. Perform O&M operations on the node. For more information, see Basic O&M for auto-triggered nodes.
Previous：Data push nodeNext：Data quality monitoring node