DocsICP FilingConsole
Documentation
Home Page DataWorks User Guide DataStudio (legacy) Node development Supported node types Algorithms Create and use PAI Designer nodes

Create and use PAI Designer nodes

Updated at:
Copy as MD
Product
My Favorites

When you want to run a Machine Learning Designer pipeline on a recurring schedule without manual intervention, create a PAI Designer node in DataWorks and attach the pipeline to it. DataWorks then triggers the pipeline automatically based on the scheduling configuration you define.

After completing these steps, you will have:

  • Created a PAI Designer node in DataStudio

  • Attached or built a Machine Learning Designer pipeline on the node

  • Configured periodic scheduling

  • Debugged and committed the node for production

Prerequisites

Before you begin, ensure that you have:

  • DataWorks authorized to access PAI. Perform one-click authorization on the authorization page. For policy details, see Role 1: AliyunServiceRoleForDataworksEngine. Only an Alibaba Cloud account or a RAM user with the AliyunDataWorksFullAccess policy can perform this authorization

  • A workflow created in DataStudio. All development in DataStudio is organized into workflows. Create one before creating a node. For instructions, see Create a workflow

Step 1: Create a PAI Designer node

  1. Go to the DataStudio page. Log in to the DataWorks console. In the top navigation bar, select the target region. In the left-side navigation pane, choose Data Development and O&M > Data Development. Select the target workspace from the drop-down list, then click Go to Data Development.

  2. Right-click the target workflow and choose Create Node > Algorithm > PAI Designer.

  3. In the Create Node dialog, set the Name and Path parameters, then click Confirm. The node is created and its configuration tab opens.

Step 2: Develop tasks on the PAI Designer node

Attach or build a Machine Learning Designer pipeline

To load an existing pipeline, create it in PAI first, then search for it by name in the node configuration tab.

To build a pipeline directly on the node, use one of the following methods:

Method Description
Blank pipeline Add components and use drag-and-drop to assemble a model from scratch. See Create a blank pipeline.
Preset template Start from a Machine Learning Designer preset template, then modify components or their configurations. See Create a pipeline from a preset template.
Custom template Use a stable pipeline saved as a custom template — shared within your workspace. See Create a pipeline from a custom template.
For step-by-step examples of building pipelines for scheduled use, see Use DataWorks tasks to schedule pipelines in Machine Learning Designer (preset template) and Create a custom pipeline (custom pipeline).

Use scheduling parameters

Scheduling parameters inject dynamic values into node code at run time.

Define a variable in the node code using the ${Variable} format, then assign a value to it in the Scheduling Parameter section on the Properties tab. For supported formats, see Supported formats of scheduling parameters.

Example:

--command='echo '\''${Variable}'\'';' \ --Scheduling parameters are supported.

Step 3: Configure scheduling properties

To run the node on a recurring schedule, click Properties in the right-side navigation pane of the node configuration tab.

Parameter Required Description
Rerun Required Defines the rerun policy when the node fails or is manually retriggered. Must be set before committing.
Parent nodes Required Declares upstream dependencies. Must be set before committing.
Other scheduling fields Optional Frequency, start time, and other options.

For all scheduling configuration options, see Overview.

Step 4: Debug the node

Run the node in the development environment to verify it behaves as expected before committing.

  1. (Optional) Select a resource group and assign test values to scheduling parameters. Click the 高级运行 icon in the top toolbar. In the Parameters dialog, select the scheduling resource group for the debug run. If the node code uses scheduling parameters, assign test values to those variables. For details on value assignment across run modes, see What are the differences in the value assignment logic of scheduling parameters among the Run, Run with Parameters, and Perform Smoke Testing in Development Environment modes?

  2. Save and run the node. Click the 保存 icon to save, then click the 运行 icon to run the node.

  3. (Optional) Perform smoke testing. When you commit the node or after you commit the node, you can perform smoke testing in the development environment to confirm the node runs as expected. For instructions, see Perform smoke testing.

Step 5: Commit and deploy the node

After the node is configured and debugged, commit it to activate scheduling.

Important

Configure Rerun and Parent nodes on the Properties tab before committing.

  1. Click the 保存 icon to save the latest changes.

  2. Click the 提交 icon to commit. The Submit dialog opens. In the Submit dialog:

    • Fill in Change description to summarize what changed.

    • Select whether to request a code review after committing.

    Tip: Code review ensures code quality and prevents scheduling failures. When enabled, deployment requires reviewer approval. See Code review.

  3. (Standard mode only) Deploy to the production environment. If your workspace uses standard mode, click Deploy in the upper-right corner of the configuration tab after committing. For details, see Deploy nodes.

What's next

After the node is deployed, DataWorks runs the pipeline automatically on the schedule you configured.

To monitor runs, click Operation Center in the upper-right corner of the configuration tab. For details, see View and manage auto triggered tasks.

Previous：AlgorithmsNext：Create and use a PAI DLC node