Create a Shell node

Procedure

1. In the top menu bar of the Dataphin homepage, choose Develop > Data Development.

2. In the top menu bar of the Develop page, select a project. If you are using the Dev-Prod mode, also select an environment.

3. In the navigation pane on the left, choose Data Processing > Script Task. In the Script Task list, click the icon and select Shell.

4. In the Create Shell Node dialog box, configure the following parameters.

   Parameter	Description
   Task Name	Enter a name for the offline computing node. The name can be up to 256 characters long and cannot contain vertical bars (|), forward slashes (/), backslashes (\), colons (:), question marks (?), angle brackets (<>), asterisks (*), or double quotation marks (").
   Schedule Type	Select the Scheduling Type for the job. The Scheduling Type can be one of the following: Recurring Task: The node is periodically scheduled by the system. One-Time Task: The node must be triggered manually.
   Select Directory	Select the task folder. If no folder exists, you can create a new folder as follows: Above the list of computing nodes on the left, click the icon to open the Create Folder dialog box. In the Create Folder dialog box, enter a Name for the folder and select a Directory as needed. Click OK.
   Use Template	Turn on the Use Template switch to use a code template. If you turn on this switch, you must also select a Template and Template Version. Use a code template for efficient development. The code in a template node is read-only. You only need to configure the template parameters to complete code development. For more information, see Create an offline computing template.
   Third-party Python packages	Select one or more third-party Python packages to import. For more information, see Install Python Module. Note After you add a third-party module to a Python package, you must declare a reference in the node before you can import (import) the module in your code. You can edit the referenced module under Compute Node Properties > Third-party Python Package configuration item.
   Description	Enter a brief description of the task. The description cannot exceed 1000 characters.

5. Click OK.

6. In the code editor on the current Shell node tab, write the code for the Shell offline computing node. After you finish writing the code, click Run above the code editor.

7. Click Property in the sidebar on the right. In the Property panel, configure the node's Basic Information, Runtime Resource, Python Module, Dataset, Runtime Parameter, Scheduling Property (for recurring tasks), Schedule Dependency (for recurring tasks), Runtime Configuration, and Resource Configuration.

   • Basic Information

     Set the basic information for the scheduled node, such as its name, owner, and description. For more information, see Configure basic information for a node.

   • Runtime Resource

     Set the CPU and memory resources that are allocated to run the computing node. The default is 0.1 cores and 256 MB. For more information, see Configure runtime resources for an offline node.

   • Third-Party Python Packages

     Select the third-party Python packages to import. For more information, see Install a Python module.

   • Dataset

     Select the datasets to reference. You can reference up to five datasets. For more information, see Configure datasets referenced by Python or Shell nodes.

     Important

     • You can add datasets only after you purchase the unstructured data feature for the current instance.

     • Only Shell nodes in Basic projects support dataset referencing.

   • Runtime Parameter

     If your node calls parameter variables, you can assign values to them. The parameter variables are automatically replaced with their corresponding values during node scheduling. For more information, see Configure and use node parameters.

   • Scheduling Property (for recurring tasks)

     If the schedule type of the offline computing node is Recurring Task, you must configure its scheduling properties in addition to the Basic Information. For more information, see Configure scheduling properties.

   • Schedule Dependency (for recurring tasks)

     If the schedule type of the offline computing node is Recurring Task, you must configure its schedule dependencies in addition to the Basic Information. For more information, see Configure schedule dependencies.

   • Runtime Configuration

     You can configure a node-level runtime timeout period and a retry policy for failed node runs, based on your business scenario. If not configured, the node inherits the default tenant-level settings. For more information, see Configure runtime settings for a computing node.

   • Resource Configuration

     You can configure a scheduling resource group for the current computing node. When the node is scheduled, it uses the resource quota of this resource group. For more information, see Configure resources for a computing node.

8. On the current Shell node tab, save and submit the node.

   1. Click the icon above the code editor to save the code.

   2. Click the icon above the code editor to submit the code.

9. On the Submitting Log page, review the details in the Content to Submit and Pre-check sections, and enter any remarks. For more information, see Submit an offline computing node.

10. After you confirm, click OK and Submit.