Manage offline computing tasks

Updated at:

The offline computing task list in Dataphin lists all offline computing tasks in the platform. It displays basic information and management options for each task. This topic describes these management features.

Procedure

  1. In the top navigation bar of the Dataphin homepage, choose R&D > Data Development.

  2. On the Develop page, select a project from the top navigation bar. If you use Dev-Prod mode, you must also select an environment.

  3. In the left-side navigation pane, choose Data Processing > Computing Task.

  4. At the bottom of the computing task list, click Computing Task List to open the Offline Computing Task List tab.

  5. On the Offline Computing Task List tab, perform the operations described in the following table.

    Actions

    Description

    Filter

    You can filter tasks by entering a Computing Task ID or Computing Task Name to quickly find a specific task. You can also select the Developed by Me checkbox to filter tasks. For more options, click Filter to view advanced filter criteria.

    • Task type: Filter by the task types supported by the current compute engine.

    • Scheduling method: Filter by the scheduling method of the offline computing task. Options include Normal Scheduling, Dry Run, Paused, and Manual Task. For more information, see Scheduling method overview.

    • Developer in charge: Filter by the developer in charge of the task.

    • Locker: Filter by the user holding the task's edit lock. You can select multiple project members.

    • Last commit status: Filter by the status of the last commit. Options include Submitting, Succeeded, Failed, and Succeeded with Risks.

    • Output table: Filter by the name of the task's output table.

    • Creation time: Filter by the task's creation time.

    • Referenced compute template: Filter by the compute template that the task references.

    • Task status: Filter by the development status of the task. Options include Draft, Developing, Submitted, and Submitting.

    • Scheduling cycle: Filter by scheduling cycle. Options include Minute, Hour, Day, Week, Month, and Year.

    • O&M engineer: Filter by the O&M engineer responsible for the task.

    • Published: Filter by whether the task has been published.

    • Conditional scheduling: Filter by whether conditional scheduling is Enabled or Disabled for the task.

    • Last modified time: Filter by the last modified time of the task. You can choose from the last 3 days, 7 days, 1 month, or a custom time range.

    • Tag: Filter by one or more task tags.

    • Parent directory: Filter by the immediate parent directory of the task. If you select Include tasks in subdirectories, the results will include all tasks in the selected directory and its subdirectories.

      Note

      If directory permissions are enabled, you can only select directories to which you have access. The list shows only tasks that you are authorized to view.

    Click Reset to clear all filter criteria and restore the default view.

    View batch operation records

    Click View Batch Operation Records. In the Batch Operation Records panel, click the Commit or Other Operations tab to view the corresponding records.

    Create computing task

    Click + Create Computing Task and select the type of offline computing task to create.

    Configure list headers

    Click the image icon above the list to configure the list headers by selecting which columns to display.

    Refresh

    Click the image icon above the list to refresh the current list of offline computing tasks.

    Open task

    In the Actions column for the target offline computing task, click the image icon to open the task in a new tab. In the task tab, you can edit and configure the task.

    Acquire lock

    Acquire the edit lock for an offline computing task.

    • Single operation: In the Actions column for the target task, click the image icon to acquire its edit lock.

    • Batch operation:

      1. In the offline computing task list, select multiple tasks, or select the checkbox next to Select at the bottom of the page to select all tasks.

      2. Click Batch Acquire Locks at the bottom of the page to acquire the edit locks for all selected tasks.

    Note

    You cannot release locks from the offline computing task list. To release a lock, you must open the tab for that task.

    Locate in directory

    In the Actions column for the target task, click the image icon to locate the task in the computing task directory tree on the left.

    Transfer developer in charge

    Change the developer in charge for an offline computing task to another user.

    • Single operation:

      1. In the Actions column for the target task, click the image icon and select Transfer Developer in Charge.

      2. In the Transfer Developer in Charge dialog box, select the target user. Then, click OK.

    • Batch operation:

      1. Select multiple tasks in the list, or select the checkbox next to Select at the bottom of the page to select all tasks.

      2. Click the image icon at the bottom of the page and select Batch Transfer Developer in Charge.

      3. In the Batch Transfer Developer in Charge dialog box, select the target user. Then, click OK.

    Go to O&M

    Navigates to the O&M center for the task.

    In the Actions column for the target task, click the image icon and select Go to O&M to go to the task O&M page. For more information, see View and manage script-based tasks and View and manage manual tasks.

    Modify referenced template version

    You can modify the referenced compute template version for offline tasks that use one.

    • Single operation:

      1. In the Actions column for the target task, click the image icon and select Modify Referenced Template Version.

      2. In the Modify Referenced Template Version dialog box, select a Template Version. Then, click OK.

    • Batch operation:

      1. In the list, select multiple computing tasks that use the same compute template.

      2. Click the image icon at the bottom of the page and select Batch Modify Referenced Template Version.

      3. In the Batch Modify Referenced Template Version dialog box, select a Template Version. Then, click OK.

    Note

    Changes to the template version take effect only after the task is submitted and published.

    Take offline

    You can take online computing tasks offline.

    • Single operation:

      1. In the Actions column for the target task, click the image icon and select Take Offline.

      2. In the Take Offline dialog box, enter Remarks (up to 128 characters).

      3. Click OK and Submit.

    • Batch operation:

      1. Select multiple tasks in the list, or select the checkbox next to Select at the bottom of the page to select all tasks.

      2. Click the image icon at the bottom of the page and select Batch Take Offline.

      3. In the Batch Take Offline dialog box, enter Remarks (up to 128 characters).

      4. Click OK and Submit.

    Note
    • You cannot take a task offline if it has a cross-tenant downstream task.

    • The operation fails if the task has downstream dependencies.

    • After a successful submission, publish the task to the production environment from the to-be-published list.

    Take offline and delete

    You can take offline and delete computing tasks that are online.

    • Single operation:

      1. In the Actions column for the target task, click the image icon and select Take Offline and Delete.

      2. In the Take Offline and Delete dialog box, enter Remarks (up to 128 characters).

      3. Click OK and Submit.

    • Batch operation:

      1. Select multiple tasks in the list, or select the checkbox next to Select at the bottom of the page to select all tasks.

      2. Click the image icon at the bottom of the page and select Batch Take Offline and Delete.

      3. In the Batch Take Offline and Delete dialog box, enter Remarks (up to 128 characters).

      4. Click OK and Submit.

    Note
    • This operation fails if the task has dependent downstream nodes. For tasks published to the production environment, this operation only takes them offline and does not delete them.

    • After the submission is successful, you can go to the to-be-published list to publish the object to the production environment.

    • If the Allow deleting published objects in the development environment option is disabled for your tenant under R&D Platform Settings > Take Offline and Delete, you cannot take offline and delete online development objects in a Dev-Prod project.

    Delete

    You can delete computing tasks that are in the Draft state.

    In the Actions column for the target task, click the image icon and select Delete. In the Delete dialog box, enter Remarks, and then click OK and Submit.

    Note

    If the Allow deleting published objects in the development environment option is disabled for your tenant under R&D Platform Settings > Take Offline and Delete, you cannot delete a draft object in a Dev-Prod project if a corresponding production object exists.

    Batch modify tags

    You can replace or append tags for multiple offline computing tasks at once.

    1. Select multiple tasks in the list, or select the checkbox next to Select at the bottom of the page to select all tasks.

    2. Click the image icon at the bottom of the page and select Batch Modify Tags.

    3. In the Batch Modify Tags dialog box, select a Modification Type and the desired Tags. If you select Replace, all existing tags on the selected tasks are removed and replaced with the new tags. If you select Append, the new tags are added to the existing tags on the selected tasks.

    4. Click OK to apply the tag modifications.

    Batch move

    You can move multiple offline computing tasks to another directory simultaneously.

    1. Select multiple tasks in the list, or select the checkbox next to Select at the bottom of the page to select all tasks.

    2. Click the image icon at the bottom of the page and select Batch Move.

    3. In the move dialog box, select the target Directory and enter a folder Name.

      The folder name can be up to 256 characters long and cannot contain the following characters: vertical bar (|), forward slash (/), backslash (\), colon (:), question mark (?), angle brackets (<>), asterisk (*), or double quotation mark (").

    4. Click OK to move the tasks.

    Batch scheduling configuration

    Important
    • Batch scheduling configuration applies only to periodic tasks.

    • After batch configuration, the existing scheduling properties of the selected periodic tasks will be overwritten. Proceed with caution. If a selected task has conditional scheduling enabled, setting the scheduling type to Dry Run or Paused will disable conditional scheduling for all selected tasks.

    You can modify the scheduling configuration for multiple offline computing tasks at once. You can batch-modify basic scheduling and conditional scheduling settings independently.

    • Batch modify basic scheduling: When enabled, you can batch-modify the Configuration Method, Scheduling Type, and Scheduling Cycle for the selected tasks. For configuration details, see Configure scheduling for an offline pipeline.

    • Batch modify conditional scheduling: This option is disabled by default. When enabled, you can batch-modify the Configuration Method and Scheduling Conditions for the selected tasks. For configuration details, see Conditional scheduling.

      Note
      • Conditional scheduling applies only to tasks with a scheduling method of Normal Scheduling.

      • When the configuration method is set to Custom, cross-node parameters are not supported as conditions.

      • The Start Time parameter applies only to tasks with a Day, Week, or Month scheduling cycle.

    Batch dependency configuration

    You can configure dependencies for multiple offline computing tasks at the same time.

    1. Select multiple tasks in the list, or select the checkbox next to Select at the bottom of the page to select all tasks.

    2. Click the image icon at the bottom of the page and select Batch Dependency Configuration.

    3. In the Batch Dependency Configuration dialog box, add, delete, or edit dependencies. To add multiple upstream dependencies, click Add Dependency and select Physical Node, Logical Table Node, or Cross-tenant Dependency. For details about scheduling dependencies, see Configure scheduling dependencies for an offline task.

    4. Click OK to complete the configuration.

    Note

    The list in the batch dependency configuration dialog box shows only manually added upstream dependencies that are common to all selected tasks (meaning the dependency object and settings are identical). Any additions, deletions, or edits to dependencies apply to all selected tasks.

    You can add a cross-tenant dependency only if this feature is enabled for the current tenant.

    Batch parameter configuration

    You can configure parameters for multiple offline computing tasks at the same time. To add multiple parameters, click + Add Parameter.

    1. Select multiple tasks in the list, or select the checkbox next to Select at the bottom of the page to select all tasks.

    2. Click the image icon at the bottom of the page and select Batch Parameter Configuration.

    3. In the Batch Parameter Configuration dialog box, configure the parameters. To add more parameters, click + Add Parameter. For details about run parameters, see Configure run parameters for an offline task.

    4. Click OK to complete the configuration.

    Note

    The available parameters are a combination of all local variables from the selected tasks. The system overwrites the value of the corresponding parameter for each selected task. For example, if you set paramA=valueA, the system updates only the tasks that already contain the paramA parameter. Tasks without this parameter are not affected. You cannot batch-configure values for namespace variables or global variables.

    Batch run configuration

    You can configure the run settings for multiple offline computing tasks at the same time.

    1. Select multiple tasks in the list, or select the checkbox next to Select at the bottom of the page to select all tasks.

    2. Click the image icon at the bottom of the page and select Batch Run Configuration.

    3. In the Batch Run Configuration dialog box, configure the Run Timeout and Automatic Rerun upon Failure settings. For details about run configurations, see Configure compute task run settings.

    4. Click OK to complete the configuration.

    Important

    This action overwrites the existing run settings of the selected tasks. Proceed with caution.

    Change owning project

    You can change an offline computing task's owning project.

    • Single operation:

      1. In the directory tree on the left, click the image icon next to the target offline computing task and select Change Owning Project.

      2. In the Change Owning Project dialog box, select a Target Project, Directory, and Compute Template (if applicable), and enter a Task Name.

        • Target project: You can select a project within the current tenant for which you have development permissions. The target project must have the same mode (both Basic mode or both Dev-Prod mode) and the same engine type as the current project. You can search by project name.

        • Directory: Select a directory from the target project's offline computing task directory tree. The default is / (the root directory).

        • Compute template: If the task references a compute template and a template of the same task type exists in the target project, you can select a new compute template and version for the task.

        • Task name: The name of the task after it is moved to the target project. It defaults to the current task name. You must change the name if a task with the same name already exists in the target project.

      3. Click OK.

    • Batch operation:

      1. Select multiple tasks in the list, or select the checkbox next to Select at the bottom of the page to select all tasks.

      2. Click the image icon at the bottom of the page and select Change Owning Project.

      3. In the Change Owning Project dialog box, select a Target Project, Directory, and Compute Template.

        • Target project: You can select a project within the current tenant for which you have development permissions. The target project must have the same mode (both Basic mode or both Dev-Prod mode) and the same engine type as the current project. You can search by project name.

        • Directory: Select a directory from the target project's offline computing task directory tree. The default is / (the root directory).

        • Compute template: If the selected tasks are of the same type and some of them reference an offline compute template, you must select a compute template of the same task type and a version from the target project.

      4. Click OK to change the project.

    After you change a task's owning project:

    • The task ID, code, and upstream and downstream dependencies remain unchanged. The system also updates the owning project for the task's output table and any downstream tasks that reference it.

    • You must go to the target project to submit and publish the task.

    • Historical versions of the task in the original project are retained, but you cannot roll back to a pre-migration version. If a batch operation to change the owning project fails, you can view the failure logs in the batch operation records.