Offline workflow properties and operator resource configuration
Prerequisites
You can use offline workflows only after you purchase the unstructured data feature for the current instance.
Workflow properties
In the top menu bar of the Dataphin home page, choose Development > Data Development.
In the top menu bar of the Development page, choose Project.
In the navigation pane on the left, choose Data Processing > Workflow. In the Workflow list on the left, click the target offline workflow. In the sidebar on the right, click Property.
Configure the Basic Information, Scheduling Configuration, Scheduling Dependencies, Runtime Parameters, Runtime Configuration, and Resource Configuration for the current offline workflow. The configuration process is the same as for offline computing nodes.
In the Runtime Configuration of an offline workflow, you can also configure how to handle operator processing exceptions. You can choose to Ignore abnormal data and continue execution or Stop the node after an exception.
Ignore abnormal data and continue execution: If an operator encounters an exception, the system skips the data that caused the exception and continues to run.
Stop the node after an exception: If an operator encounters an exception, the workflow stops immediately.
Operator resource configuration
To prevent out-of-memory (OOM) errors when a workflow runs, you can configure CPU and memory resources for each operator. When you submit a workflow, the system checks the resources. If the resources requested by the workflow exceed the maximum resources of the resource group, the submission fails.
In the top menu bar of the Dataphin home page, choose Development > Data Development.
In the top menu bar of the Development page, choose Project.
In the navigation pane on the left, choose Data Processing > Workflow. In the Workflow list on the left, click the target workflow to open its canvas. On the canvas, click the target operator. In the operator configuration panel on the right, click Resource Configuration.
In the Resource Configuration dialog box, configure the CPU and memory. The maximum resources for a single operator are 4 cores and 16,384 MB (16 GB). The total CPU and memory for all operators in the workflow must be less than or equal to the maximum resources of the assigned resource group.
The default memory value depends on whether you select the system built-in model or the multi-modal Large Language Model (LLM). When you switch models, the system initializes the resource configuration for any unconfigured operators. If you have already configured the resources for an operator, your settings are preserved after the switch.
When you are finished, click OK.