Workflow-Dataphin(Dataphin)-阿里云帮助中心

A workflow is a core feature that automates processing unstructured data in a dataset. By orchestrating multiple operators in a workflow, you can build a complete data processing pipeline to handle various data types in batches, including audio, video, images, text, and documents.

Overview

A workflow is a customizable, visual orchestration tool for integrating complex transformation steps. It processes data from a source dataset, uses operators, and writes the results to a target dataset, creating analyzable, structured assets. There are two types of workflows: batch workflows and real-time workflows.

Real-time workflows are designed for unstructured data processing scenarios and provide event-driven automation capabilities. This solution overcomes the latency of batch processing and the difficulty of integrating with online systems, helping enterprises unlock data value in real time.

A workflow consists of one or more operators arranged in sequence, where each operator performs a specific data processing task. The output of an upstream operator can serve as the input for a downstream operator, forming a data processing pipeline. The general process for using a workflow is: Create a new workflow task > Add and configure operators > Configure workflow task properties > Run the workflow > Submit the workflow.

The workflow page has five parts: the top operation bar, the operator library, the canvas, the information pane on the right, and the bottom status bar.

Component	Description
①Top operation bar	Save: Saves the current changes to the workflow. Run: Executes the entire workflow. Running a workflow consumes Credit. When you click Run, if you have 100 or fewer Credits remaining, the workflow may fail. We recommend that you purchase a resource plan and try again. If you have 0 or fewer Credits remaining, the workflow fails. If your Credit balance runs out during a run, the workflow terminates immediately. You must purchase a resource plan and retry the run. Credit is still deducted for any operators that completed successfully. Submit: Submits the workflow to the production environment after passing submission checks. For more information, see Workflow submission description.
②Operator library	The operator library panel contains all available operators. You can search by keyword or filter by category. The library can be expanded or collapsed. Operators are grouped by function into the following categories: General: MD5 exact deduplication, basic file information, basic audio information, basic video information. Text: special character removal, non-compliant content replacement, private information masking, SimHash value calculation, text reasoning (LLM), multilingual text quality scoring, Simplified-to-Traditional Chinese conversion, HTML body extraction, text chunk splitting. Document: PDF parsing. Image: not safe for work (NSFW) detection, image aesthetic scoring, image OCR, image quality scoring, image understanding, and more. Audio: audio slicing, speech-to-text (ASR), audio timestamping, audio language detection, voice activity detection (VAD), audio transcoding, audio enhancement, audio quality scoring, speaker diarization (DIA). Video: video to audio extraction, video quality scoring. Vector: image embedding, text embedding. You can drag operators from the operator library onto the canvas.
③Canvas	The canvas is the main editing area for a workflow. After you drag operators from the operator library on the left onto the canvas, you can connect them to build a complete data processing flow. The bottom of the canvas has zoom controls to adjust the display scale. You can run, copy, or delete operators on the canvas. Run: Runs the current operator or runs from the root node to the current operator. Copy: Creates a copy of the operator with the same configuration. Delete: Deletes the current operator.
④Information pane	Properties: View the properties of a batch or real-time workflow, including basic information, run configurations, and resource configurations. For more information, see Batch workflow properties and operator resource configuration. Related: View information related to the current workflow. For more information, see Introduction to workflow-related objects. History: View the workflow's operation history. For more information, see View historical versions of a workflow.
⑤Bottom status bar	View console: After running a workflow, click to expand the console and view run information and raw logs. The run information includes the Credit consumed by successfully completed operators (both the total consumed Credit and the Credit consumed by each operator). Status: Displays the current status of the workflow. The status can be draft, developing, submitting, or submitted. Submitted details: For workflows that are submitting or have been submitted, click to view the submission record. For more information, see Workflow submission description. Last saved: Shows the time of the last save.