Image understanding

Prerequisites

• The workflow feature is available only after the unstructured data feature is purchased for your instance.

• You have created the workflow. For more information, see Create a workflow.

Background information

• The Image Understanding operator supports the following file formats: JPG, PNG, BMP, and WebP.

• By default, the Image Understanding operator is assigned 0.5 cores and 1 GB of resources.

Permissions

In a Basic project, the following roles can use any dataset within the project as an input or output dataset for the operator: project administrators, developers, analysts, and custom roles that are assigned the Dataset - Use permission.

Procedure

1. In the top menu bar of the Dataphin home page, choose Developer > Data Development.

2. In the top menu bar of the Developer page, select a Project.

3. In the navigation pane on the left, choose Data Processing > Workflow. In the Workflow list, click the target workflow to open its canvas. From the Operator Library on the left, drag the Image Understanding operator to the canvas. In the Image Understanding panel on the right, configure the following parameters.

   Parameter		Description
   Step Name		Enter a name for the operator. The name can contain any characters and be up to 256 characters long.
   Input Configuration	Dataset	Select the dataset to process. Only Hybrid Datasets are supported. After you select a dataset, select a Version. By default, the first version of the dataset is selected. If this operator is a downstream node of another operator, the selected dataset is typically the same as the output dataset of the upstream operator.
   	Input Field	The storage URL of the image to process. Select a source field from the input dataset. You can only select fields for which the Is URL parameter is set to Yes.
   	Filter Condition (Optional)	Use SQL syntax. Enter only the filter condition that follows the `WHERE` clause. You can use system global variables, such as the data timestamp `${bizdate}`.
   Model Configuration	Model	Specify the model for the operator. You can only select models that are enabled and have a model type that includes Vision (Image and Video) in Intelligent Application Management > Model Configuration. Note Only models that are compatible with the OpenAI protocol can be called.
   	Prompt	Enter a prompt. The prompt serves as an instruction for the interaction with the Large Language Model (LLM). Use prompts to define task goals, set response roles, and specify output formats. This guides the model to generate accurate and expected results. The default prompt is Please provide a detailed description of this image. You can drag the lower-right corner of the text box to resize it.
   	Multi-column Output (Optional)	This feature is disabled by default. If you enable it, the system automatically appends an output format constraint to the end of the prompt. This ensures that the model always generates a response in the specified JSON format. Do not include content related to the output format in your prompt to avoid conflicts with the system constraint. When enabled, a list of multi-column output fields is displayed. You can manage the fields in the following ways: Add Field: Click to open the Add Field dialog box. In the dialog box, manually configure the Name, Type, Description, and Example fields. Click OK to add a new field row to the list. Import JSON: Click to open the Import JSON dialog box, where you can directly enter JSON content. Existing fields from the list are automatically populated in the JSON, which you can then edit. After you confirm that the content is correct, click Confirm. Delete field: Click the Delete icon to delete the corresponding field.
   	Max Tokens (Optional)	Limits the maximum number of tokens in the model's output. If the generated content exceeds this value, generation stops early. The default value is 4096. This field can be left empty.
   	Web Search, Deep Thinking (Optional)	Disabled by default. You can enable them manually. Deep thinking can be enabled only if the selected model supports it. This means the model is supported by the Thinking Mode in Intelligent Application > Model Call Configuration.
   Output Configuration	Dataset	You can select Source Dataset or Other Dataset. If the metadata table of the source dataset does not have a primary key, the output dataset defaults to other dataset. When you select Other Dataset, you must also specify a Hybrid Dataset and its Version as the output dataset. By default, the first version of the dataset is selected. You can also select other versions of the source dataset. When you select Source Dataset, the metadata generated by the operator is written to the metadata table that corresponds to the same version of the source dataset.
   	Loading Policy	If the metadata table of the selected output dataset has a primary key, the default policy is to update on primary key conflict. If no primary key exists, the default policy is to append data. Append Data: Appends new data directly to the target table. An error is reported if a primary key conflict or constraint violation occurs. Update On Primary Key Conflict: When a primary key or constraint conflict occurs, the entire row of existing data with the duplicate primary key is first deleted, and the new data is then inserted. Overwrite: Deletes the data in the target table and then writes the new data. Note When the output dataset is set to Source Dataset, the only supported loading policy is Update on Primary Key Conflict.
   	Field Mapping	If Multi-column Output is disabled, only the image_content field is output. If Multi-column Output is enabled, the fields configured in the multi-column output list are output. By default, the operator's output fields are mapped to fields with the same names in the output dataset. You can also manually select destination fields for custom mapping. In addition, you can map pass-through fields from the source dataset to the destination dataset. You can manage field mappings in the following ways: Add Output Field: Adds a new, empty row for field mapping. This feature is supported only when you select Other Dataset as the output dataset. Batch Mapping: Click to select batch manual mapping, same-name mapping, or same-row mapping. This feature is supported only when you select Other Datasets for the output dataset. Delete mapping: You cannot delete the field mappings for the image_content field or fields in the multi-column output list here. When you delete a field from the multi-column output list, its corresponding field mapping is automatically deleted. For output fields that you manually add, you can click the Delete icon to delete the field mapping for the corresponding row.

4. After completing the configuration, click Save in the menu bar at the top of the canvas.

   After you configure a single operator, you can continue to select upstream or downstream operators to configure the entire workflow.