Multimodal metadata management

Prerequisites

• You have an OSS bucket that contains data.

• The operating account must have the Workspace Administrator or DataStudio role in the corresponding DataWorks workspace and have read permissions on the target OSS.

Step 1: Register a dataset

1. In the DataWorks console, click Go to Data Map.

2. In the left-side navigation pane, click the icon and navigate to Data Map > Data Catalog.

3. In the directory tree on the left, click DataSet, select the target workspace, and click Create Dataset on the workspace details page.

4. In the dialog box, enter the following information to register the dataset.

   Parameter	Description	Example
   Name	A clear, business-friendly name.	2025-Gaokao-Corpus-TrainingSet
   Storage Type	Your file storage service.	For this example, select OSS.
   Content Type	Helps the system better parse and preview the data.	Text. If you are unsure, select Common.
   Description	Describe the data content, purpose, source, and other relevant details.	Raw corpus of college entrance exam questions for a smart tutoring application, updated daily.
   OSS Path	The path to the file or folder. Click the folder icon on the right to select an OSS path. Note Best practices for OSS path design: Isolate by business domain or region: /gaokao/math/, /iot/sensor/. Partition folders by year or date: /gaokao/math/2025/. The folder path used to register the dataset should be a prefix path, such as /gaokao/2025/. Avoid oversized directories: We recommend fewer than 100,000 files per path.	/Demo/gaokao/train/
   Default Mount Path	Select the default mount path for the data. This path will be used by DataWorks task nodes that require mounted files, such as Python and Shell tasks.	/mnt/data/gaokao/train

   A blue banner at the top of the form indicates that after the dataset is created, users with the data development role have read and write permissions by default. The system automatically populates the workspace and owner fields. On the right are two tabs: Dataset Configuration and Version Configuration. The Basic Information section on the Dataset Configuration tab is expanded by default.

For more detailed instructions, see Manage datasets.

Step 2: Search and preview

1. On the Data Map page, click the top search bar, select Dataset as the type, and enter a keyword in the search box (for example, the dataset name Gaokao, description content corpus, or path fragment gaokao).

2. Find your dataset in the search results and click it to go to the details page.

3. The details page shows the following:

   • Basic Information: Metadata such as the owner, storage path, and creation time.

   • Data Preview: By default, data is displayed as a Smart Table. If the file format is supported, the system automatically displays a preview of the file's content. Double-click a field to view its details. You can also switch to the Raw Text mode.

     For information about data formats supported for preview, see Overview of metadata management capabilities.

   • Data lineage: View which task nodes use this dataset.

Step 3: Use dataset in tasks

Use the dataset as an input to decouple code from physical storage paths.

1. In DataStudio, create or open a Python, Shell, or Notebook node.

2. In the Scheduling > Scheduling Policy > Dataset section on the right side of the node configuration, click + Add to add a dataset to mount.

3. Click Dataset and select the dataset you registered in Step 1 from the list that appears.

4. In your code, you can access the data path by using system-injected environment variables. For more information, see Using datasets.

   Important

   At runtime, DataWorks mounts the dataset path to the task's local file system (read-only), so you do not need to worry about storage replication or security. Access is still controlled by the native permissions of OSS or NAS.

For more detailed instructions, see Use datasets.

FAQ

• Q: Why can't I preview my data?

  • Preview not supported: The file format (such as Parquet or video) might not support preview. This is expected behavior. You can go to the OSS console to view the file list.

  • File read failure: Check whether your account has read permissions for the OSS path, or if the compressed file is encrypted.

  • Blank preview: The first file in the specified path may be empty or corrupted.

• Q: Can a single dataset version correspond to multiple OSS paths?

  • No. A dataset version can correspond to only one root path.

• Q: Can I register the same OSS path to multiple datasets?

  • This is not recommended, as it can interfere with subsequent data lineage analysis.

• Q: Can the previewed content be exported?

  • You cannot export content directly from the preview interface. However, you can mount the dataset to a Notebook node and then write code to read and export the data.

1. Create an Advanced image dataset

Goal: Create a container to store your image data and its enhanced metadata.

1. Create the dataset

   • Path: Go to the PAI console. In the list of workspaces, select the one that has the same name as your DataWorks workspace, and then navigate to AI asset management > Dataset.

   • Click Create Dataset and fill in the following key parameters:

     • Type: Ensure that you select Advanced.

     • Content Type: Select Image.

     • OSS Path: Specify the OSS directory where your raw images are stored.

   • Give the dataset a meaningful name (for example, autonomous_driving_image_dataset_test) and click OK.

     Important

     This name is used as the dataset name in DataWorks.

2. Initialize metadata

   • Path: Return to the dataset list and go to the details page of the dataset you just created.

   • On the Version Details > Metadata tab, click Update Now.

   • PAI starts a background task to scan all image files in your OSS path and collects basic metadata. Wait for the task to complete.

2. Enable tag filtering

Goal: Use a large model to automatically generate business tags for your images.

1. Create an intelligent tagging model connection: To allow PAI tasks to access external large model services, you must create a connection.

   • Path: Go to AI asset management > Connect.

     • Click the Model Service tab and click Create Connection.

     • Select Model Studio Large Model Service and enter your API key.

       

2. Define the tag schema

   • Path: Go to AI asset management > Dataset and open the Intelligent Tag Definition tab.

   • Click Create Intelligent Tag Definition to configure your tagging rules.

     • Guiding Prompt: Give the model a role and context, for example: "You are an experienced traffic scene analyst..."

     • Tag Definition: Use JSON format to define your tags and their descriptions. For example:

       {
         "speed_bump": "Typically yellow and black striped, a raised bar across the road used to slow down vehicles.",
         "non_motor_vehicle": "Includes bicycles, e-bikes, wheelchairs, etc."
       }

3. Run the intelligent tagging task

   • Path: Go to AI asset management > Tasks > Dataset Tasks.

   • Select the dataset you created in Step 1, and then click Create Node > Intelligent Tagging.

     

   • Configure the following parameters:

     • Intelligent Tagging Model Connection: Select the Model Studio model connection you created.

     • Intelligent Tagging Model: Select Qwen-VL Max or Qwen-VL Plus.

     • Intelligent Tag Definition: Select the tag definition you just created.

     • Tagging Mode: For the initial tagging job, select Full.

   • Click OK to start the task.

3. Enable semantic search

Goal: Generate vector features for your images and store them in a database for semantic search or search by image.

1. Create a semantic index model connection

   • Go to AI asset management > Connect and open the Model Service tab.

   • Click Create Connection, select General Multimodal Embedding Model Service, and link it to your deployed GME EAS service.

     If you have not deployed an EAS service, go to the console home page, navigate to Quick Start > Model Gallery, and deploy the service.

     

2. Create a vector database connection

   • Go to AI asset management > Connect > Database.

   • Click Create Connection, select your database type (such as Milvus), and enter the instance's URI, token, and database name.

     For a guide on how to fill in these fields, see 2.3 Create a vector database connection.

3. Configure the index

   • Path: Return to the dataset details page and find the Index Configuration section.

   • Click Edit and configure the information.

     

     • Index Model Connection: Select the GME model service connection you created.

     • Index Database Connection: Select the vector database connection you created.

     • Index Database Table: Enter the name of the table or collection in your vector database used for storing vectors (for example, traffic_image_embeddings).

4. Run the semantic index task

   • Path: Go to AI asset management > Tasks and open the Dataset Tasks tab.

   • Select the OSS dataset you created and click Create Node > Semantic Indexing.

     

   • Select the Dataset Version to process and click OK to start the task.

1. Discover the dataset

First, find your dataset in DataWorks.

1. In the DataWorks console, go to Go to Data Map. In the left-side navigation pane, click the icon to go to the search page.

2. In the search bar at the top, enter keywords related to your dataset. You can search in several ways:

   • By name: For example, enter autonomous driving.

   • By description: For example, enter traffic or image.

   • By path: For example, enter a part of the OSS path, such as traffic_images.

3. The system displays all matching datasets in real time. You can:

   • Sort: Sort the results by Correlation or Update Time.

   • Switch views: Switch between Gallery and List modes. Hover over an item to quickly preview its basic information.

   • Refine results: Use the filter panel on the left to narrow your results by data source, workspace, and other criteria.

2. Explore dataset details

Once you find your target dataset, click it to open its details page.

• View core properties
  On the details page, you can see the dataset's Name, Description, Owner, Last Updated, and other metadata. The Dataset Version list allows you to trace and view the detailed configuration of different versions, such as the storage Path and Default Mount Path.
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  

• Read usage instructions
  If the dataset owner has provided detailed usage instructions, you can find user manuals, field dictionaries, sample code, or compliance statements here to help you better understand and use the data.
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  

3. Interact with data

This is the core step for using the PAI-enhanced capabilities. On the dataset details page, switch to the View Data tab.

Prerequisites

Before you begin, ensure that the following conditions are met:

Key operations

1. Create a document knowledge base

   • Path: Go to LangStudio, select a workspace, and then select the Knowledge Base tab.

   • Click Create Knowledge Base and configure the following core parameters:

     • Knowledge Base Name: Enter a name for your knowledge base, for example, data_governance_kb.

     • Knowledge Base Type: Select Documents.

     • Data Source OSS Path: Specify an OSS directory to store your source documents.

     • Vector Database Type: Select Milvus.

   • After the knowledge base is created, click Upload on the details page to upload your documents (such as PDF and Word files) to the data source.

     Recommended document naming convention: {domain}_{topic}_v{version}, for example, finance_compliance_v2.

     

2. Build a vector index

   • Path: Go to the details page of the knowledge base that you just created.

   • In the upper-right corner of the page, click Update Index. The system starts a background task that automatically performs the following actions:

     Important

     To automatically process newly added documents, you need to configure a scheduled trigger to rebuild the vector index in LangStudio.

     1. Document parsing and chunking: Splits long documents into semantically complete text chunks.

     2. Vectorization (Embedding): Calls the built-in embedding model to convert each text chunk into a numerical vector.

     3. Storage and indexing: Stores the generated vectors in your selected vector database (Milvus) and creates an index.

   • Wait for the task to complete until the File Status changes to "Indexed" or the task status in Operation Records shows "Succeeded".