Multimodal Data Ingestion

With the multimodal data ingestion feature of AnalyticDB for MySQL, you can upload documents, images, videos, and audio to AnalyticDB for MySQL Lake Storage. The system handles file discovery, model inference, semantic understanding, vector generation, and ingestion end-to-end, writing results to a multimodal table for hybrid vector, full-text, and scalar retrieval with SQL.

How it works

With the AnalyticDB for MySQL multimodal data ingestion feature, you can:

End-to-end multimodal processing: Upload files to AnalyticDB for MySQL Lake Storage to obtain semantic tags and vector representations.
Unified multimodal storage and retrieval: Results are stored in AnalyticDB for MySQL's multimodal tables with object metadata, text summaries, and vector fields. Use SQL for hybrid keyword + semantic retrieval.

The processing flow:

Step	Description
Upload files	Upload files to the specified folder in AnalyticDB Lake Storage (Managed OSS).
Multimodal data ingestion	After you create a multimodal data ingestion job, the system automatically: Incremental file discovery: Detects new files and writes their object metadata (path, size, suffix, upload time) to the file_meta table. Multimodal processing: Reads new files from the file_meta table and processes them based on the job type. Data ingestion: Writes object metadata, processed text, and vectors to the destination table.
Data retrieval	Query vectors, full-text, and scalars with SQL.

Prerequisites

AnalyticDB for MySQL cluster inEnterprise Edition, Basic Edition, or Data Lakehouse Edition.
The AnalyticDB for MySQL cluster kernel version is 3.2.6 or later.

Note
To view and update the minor version, go to the Configuration Information section on the Cluster Information page in the AnalyticDB for MySQL console.
Create a Ray resource group with ENI (Elastic Network Interface) enabled. You can also modify an existing resource group to enable ENI. ENI is required for the ingestion job to write data to the destination table.
Deploy multimodal understanding models and embedding models in the AnalyticDB for MySQL inference service.
Submit a ticket to enable the multimodal data ingestion feature.

Procedure

Log on to the AnalyticDB for MySQL console. In the upper-left corner of the console, select a region. In the left-side navigation pane, click Clusters. Find the cluster that you want to manage and click the cluster ID.
In the left navigation pane, click Data Ingestion > Multi-Modal Data Ingestion.

Click Create Job and configure the following parameters.

Parameter	Description
Description	Enter a job description.
Data Source	Select ADB Lake Storage (Managed OSS).
Path	Select a folder in AnalyticDB Lake Storage for uploading multimodal files. If no Lake Storage exists, clickCreate to create a Lake Storage bucket.
Job Type	Select a job processing type: Multimodal understanding and vectorization (General): Performs LLM-based understanding and vectorization for documents, videos, images, and audio, then writes results to the destination table. Keyframe extraction (Video): Extracts PPT page keyframes based on visual changes and writes them to the destination table. Segment understanding (Video): Splits videos into segments, then writes model understanding results and vectors for each segment to the destination table.
Target Table	Enter the destination database name and table name. Important The destination table is automatically created based on the job type. The destination table must not exist when creating the job.
Ray Resource Group	Select an existing Ray resource group. Note This list only displays Ray resource groups with ENI network enabled.
Multimodal Understanding Model	Select a multimodal understanding model deployed on the AnalyticDB for MySQL inference service.
Embedding Model	Select an embedding model deployed on the AnalyticDB for MySQL inference service.
User Instruction Template	Select a built-in instruction template. Adjust it as needed.
User Instruction	Enter instructions to help the LLM understand and process file content.
Whitelist	When enabled, the vSwitch CIDR block of the AnalyticDB for MySQL cluster is added to the AnalyticDB for MySQL access whitelist, allowing data writes to the destination table.

Click Start Job.

After the job starts, it monitors the data source path for new files, processes them automatically, and writes results to the destination table.