AI VibeFlow getting started

Step 1: Create a DataPipeline

A DataPipeline is the core object in AI VibeFlow. It defines the complete rules for reading raw data from the source bucket, calling a Model Studio model for vectorization, and writing the vectors to a vector bucket.

1. Log on to the OSS console.

2. In the left-side navigation pane, click AI VibeFlow.

3. On the AI VibeFlow page, click Create DataPipeline.

4. Follow the wizard to complete the following configurations:

   • Basic settings

     • Rule Name: up to 64 characters. Uppercase and lowercase letters, digits, hyphens (-), and underscores (_) are supported. The name must be unique within the same account and region.

     • Rule Description: optional, up to 200 characters.

   • Source data settings

     • Source Bucket: first select the region, and then select the target bucket.

     • Source File Scope: All Files processes all objects in the bucket; Specified Object Name Prefix processes only the objects that match the prefix.

     • Source File Types: you can select multiple types from Text, Image, and Video. The selection must match the capabilities of the vector model you use later.

     • Processing Mode: Process only existing initial data (files that already exist when the pipeline is created); Process only subsequent incremental data (newly uploaded files trigger processing automatically); Process both existing and subsequent incremental data.

   • Vectorization settings

     • Vector Model Source: currently only Alibaba Cloud Model Studio is supported.

     • API KEY: enter the Model Studio API key, which starts with sk-.

     • Vector Model: text models such as text-embedding-v4 (default) and text-embedding-v3; multimodal models such as tongyi-embedding-vision-plus and tongyi-embedding-vision-flash. Make sure that the model output dimensions are the same as the vector index dimensions.

     • Video Frame Capture Frequency: can be configured only for multimodal models. The value range is (0, 1], and the default value is 1.0. A smaller value captures fewer frames.

     • Cascade Deletion Settings: Retain the corresponding vector data when the source object is deleted (default); Delete the corresponding vector data when the source object is deleted.

   • Destination data settings

     • Vector Bucket: must be located in the same region as the source bucket.

     • Vector Index: select the target index from the index list of the selected vector bucket. The index dimensions must be the same as the output dimensions of the vector model.

     • Vector Key Naming Rule: optional. If you enter a prefix, the format is {prefix}/{ObjectKey}. If you leave it empty, the vector key is the same as the ObjectKey.

     • Use ObjectTag as metadata: optional, up to 10 keys, used for subsequent scalar filtering during retrieval.

     • Use UserMeta as metadata: optional, up to 10 keys. The system automatically adds the X-OSS-META- prefix.

   • Exception handling and authorization

     • Exception Handling Method: Skip the exception and continue with subsequent processing tasks; Skip the exception and continue with subsequent processing tasks, and obtain the exception details (you must configure a bucket and prefix to receive error messages; the receiving bucket must be located in the same region as the source bucket and cannot be the source bucket itself); Terminate subsequent tasks immediately when an exception occurs.

     • Authorized Role: select Create Role to automatically create and bind oss-embedding-{uuid}.

   • Confirm the settings: check all settings and click OK after you confirm that they are correct.

Step 2: View the running status of the DataPipeline

After a DataPipeline is created, it automatically enters the "Task Preparing" state, and then changes to the "Task Running" state.

In the rule list on the AI VibeFlow page, you can view information such as the rule name, status, and creation time of each pipeline. Click the rule name or "Details" to view the complete configuration and running statistics.

Step 3: Manage the DataPipeline (optional)

After a DataPipeline is created, you can perform the following management operations as needed.

• Pause a DataPipeline: a pipeline in the Running state can be paused manually. After it is paused, unprocessed files remain in the waiting state, and the vector data that has already been written is not affected.

• Restart a DataPipeline: a paused pipeline can be restarted to continue processing. If the pipeline was automatically paused because the index is full, create a new index first, and then create a new pipeline that points to the new index.

• Delete a DataPipeline: after a pipeline is deleted, OSS stops vectorization, and the vector data that has already been written is not deleted. Pause a running pipeline before you delete it. The deletion cannot be undone.

• View vectorization error messages: if you selected "Skip and log error messages" when you created the pipeline, the ErrorCode, ErrorMessage, and Model Studio RequestId of failed requests are written as objects to the specified bucket. The path format is {ErrorBucket}/{ErrorPrefix}/{DataPipelineName}/{ObjectKey}-{millisecond timestamp}.log.

Step 4: Perform vector-based semantic retrieval

After vectorization is complete, you can perform semantic retrieval on the data written to the vector bucket to build RAG applications or similar content search services. For the specific retrieval operations, see OSS Vectors getting started.

What to do next

All operations on AI VibeFlow DataPipelines can be completed through the console, ossutil, SDKs, or APIs. This getting started topic shows only the console operation path. For complete configurations and advanced usage, see Use AI VibeFlow to produce vector data.=