Training set and evaluation set

Dataset overview

Alibaba Cloud Model Studio Data Management centrally manages all large model-related datasets in your business workspace. Datasets are divided into Training Set (used for model fine-tuning, supporting SFT/DPO/CPT) and Evaluation Set (used for Model evaluation).

Training sets support four training scenarios: text generation, multimodal understanding, image-to-video (first frame), and image-to-video (first and last frame). Evaluation sets support only the text generation scenario. The dataset type cannot be changed after creation; choose carefully based on your downstream usage.

Dataset names can be up to 50 characters long (Chinese, English, digits, underscores, hyphens, and periods are supported), and descriptions up to 200 characters. Creating or publishing empty datasets is not supported. There is no limit on the number of datasets you can create, and no upper bound on the amount of data you can import.

DPO/CPT training methods and cloud storage mounting are supported only in the Beijing region.

Training methods and scenarios

Training sets support three training methods with different applicable scenarios and minimum data requirements:

• SFT (supervised fine-tuning): instruction fine-tuning that enables the model to solve specific tasks. At least one thousand entries recommended. Available on all sites.

• DPO (direct preference optimization): preference alignment training. At least one hundred entries recommended.

• CPT (continual pre-training): domain knowledge injection. At least 50 million tokens of data recommended. Draft status and data inheritance are not supported.

CPT and image-to-video training sets do not support draft status; they can only be published immediately at creation. SFT and DPO text-generation training sets can be saved as drafts before publishing.

Import methods

Datasets support three import methods with different applicable scenarios and prerequisites:

• Upload Local File: no prerequisites; directly upload local files, suitable for small batches of data.

• OSS Import: add the tag bailian-datahub-access=read to the target bucket; suitable for large batches of data. Not supported for evaluation sets.

• Log Backflow: automatically extract training data from SLS inference logs; requires authorizing a service-linked role, forming an inference → data → fine-tuning enhancement loop.

Log backflow supports only logs from the last 30 days, with a one-time import limit of 100,000 entries, and requires specifying an API Key and model filter conditions.

Imported data is automatically encrypted with OSS server-side encryption (SSE-OSS, AES256). Log backflow is available. For the complete list of restrictions, see Dataset overview.

Data Preparation

Preparing high-quality data is key to model fine-tuning results. For the recommended data volumes of each training method (SFT/DPO/CPT text generation), see Tuning data upload rules.

There are no official recommended data volumes for multimodal understanding and image-to-video training sets; prepare sufficient samples based on your actual scenario.

Create a dataset

Before creating a dataset, the following prerequisites must be met: Model Studio service is activated; for OSS import, add the tag bailian-datahub-access=read to the target bucket; for log backflow, authorize a service-linked role.

On the Data Management > Dataset list page, click Add Dataset, then follow these steps:

1. Enter a dataset name (up to 50 characters) and a description (up to 200 characters, optional).

2. Select a dataset type: Training Set or Evaluation Set (cannot be changed after creation).

3. Select a training scenario: text generation, multimodal understanding, image-to-video (first frame), or image-to-video (first and last frame). Evaluation sets support only text generation.

4. Select a training method: SFT, DPO, or CPT.

5. Select a storage location: Platform OSS storage (free, no data volume limit) or Mount Cloud Storage (not available for evaluation sets).

6. Select an import method and upload data: local upload, OSS import, or log backflow (see the tab descriptions below).

7. Configure publishing options: Draft or Publish. CPT and image-to-video training sets do not support drafts and can only be published immediately.

8. Click Submit to finish creating the dataset.

The three import methods are operated in different tabs, with different fields and operations in each tab. SFT text-generation and DPO text-generation training sets support uploading multiple files at the same time.

Version management

A dataset can have multiple independent versions, with version numbers auto-incrementing. On the dataset detail page, click Add Version to create a new version.

When adding a version, select a data inheritance strategy:

• Inheritance Mode: incrementally modify data based on the previous version; suitable for minor iterations.

• Create Mode: re-import all data; suitable for large-scale updates.

CPT training sets do not support data inheritance; each new version requires new data.

Draft dataset versions can be used for Data Processing (data cleaning and data augmentation) to improve data quality before publishing.

Dataset management

On the Data Management > Dataset list page, you can view all datasets, filter by type (training set/evaluation set), and search by name. Datasets being imported auto-refresh their status every 5 seconds; no manual refresh is needed.

Billing

The Data Management feature is free, but dataset storage and downstream resources incur fees; see the Model Studio billing page for details.

• Platform OSS storage: fees are based on the Model Studio billing page.

• Mount Cloud Storage: fees are billed to the user's own OSS account.

• Log Backflow: SLS fees incurred are billed to the SLS product account.

Review the Model Studio billing documentation for the latest prices before use.

Next steps

After a dataset is created and published, it can be used in the following downstream scenarios:

• After a training set is published, it is used for model fine-tuning (SFT/DPO/CPT).

• After an evaluation set is published, it is used for Model evaluation.

• Both draft and published datasets can be used for Data processing (data cleaning and augmentation).

To learn the full operational details of Log backflow (authorization flow, filter logic), see the dedicated log backflow topic.

Dataset CRUD is currently supported only in the console; the model fine-tuning API can reference published training set IDs via the training_file_ids parameter.