Training set and evaluation set

Datasets are the foundation of model training and evaluation. The Alibaba Cloud Model Studio Data Management feature helps you efficiently create and manage datasets.

Dataset overview

Alibaba Cloud Model Studio Data Management centrally manages all large model-related datasets in your business workspace. Datasets are divided into Training Set (used for model fine-tuning, supporting SFT/DPO/CPT) and Evaluation Set (used for Model evaluation).

Training sets support four training scenarios: text generation, visual understanding, image-to-video (first frame), and image-to-video (first and last frame). Evaluation sets support only the text generation scenario. The dataset type cannot be changed after creation; choose carefully based on your downstream usage.

Dataset names can be up to 50 characters long (Chinese, English, digits, underscores, hyphens, and periods are supported), and descriptions up to 200 characters. Creating or publishing empty datasets is not supported. There is no limit on the number of datasets you can create, and no upper bound on the amount of data you can import.

DPO/CPT training methods and cloud storage mounting are supported only in the Beijing region.

Training methods and scenarios

Training sets support three training methods with different applicable scenarios and minimum data requirements:

• SFT (supervised fine-tuning): instruction fine-tuning that enables the model to solve specific tasks. At least one thousand entries recommended. Available on all sites.
• DPO (direct preference optimization): preference alignment training. At least one hundred entries recommended.
• CPT (continual pre-training): domain knowledge injection. At least 50 million tokens of data recommended.

All training sets are published immediately upon submission of creation; saving as a draft first is no longer supported. Historically created draft versions can still be edited online or deleted.

Training method	Purpose	Recommended data volume	Site availability
SFT	Supervised fine-tuning (instruction fine-tuning)	1,000+ entries	All sites
DPO	Direct preference optimization (preference alignment)	100+ entries	Beijing region
CPT	Continual pre-training (domain knowledge)	50 million tokens	Beijing region

Import methods

Datasets support four import methods with different applicable scenarios and prerequisites:

• Upload Local File: no prerequisites; directly upload local files, suitable for small batches of data.
• Import from OSS: add the tag bailian-datahub-access=read to the target bucket; suitable for large batches of data. Not supported for evaluation sets.
• Log Backflow: automatically extract training data from SLS inference logs; requires authorizing a service-linked role, forming an inference → data → fine-tuning enhancement loop.
• API Upload: upload data files programmatically via the Model Studio API, suitable for scenarios requiring automated integration. See Fine-tuning via API or CLI.

For the complete limitations of log backflow (30-day range, 100,000 entry limit, API Key/model filter) and authorization requirements, see the Log backflow topic.

Imported data is automatically encrypted with OSS server-side encryption (SSE-OSS, AES256). Log backflow is available. For the complete list of restrictions, see Dataset overview.

Import method	Prerequisites	Applicable scenario	Evaluation set supported
Local upload	None	Small batches of data	Supported
OSS import	Bucket tag bailian-datahub-access=read	Large batches of data	Not supported
Log backflow	Authorization + API Key/model filter	Build training data from inference logs	Supported

Data Preparation

Preparing high-quality data is key to model fine-tuning results. For the recommended data volumes of each training method (SFT/DPO/CPT text generation), see Tuning data upload rules.

There are no official recommended data volumes for visual understanding and image-to-video training sets; prepare sufficient samples based on your actual scenario.

Data format and templates

For the data file format specifications of each training scenario (SFT text/visual understanding (image/video → text), DPO, CPT, evaluation sets), see Tuning data upload rules.

On the Data Management dataset creation page, download the data template for the corresponding scenario; preparing data according to the template structure can prevent import failures. Evaluation sets support the Text generation (prompt&completion) single-turn format and the Text generation (ChatML-messages) multi-turn dialogue format; for detailed field specifications and examples, see Tuning data upload rules - Evaluation set format.

For data diversity and augmentation strategies, see the model fine-tuning documentation. For evaluation sets, prepare an independent set of non-overlapping data to objectively assess model generalization.

Create a dataset

Before creating a dataset, the following prerequisites must be met: Model Studio service is activated; for OSS import, add the tag bailian-datahub-access=read to the target bucket; for log backflow, authorize a service-linked role.

On the Data Management > Dataset list page, click Create Dataset to enter the standalone creation page, then follow these steps:

1. Enter a dataset name (up to 50 characters) and a description (up to 200 characters, optional).
2. Select a dataset type: Training Set or Evaluation Set (cannot be changed after creation).
3. Select a training scenario: training sets support text generation, visual understanding, image-to-video (first frame), image-to-video (first and last frame), and more; for evaluation sets, select the sample format here — Text generation (ChatML-messages) (multi-turn dialogue) or Text generation (prompt&completion) (single-turn).
4. Select a training method: SFT, DPO, or CPT (required only for training sets; evaluation sets have no such configuration).
5. Select a data format: Jsonl format or Excel format. The file types of evaluation sets are determined by the sample format: ChatML-messages supports jsonl, xls, and xlsx (up to 200 MB per file), and prompt&completion supports only xls and xlsx (up to 20 MB per file); sample templates for both formats are available for download on the creation page.
6. Select a storage location: Platform OSS storage (free, no data volume limit) or Mount Cloud Storage (not available for evaluation sets).
7. Select an import method and upload data: local upload, import from OSS, log backflow, or API upload (see the tab descriptions below).
8. Click Confirm; the dataset is published immediately, completing creation.

The three import methods are operated in different tabs, with different fields and operations in each tab. SFT text-generation and DPO text-generation training sets support uploading multiple files at the same time.

Parameter	Description	Required	Value description
Dataset name	Dataset identifier	true	Up to 50 characters; Chinese/English/digits/underscore/hyphen/period
Dataset description	Dataset description		Up to 200 characters
Dataset type	Training set or evaluation set; cannot be changed after creation	true	Training set/Evaluation set
Training scenario	Training scenario the data applies to	true	Text generation/Visual understanding/Image-to-video (first frame)/Image-to-video (first and last frame)
Training method	Training algorithm type	true	SFT/DPO/CPT
Data format	Data file format	true	Jsonl format/Excel format; evaluation sets provide two sample formats: Text generation (ChatML-messages)/Text generation (prompt&completion)
Storage location	Dataset storage method	true	Platform OSS storage/Cloud storage mounting
Import method	Data import source	true	Local upload/Import from OSS/Log backflow/API upload

Local upload

Click Upload File to select local data files. SFT text-generation and DPO text-generation support multi-file upload. The file format must meet the scenario requirements (see the Data preparation section), and imported data is automatically encrypted with SSE-OSS.

Import from OSS

After adding the tag bailian-datahub-access=read to the target bucket, select the bucket and specify the data file path. Suitable for large-scale data import; not supported for evaluation sets.

Log backflow

Authorize a service-linked role first. Select an API Key and a model, then filter inference logs from the last 30 days (one-time limit of 100,000 entries); the system automatically extracts training data from the logs, forming an inference → data → fine-tuning enhancement loop. For full operational details, see the dedicated log backflow topic.

Version management

A dataset can have multiple independent versions, with version numbers auto-incrementing. On the dataset detail page, click Add Version to create a new version.

When adding a version, the create-new mode is adopted: all data must be re-imported, and the inheritance mode that incrementally modifies the previous version's data is no longer supported.

• Create Mode: re-import all data; suitable for large-scale updates.

Historically created draft dataset versions can be used for Data Processing (data cleaning and data augmentation) to improve data quality before publishing.

Dataset management

On the Data Management > Dataset list page, you can view all datasets. The list displays the following 9 columns: dataset name, type, latest version, data volume, import status, publish status, storage location, import method, and operations (add version/publish/delete). You can filter by storage location and import method. Datasets being imported auto-refresh their status every 5 seconds; no manual refresh is needed. Import status and publish status are two independent columns, reflecting data import progress and whether the dataset has been published, respectively.

Publish and delete

Publish is irreversible; published versions cannot be edited. Deleting a published version is also irreversible; only historically created draft versions can be deleted. Deleting an entire dataset removes all its versions. All these operations are irreversible; proceed with care.

Edit and export

Only historically created draft versions can be edited online (Prompt/Completion content is editable); newly created datasets are published immediately upon submission and no longer produce draft versions, and editing is unavailable for published versions. When exporting datasets, each type has a different format: SFT training sets are exported as jsonl, visual understanding training sets as zip, and evaluation sets as xlsx. Exporting empty datasets is not supported.

Billing

The Data Management feature is free, but dataset storage and downstream resources incur fees; see the Model Studio billing page for details.

• Platform OSS storage: fees are based on the Model Studio billing page.
• Mount Cloud Storage: fees are billed to the user's own OSS account.
• Log Backflow: SLS fees incurred are billed to the SLS product account.

Review the Model Studio billing documentation for the latest prices before use.

Next steps

After a dataset is created and published, it can be used in the following downstream scenarios:

• After a training set is published, it is used for model fine-tuning (SFT/DPO/CPT).
• After an evaluation set is published, it is used for Model evaluation.
• Published datasets (including historically created draft versions) can all be used for Data processing (data cleaning and augmentation).

To learn the full operational details of Log backflow (authorization flow, filter logic), see the dedicated log backflow topic.

Dataset CRUD is currently supported only in the console; the model fine-tuning API can reference published training set IDs via the training_file_ids parameter.