Dataset Management

Updated at:

Bailian CLI command reference for dataset upload, validation, and management.

Manage datasets for model fine-tuning from the terminal: upload, validate locally, view, and delete.

ImportantAppend --help to any command for full parameter details, e.g. bl dataset upload --help. Command details on this page are verified against bl 2.2.1.

Command reference

bl dataset upload

Upload a dataset file (.jsonl or .zip) to Bailian. Requires API Key authentication.

Usage

bl dataset upload --file <path> [--purpose <name>] [--schema <chatml|dpo|cpt|tts|image|video>] [--no-validate] [--full-validate]
ParameterTypeRequiredDescription
--file <path>stringyesLocal dataset file (.jsonl or .zip; ≤200MB SFT/DPO, ≤300MB CPT, ≤2GB media zip)
--purpose <name>stringnoDataset purpose tag (default: "fine-tune")
--schema <s>stringnoRecord schema: "chatml" (SFT), "dpo" (chosen/rejected), "cpt" (raw text), "tts" (audio), "image" (image generation), or "video" (video generation). Default auto-detects per record.
--no-validateswitchnoSkip the local JSONL pre-flight check (not recommended)
--full-validateswitchnoJSON.parse every line instead of sampling (slower)

Example: Upload dataset file to Bailian

bl dataset upload --file train.jsonl
  • Supports .jsonl (text) and .zip (audio/image/video archives with a data.jsonl manifest). Six record schemas are recognized: chatml = {messages:[...]} (SFT); dpo = {messages:[...], chosen, rejected}; cpt = {text:"..."} (continual pre-training, raw text); tts = {wav_fn:"train/xxx.wav", text:"..."} (audio fine-tuning); image = {img_path:"..."} (image generation); video = {first_frame_path:...} (video generation).
  • With no --schema, a record carrying wav_fn is validated as TTS, img_path as image, chosen/rejected as DPO, text (no messages) as CPT, otherwise ChatML.
  • Upload cap: 200MB SFT/DPO text, 300MB CPT, 2GB media zip. Upload uses the OpenAI-compatible /compatible-mode/v1/files endpoint so the purpose tag is persisted (the DashScope-native /api/v1/files drops it).

bl dataset validate

Locally validate a dataset file (.jsonl or .zip) without uploading. No authentication required.

Usage

bl dataset validate --file <path> [--full-validate] [--schema <chatml|dpo|cpt|tts|image|video>]
ParameterTypeRequiredDescription
--file <path>stringyesLocal dataset file (.jsonl or .zip)
--full-validateswitchnoJSON.parse every line instead of sampling (slower)
--schema <s>stringnoRecord schema: "chatml" (SFT), "dpo" (chosen/rejected), "cpt" (raw text), "tts" (audio), "image" (image generation), or "video" (video generation). Default auto-detects per record.

Example: Locally validate dataset file

bl dataset validate --file train.jsonl
  • Default scan: every line gets a structural check, then ~160 lines (front 50, evenly spaced 100, last 10) are JSON.parsed against the active schema.
  • Schemas: chatml = {messages:[...]} (SFT); dpo = {messages:[...], chosen, rejected}; cpt = {text:"..."} (continual pre-training, raw text); tts = {wav_fn:"train/xxx.wav", text:"..."} (audio fine-tuning); image = {img_path:"..."} (image generation); video = {first_frame_path:"...", video_path:"..."} (video generation, i2v first-frame or kf2v first+last-frame with last_frame_path).
  • With no --schema, a record carrying wav_fn is validated as TTS, img_path as image, first_frame_path/video_path as video, chosen/rejected as DPO, text (no messages) as CPT, otherwise ChatML. Pass --schema to require a specific shape on every record.
  • ZIP archives (.zip) are validated structurally (data.jsonl present, media references resolve) in addition to per-record content checks. Use --full-validate to JSON.parse every line.

bl dataset list

List uploaded dataset files. Requires API Key authentication.

Usage

bl dataset list [--page <n>] [--page-size <n>] [--purpose <name>]
ParameterTypeRequiredDescription
--page <n>numbernoPage number (default: 1)
--page-size <n>numbernoResults per page (default: 10, max 100)
--purpose <name>stringnoFilter by purpose (e.g. "fine-tune", "evaluation"). Omit to list all.

Example: List datasets filtered by purpose

bl dataset list --purpose fine-tune

bl dataset get

Get details of a single dataset file. Requires API Key authentication.

Usage

bl dataset get --file-id <id>
ParameterTypeRequiredDescription
--file-id <id>stringyesDataset file ID

Example: Retrieve dataset file details by ID

bl dataset get --file-id file-xxx

bl dataset groups

List Console dataset groups. Requires Console authentication.

Usage

bl dataset groups [--page <n>] [--page-size <n>] [--name <name>]
ParameterTypeRequiredDescription
--page <n>numbernoPage number (default: 1)
--page-size <n>numbernoGroups per page (default: 10)
--name <name>stringnoFilter by name (server-side filtering; trimmed before sending)
--workspace-id <id>stringnoWorkspace ID (env: BAILIAN_WORKSPACE_ID)

Example: List Console dataset groups

bl dataset groups --page 2 --page-size 20
  • Fetches only the requested page. records contains dataset groups; items is the server's total count for the query. Each group's datasets is only a one-version preview, not its complete version list; use dataset version --dataset-group-id to get the full version list for that group.

bl dataset version

List versions by group or get a single version by ID. Requires Console authentication.

Usage

bl dataset version (--dataset-group-id <id> | --dataset-id <id> [--include-status])
ParameterTypeRequiredDescription
--dataset-group-id <id>stringnoList Console dataset versions by group; mutually exclusive with --dataset-id
--dataset-id <id>stringnoConsole dataset version ID (not an API file ID)
--include-statusswitchnoAlso query import/release state; only valid with --dataset-id
--workspace-id <id>stringnoWorkspace ID (env: BAILIAN_WORKSPACE_ID)

Example: List dataset versions by group

bl dataset version --dataset-group-id example-group-id

Example: Get a single version with status details

bl dataset version --dataset-id example-version-id --include-status --output json
  • Exactly one of --dataset-group-id or --dataset-id is required. --include-status is only valid with --dataset-id.
  • Use dataset groups to find a datasetGroupId, then list versions with --dataset-group-id; pick a datasetId from the result and query details with --dataset-id. fileId belongs to the file/training workflow and cannot substitute for a Console ID.

bl dataset delete

Delete a dataset file by ID. Requires API Key authentication.

Usage

bl dataset delete --file-id <id>
ParameterTypeRequiredDescription
--file-id <id>stringyesDataset file ID
--yesswitchnoConfirm this high-risk operation

WarningThis permanently deletes the specified dataset file and cannot be undone.

Example: Delete a dataset file by ID

bl dataset delete --file-id file-id-xxx