OPD Distillation Training Overview

Bailian OPD Distillation Training (OPD) training overview — basic concepts, training method selection, end-to-end flow, environment preparation, and job submission

This article helps you understand the basic concepts, applicable scenarios, and end-to-end flow of Bailian OPD Distillation Training (OPD) training. This article provides an overview and key explanations; for detailed steps of each phase, see the corresponding documents:

Full Process Overview

OPD training consists of 5 steps, from environment preparation to model deployment:

1. Environment preparation — install the SDK, obtain an API Key, and complete authorization
2. Prepare data and code — JSONL data; pure distillation requires no function code
3. Submit the training job — pass teacher_model to trigger distillation; pure distillation with zero code
4. Monitor training status — check the distillation loss, whether the teacher model signal is healthy, and validation set metrics
5. Publish and deploy the model — the final Checkpoint is published automatically, and the deployed model can be called via API

Working Mechanism

The data flow of each training round is as follows:

训练数据（用户问题 + 参考答案）
            │
            ▼
学生模型生成：学生模型按当前策略生成候选输出
            │
            ▼
教师模型前向：教师模型对学生模型输出逐 token 给概率分布
            │
            ▼
蒸馏更新：学生模型最小化自身分布与教师模型分布差异（KL 散度）
            │
            ▼
（可选）Reward 评分：启用 Reward 时叠加任务奖励信号
            │
            ▼
循环迭代：用更新后的模型重新采样，逐轮向教师模型靠拢

The teacher model only provides probability distributions; it does not generate text or participate in dialogue or tool calls. Pure distillation is guided solely by the teacher model token by token; when Reward is also enabled, the teacher model distribution and the task reward jointly drive the student model.

Models and Training Units

The teacher model must be a model already enabled in Bailian, from the same series as and stronger than the student model.

Quick Start

Complete the 5-step full flow with the basic pure distillation example — zero code is enough to start training. Pure distillation requires no function code and is guided solely by the teacher model token by token during training of the student model.

The OPD distillation training entry is available among the training methods when creating a training task in the console; task submission, metrics observation, and output management take place in the Model Fine-tuning console. The complete configuration and submission flow in this and the following sections is based on the DashScope SDK (the demo ships a wheel that supports OPD).

Environment Preparation

Environment preparation covers the following items: API Key, service authorization, SDK installation, workspace directory, and core environment variables.

API Key — obtain it from the Bailian console API-KEY page and set it as the environment variable DASHSCOPE_API_KEY.

Service authorization — on first use, authorize three cloud services with one click in the model tuning console: ARMS (Application Real-Time Monitoring Service, which hosts OpenTelemetry metric reporting), Function Compute (FC), and Log Service (SLS). OpenTelemetry is an open-source observability data standard, hosted and visualized by ARMS; the three services are used for metric reporting, function execution, and log collection respectively.

SDK — install from the wheel file bundled in the example directory (the version updates with the example, currently 1.27.5):

# 安装示例目录附带的 dashscope SDK wheel 文件（版本随示例更新，当前 1.27.5）
# 该 wheel 同名相对路径需填入 FC_PYPI_LIB，供远端函数运行时安装
python -m pip install ./dashscope-*.whl

Workspace directory — obtained from the sample code package, which contains the wheel (a Python packaged install file), function code, data, dependencies, and the submission script, structured as follows; for the complete implementation of obtaining the package and the submission script, see Training configuration:

workspace/
├── dashscope-*.whl          # SDK wheel：示例附带的 dashscope 安装包，本机与远端均按此版本安装
├── functions/               # Rollout/Reward 函数代码目录；纯蒸馏 functions=None 可留空
├── data/                    # JSONL 训练/验证数据；每行一个样本（messages + rollout_extra）
├── requirements.txt         # 函数依赖清单；FC_LAYER_USED=true 时注册阶段据此构建 layer，跳过冷启动 pip
└── submit_job.py            # 提交脚本：调用 AgenticRL().run() 完成注册→上传→提交三步

Three core environment variables:

• DASHSCOPE_API_KEY (required): authentication credential.
• FC_PYPI_LIB (required): the SDK package installed in the function runtime; fill in the wheel's relative path (resolved on the remote side, so a local absolute path will not match). Pure distillation (functions=None) does not involve the function runtime, so this item can be skipped.
• FC_LAYER_USED (optional, default true): when true, a layer is built from requirements.txt during the registration phase, skipping cold-start pip installation; when false, the remote runtime installs it at cold start.

export DASHSCOPE_API_KEY="sk-your-api-key"                       # 鉴权密钥，从百炼控制台 API-KEY 页获取（必需）
export FC_PYPI_LIB=./dashscope-1.27.5-py3-none-any.whl          # 远端函数运行时安装的 SDK 包，填 wheel 相对路径；纯蒸馏可不设（必需，纯蒸馏除外）
export FC_LAYER_USED=true                                        # true：注册阶段按 requirements.txt 构建 layer，跳过冷启动 pip 安装；false：远端冷启动安装

FC_PYPI_LIB takes the actual wheel file name (the version number updates with the example).

Before submission, the example script checks whether the installed SDK's AgenticRL.run accepts the teacher_model parameter, and fails with an error immediately if it does not. Other environment variables (control plane endpoint, inference endpoint, log level, etc.) are documented in Training configuration.

Prepare Data and Code

Each JSONL line is one sample with two fields: messages and rollout_extra. messages is the dialogue input (ChatML format); rollout_extra holds the reference answer or other business data, consumed when Reward or a custom Rollout is enabled.

{
  "messages": [                          // 对话输入，ChatML 格式；每条 message 含 role 与 content
    {"role": "user", "content": "<您的业务问题>"}
  ],
  "rollout_extra": {"solution": "<参考答案或其他业务数据>"}  // 参考答案/业务数据，供 Reward 或自定义 Rollout 启用时消费；纯蒸馏仅透传不消费
}

Dozens of samples are enough for a quick validation; for formal training, use hundreds to thousands of samples depending on task complexity. In pure distillation, rollout_extra is only passed through and is not consumed.

Submit the Training Job

The entry point is AgenticRL().run(...); passing teacher_model triggers distillation. run() completes in one step: register functions to FC → upload training data → submit the training job. Pure distillation does not register functions.

import asyncio
from dashscope.finetune.agentic_rl import AgenticRL          # OPD 提交客户端
from dashscope.finetune.reinforcement import (
    TrainingDataset, ValidationDataset, DataSourceType,       # 训练/验证数据集及数据源类型
)

async def main():
    client = AgenticRL()
    result = await client.run(
        job_name="my-opd-job",                                # 任务名（控制台展示用）
        model="qwen3.5-9b",                       # 学生模型（基座模型，被蒸馏对象）
        teacher_model="your-teacher-model-id",     # 教师模型，OPD 唯一新增参数；传值即触发蒸馏
        training_datasets=[TrainingDataset(
            data_source_type=DataSourceType.FILE_ID,          # 以文件方式上传训练数据
            file_name="./data/train.jsonl")],
        validation_datasets=[ValidationDataset(
            data_source_type=DataSourceType.FILE_ID,          # 验证集同样以文件方式上传
            file_name="./data/validation.jsonl")],
        functions=None,                            # 纯蒸馏：不注册任何函数；Agentic OPD 在此传 Rollout/Reward 函数
        hyper_parameters={
            "n_epochs": 1, "n_rollouts": 8, "learning_rate": 1e-6,  # 蒸馏是微调模型已有能力，学习率起点 1e-6
            "eval_steps": 1, "ppo_mini_batch_size": 8,
            "lr_scheduler_type": "cosine",  # kl_loss_coef OPD 未启用，见训练配置
            # opd_teacher_* 部署参数见训练配置
        },
        resources={
            "charge_type": "mtu_postpaid",           # MTU 后付费计费
            "mtu_spec_code": "MTU4",                  # 训练单元规格
            "mtu_capacity": 24,                      # 容量（并发资源数）
        },
    )
    # HTTP 200 返回 job_id（查询训练状态的唯一标识）；否则打印原始响应便于排查
    print(result.output.job_id if result.status_code == 200 else result)

asyncio.run(main())                                  # 启动异步主函数

Getting Started Tips

• Start with the right example: for plain-text output tasks, start from the Model OPD example; when the output depends on API-level tool calls or multi-turn/custom interaction, start from the Agentic OPD example.
• Validate with a small dataset first: verify the pipeline with dozens of samples + n_epochs=1 before switching to the full dataset, to avoid burning MTU on the full dataset only to discover configuration errors.
• Keep hyperparameters at their defaults: start learning_rate at 1e-6 (distillation fine-tunes the model's existing capabilities rather than rebuilding them, and an overly large learning rate easily breaks the output format) and use cosine for lr_scheduler_type; when tuning, change only one variable at a time and train at least eval_steps × 3 steps before judging the trend.
• On FAILED, check the logs first: when a job fails, check the error at the end in the log tab, or call AgenticRL.logs(job_id="ft-xxx", lines=100) from the SDK.

Post-training Process

Observation and Effect Evaluation

The job_id returned by run() is the unique identifier for querying training status. Open the job detail page in the model tuning console, which contains four tabs: metrics, trajectories, outputs, and logs.

What to watch:

• trace/distillation/valid_token_rate close to 1.0 (the teacher model signal is complete).
• actor/distillation/loss decreasing or stabilizing (the student model is converging toward the teacher model).
• When Reward is enabled, additionally watch validation/data/reward/mean@1 and each dimension of trace/reward_metrics rising relative to the baseline.

Post-training evaluation requires a pre-training baseline and consistent evaluation conditions (the same eval set and decoding parameters). For complete metric meanings, trajectory viewing, acceptance criteria, and the troubleshooting decision tree, see Observability configuration.

Publish and Deploy

After training completes, the last Checkpoint is automatically published to My Models. To publish an intermediate Checkpoint, go to the outputs page of the model tuning console and do it manually. Published models can be deployed on the My Models page; once deployment completes, the model can be called via API.

Applicable Scenarios

Applicable scenarios scale up by capability complexity: pure distillation is the base capability, while Reward and custom Rollout are advanced capabilities layered on top as needed (introducing additional components and more complex training/generation pipelines).

• Pure distillation (base): only a teacher model + data, no function components written → Model OPD development
• +Reward (advanced): add a scoring function that layers task reward on top of the teacher model's token-by-token guidance (scoring against a standard answer, or scoring across multiple dimensions) → Model OPD development
• +Custom Rollout (advanced, Agentic OPD): customize the generation process when the output depends on tool calls or multi-turn interaction; can be combined with Reward or used alone → Agentic OPD development

The two advanced capabilities can be enabled independently and combined freely; selection starts from the generation path: single-turn generation suffices → the first two levels (Model OPD); structured mechanisms (such as native tool calls) or custom interaction are required → the third level (Agentic OPD).

Next Steps

• Model OPD development: complete details of function development (Rollout/Reward functions, Tracing integration).
• Agentic OPD development: custom Rollout (passing the tools parameter, structured output), Agentic Reward development.
• Training configuration: submission methods, field-by-field descriptions, hyperparameters, and environment variables.
• Observability configuration: metric meanings, trajectory viewing, acceptance criteria, and the troubleshooting decision tree.

FAQ

When to choose OPD over SFT

Per-token error correction is needed and the student model should learn from its own generations → OPD; only fixed annotations are available, the budget is sensitive, and online correction is not required → SFT.

Can training effectiveness be judged without a reward

Yes. Pure distillation relies on valid_token_rate approaching 1.0 and loss decreasing and stabilizing to judge whether the teacher model signal has been absorbed; the final effect is judged by an independent post-training evaluation (keep the baseline and compare under the same criteria). If you need to optimize online against task objectives during training, add Reward (an advanced usage of Model OPD), at the cost of providing a computable Reward function.