Best practices for decision model fine-tuning

Best practices for Alibaba Cloud Model Studio decision model fine-tuning — the full workflow of fine-tuning, deployment, and post-deployment invocation

This guide helps you complete decision model fine-tuning, deployment, and post-deployment invocation. The Model Studio decision model is provided under the model name decision-model-preview-2026-09-24 and supports direct invocation and fine-tuning; it does not generate text — a single forward pass outputs classification, scoring, and yes/no judgments together with their probabilities, making it suitable for high-frequency decision scenarios such as ticket routing, content moderation, agent routing, and result verification.

Full fine-tuning workflow: Environment preparation → Data preparation → Training submission → Model deployment → Post-deployment invocation.

Model Studio supports fine-tuning of the following models:

Before invocation, complete the following:

• API Key: Obtain an API Key from the Model Studio console API-KEY page
• Authorization: Contact your account manager to enable access
• Environment variable: export DASHSCOPE_API_KEY="你的百炼 API Key"

Data preparation

Download the training data sample package (includes the sample data tickets.train.jsonl / tickets.development.jsonl, the question configuration workload.json, and the data scripts gen_data.py / split_data.py). The sample data is already split and ready to use; when using your own data, use the scripts to synthesize or split it (see README.md in the sample package).

gen_data.py calls a Model Studio large model for automatic annotation, which incurs usage fees.

Each JSONL record contains the input state and the question table questions. Each question carries a type definition, a label (hard label), and an optional target (soft label):

{
  "state": {"content": "发票抬头写错了，请帮忙重新开票。产品功能正常，也没有其他异常。"},
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "应该由哪个团队处理？",
      "criteria": {"billing": "支付、退款和账单问题", "technical": "产品故障和集成问题"},
      "label": "billing",
      "target": {"billing": 0.96, "technical": 0.04}
    },
    "escalate": {"type": "noul", "instructions": "是否需要立即通知值班人员？", "label": false},
    "severity": {
      "type": "score", "instructions": "这个问题有多严重？",
      "criteria": ["轻微问题，不影响功能", "部分功能受影响，但存在替代方案", "核心功能不可用，没有替代方案", "造成严重业务或安全影响"],
      "label": 0
    }
  }
}

Rules for filling in label and target:

target should include all options, with probabilities between 0 and 1 that sum to 1. The question ID, type, instructions, and criteria must remain consistent across training, evaluation, and online requests for the same task; when you modify the options, update the labels and probability keys accordingly.

Training submission

Decision model fine-tuning can be submitted in two ways: in the console (recommended, no coding required) or through the OpenAPI.

Submitting in the console

1. Go to the Model Fine-tuning page and click Create Training Task.
2. For the training type, choose SFT fine-tuning.
3. Model: search for and select the decision model (decision-model-preview-2026-09-24), then click Confirm.
4. Training method: only Efficient Training is supported; full-parameter training is not selectable.
5. Hyperparameter configuration: only batch_size (range [1, 1024]), learning_rate, and n_epochs (range [1, 200]). For their meanings and defaults, see the hyperparameter table in Submitting through the OpenAPI.
6. Data configuration:
   • Training set: Dataset Selection (choose an existing dataset, including datasets uploaded through the OpenAPI) or Dataset Mounting (mount a data file from OSS).
   • Validation set: Select Dataset or randomly split from the current training set (split a proportion of the training set).
7. Training resources: choose the billing method — pay by token (limited-time 0 CNY per 1,000 tokens), subscription training units, or pay-as-you-go training units.
8. Training output: set the model name (auto-generated by default), the export limit, and the checkpoint save interval; model encryption can be enabled.
9. Click Start Training.

After submission, track progress in the job list: the base model column shows the decision model, and the training method column shows SFT efficient training. The job detail page has three tabs — Metrics, Logs, and Output. For the metric definitions, see Training metrics.

Submitting through the OpenAPI

Datasets are uploaded through the Model Studio OpenAPI (in JSONL format). After obtaining a file_id, reference it in the fine-tuning request. The China site endpoint is https://dashscope.aliyuncs.com and the Singapore site endpoint is https://dashscope-intl.aliyuncs.com; the API Key must match the site:

curl --request POST 'https://dashscope.aliyuncs.com/api/v1/files' \
  --header 'Authorization: Bearer '${DASHSCOPE_API_KEY} \
  --form 'files=@"/path/to/train.jsonl"' \
  --form 'purpose="fine-tune"' \
  --form 'descriptions="decision-model-preview training dataset"'

The response returns a file_id (e.g. 976bd01a-...); fill it into the fine-tuning request below. For details, see Training set and evaluation set and Fine-tuning data upload rules.

Dataset upload and fine-tuning requests go through the Model Studio OpenAPI, authenticated with Authorization: Bearer <API-Key>.

Submit a fine-tuning job via the API:

curl --location --request POST 'https://dashscope.aliyuncs.com/api/v1/fine-tunes' \
  --header "Authorization: Bearer ${DASHSCOPE_API_KEY}" \
  --header 'Content-Type: application/json' \
  --data-raw '{
    "model": "decision-model-preview-2026-09-24",
    "training_datasets": [
      {"data_source_type": "file_id", "file_id": "your-file-id"}
    ],
    "validation_datasets": [
      {"data_source_type": "file_id", "file_id": "your-file-id"}
    ],
    "hyper_parameters": {
      "n_epochs": 2,
      "learning_rate": "2e-5",
      "batch_size": 1,
      "save_strategy": "epoch",
      "save_total_limit": 1
    },
    "training_type": "efficient_sft",
    "finetuned_output_suffix": "mytune"
  }'

Input parameters:

Hyperparameters:

Response example (excerpt):

{
  "request_id": "your-request-id",
  "output": {
    "job_id": "ft-xxxxxxxx",
    "status": "PENDING",
    "finetuned_output": "decision-model-preview-2026-09-24-ft-xxxxxxxx",
    "model": "decision-model-preview-2026-09-24",
    "training_type": "efficient_sft"
  }
}

finetuned_output is the model name produced by fine-tuning. After deployment, call it through the decision invocation API (pass this model name in the model field).

Training metrics

After submitting the job, you can view training progress and artifacts in Model Studio console model tuning. The metrics page is divided into three groups:

In the Calibration group, check whether _after improves relative to _before (acc increases, brier/ece decreases) — this indicates that fine-tuning improved calibration.

Model deployment

After training completes, the last checkpoint is automatically published to My models. To publish an intermediate checkpoint, go to the artifacts page of the Model tuning console and do it manually.

On the "My models" page, deploy the finetuned_output returned by training (e.g. decision-model-preview-2026-09-24-ft-xxxxxxxx) with deployment specification MU5×1. After deployment completes, you can call it through the decision invocation API. For deployment methods and MU unit prices, see Model deployment and Training and deployment pricing.

Post-deployment invocation

After deployment, call the fine-tuned artifact through the decision invocation API. Pass the finetuned_output returned by training (e.g. decision-model-preview-2026-09-24-ft-xxxxxxxx) in the model field; the other parameters are the same as when calling decision-model-preview:

curl -sS -X POST https://dashscope.aliyuncs.com/compatible-mode/v1/systemone \
  -H "Authorization: $DASHSCOPE_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "decision-model-preview-2026-09-24-ft-xxxxxxxx",
    "state": {"content": "订单支付后超过 24 小时仍未到账，要求立即处理。"},
    "questions": {
      "department": {"type": "choice", "instructions": "应该由哪个团队处理？",
                     "criteria": {"billing": "支付、退款和账单问题", "technical": "产品故障和集成问题"}}
    }
  }'

import os, requests

resp = requests.post(
    "https://dashscope.aliyuncs.com/compatible-mode/v1/systemone",
    headers={"Authorization": os.environ["DASHSCOPE_API_KEY"], "Content-Type": "application/json"},
    json={"model": "decision-model-preview-2026-09-24-ft-xxxxxxxx",
          "state": {"content": "订单支付后超过 24 小时仍未到账，要求立即处理。"},
          "questions": {"department": {"type": "choice", "instructions": "应该由哪个团队处理？",
                       "criteria": {"billing": "支付、退款和账单问题", "technical": "产品故障和集成问题"}}}},
    timeout=60,
)
print(resp.json()["answers"]["department"])

Best practices

Soft labels: teach the model calibration

In addition to the label (the answer), the training data's target carries a probability distribution auto-annotated by gen_data.py (from the sample package), which calls a Model Studio large model (default qwen3.8-max). This tells the model not just the answer but also how certain it is, making the output probabilities better calibrated. --target-temp adjusts the softness (default 1.5; the larger the value, the smoother and the more uncertainty is preserved). When resuming a run, you can reuse the saved annotations and only adjust the temperature — no further API calls and no additional fees.

Handling inputs that cannot be judged

Some inputs lack sufficient information to be judged (e.g. "help me check my ticket" with no details provided). Use --unknowable N to generate such samples; set their target to a uniform distribution and exclude them from accuracy evaluation. This specifically reduces the model's overconfidence when information is insufficient — without such samples, the model still tends to answer incorrectly with high confidence.

Keep question definitions consistent

Fine-tuning learns the mapping "input + question definition → answer" (the question definition is the ID, type, instructions, and criteria). Changing the question definition causes distribution drift — accuracy drops and calibration fails. When you must change it, re-prepare the data with the new definition and retrain.

FAQ

Data

Q: Is the fine-tuning data format the same as the invocation format?

Not exactly. Each training data record additionally carries a label (hard label) and an optional target (soft label); these two fields are not included when invoking.

Q: What is the file size limit for fine-tuning data?

A single file can be at most 300 MB; the total quota for all valid files is 100 GB, and the maximum number of files is 10,000.

Q: Must fine-tuning data be in JSONL format?

Yes. Training data must be in JSONL format (one JSON object per line).

Q: Do both label and target need to be filled in?

label is required (hard label); target is optional (soft label). When target is present, it must cover all options, with probabilities between 0 and 1 that sum to 1.

Q: Can fine-tuning data be synthesized with gen_data.py?

Yes. The sample package includes gen_data.py, which calls a Model Studio large model for automatic annotation to generate data (this incurs usage fees).

Q: Does fine-tuning support mixed Chinese and English data?

Yes. The language field in workload.json specifies the language; both Chinese and English data can be generated.

Q: Does fine-tuning support importing data from OSS?

Yes. When creating a training job in the console, the training set can use Dataset Mounting to mount data files from OSS directly; when submitting through the OpenAPI, upload the dataset first and reference the returned file_id in the fine-tuning request.

Q: Do unknowable samples in the data affect accuracy?

No. Unknowable samples are excluded from accuracy evaluation; they are only used to reduce the model's overconfidence when information is insufficient.

Training

Q: Do I need to prepare my own GPUs for fine-tuning?

No. Model Studio hosts the training; you only need to submit the data and hyperparameters via the console or the OpenAPI.

Q: Does fine-tuning support continued training?

Yes. The model parameter can take a model ID produced by another fine-tuning job, allowing you to further fine-tune a fine-tuned artifact.

Q: Is the fine-tuned model always better than the base model?

Not necessarily. The result depends on data quality and quantity; poor-quality data may underperform zero-shot invocation. We recommend validating the pipeline with the sample data first.

Artifacts

Q: Can I still use the base model after fine-tuning?

Yes. The base model decision-model-preview and the fine-tuned artifact are independent and can be invoked at the same time.

Q: What is not supported by fine-tuned artifacts?

Same as the base model: context caching, Function Calling, web search, batch inference, system message, and sampling parameters such as temperature/top_p/max_tokens are not supported.

Q: Has the context length of fine-tuned artifacts changed?

No. Same as the base model: the maximum input is 65535 tokens.

Q: Are the rate limits for fine-tuned artifacts the same as the base model?

Yes. RPM 1200 / TPM 2 million (the same in Beijing and Singapore).

Q: Does confidence always increase after fine-tuning?

Not necessarily. Confidence is a statistic of distribution concentration; whether it improves after fine-tuning depends on the data. Check the after/before comparison in the Calibration group on the metrics page.

Q: How are fine-tuned artifact model names formed?

The finetuned_output_suffix field customizes the suffix (up to 8 characters); the artifact name format is decision-model-preview-2026-09-24-ft-{suffix}.

Q: Does fine-tuning support all three question types: choice/noul/score?

Yes. All three question types can be fine-tuned.

Invocation

Q: Is the invocation endpoint for fine-tuned artifacts the same as the base model?

Yes. Both use https://dashscope.aliyuncs.com/compatible-mode/v1/systemone (dashscope-intl.aliyuncs.com on the Singapore site).

Q: Is the authentication for fine-tuned artifacts the same as the base model?

Not exactly. Fine-tuning requests go through the Model Studio OpenAPI with Bearer authentication; inference calls to the decision invocation API do not add the Bearer prefix at the gateway.

Deployment

Q: What deployment specification does fine-tuning use?

MU5×1. Pay-as-you-go: 21 CNY/hour (billed by the minute at minimum); subscription: 10139 CNY/month (billed by the day at minimum).

Other

Q: Is fine-tuning charged?

Currently free of charge for a limited time: 0 CNY per 1,000 tokens.

Q: Which regions does fine-tuning support?

Console submission supports China (Beijing). OpenAPI submission supports Beijing (dashscope.aliyuncs.com) and Singapore (dashscope-intl.aliyuncs.com); the API Key must match the site.

Q: Can I change the question definitions after fine-tuning?

Not directly. Changing the question definitions (ID/type/instructions/criteria) causes distribution drift; you need to re-prepare the data and retrain.

Q: Will decision-model-preview change in the future?

It is a preview version and the team keeps iterating on it. For business scenarios with high stability requirements, we recommend waiting for the official release before integrating.

Q: How much data is needed to start fine-tuning?

When the training and validation datasets are uploaded separately, both datasets must contain at least batch_size samples; when only the training dataset is uploaded, it must contain at least batch_size × 10 samples. Recommended data volume: on the order of hundreds of records.

Q: How long does one fine-tuning job take?

Model Studio does not provide a specific duration (efficient training converges fast); as an open-source reference, Kev takes about 40 minutes on a single H100 (a 4B model, 10k records, 2 epochs), and the Model Studio decision model is of a similar scale.

Q: What value should n_epochs be set to?

For datasets <10k records, 3–5 is recommended; for datasets >10k, 1–2. The Model Studio decision model defaults to 2 (verified working).

Q: Can fine-tuned artifacts be deleted?

Yes. After training completes, you can delete the tuning job via DELETE /api/v1/fine-tunes/<job_id> (deletion is not allowed during training); fine-tuned artifacts are managed on the "My models" page.

Next steps

• Model deployment: Deployment methods and specifications
• Training and deployment pricing: Fine-tuning and deployment billing
• Fine-tune models in the console: View artifacts and publish in the console
• Fine-tuning data upload rules: Packaging rules, and size and quantity limits