Fine-tune models in the console

This topic describes how to run model fine-tuning tasks in the console and helps you choose the right fine-tuning method and parameters. The console provides five training methods when creating a training task: SFT fine-tuning, DPO preference training, CPT continued pre-training, RL reinforcement learning, and OPD distillation training.

Model fine-tuning process

Step 1: Select a fine-tuning method

Go to the Model Fine-tuning page and click the Create Training Task button.

After entering the Create Training Task page, select a training method from the Training Method cards in the Training Configuration area. The console provides five training methods:

Training method	Description
SFT fine-tuning	Supervised fine-tuning that improves instruction-following. Supports efficient and full-parameter training.
DPO preference training	Introduces negative feedback to reduce hallucinations and align model outputs with human preferences.
CPT continued pre-training	Unsupervised continued pre-training on unlabeled data to strengthen or add model capabilities.
RL reinforcement learning	Optimizes policies through reward signals to improve model reasoning and decision-making. See Reinforcement Learning.
OPD distillation training	Online policy distillation that lets the student model align with a teacher model on its own sampled trajectories, inheriting strong model capabilities at low cost. See OPD Distillation Training.

When you select SFT fine-tuning, you can further choose Efficient Training or Full-parameter Training. The Training Task Summary panel on the right side of the page displays an overview of the current task's model, priority, billing method, and other configurations in real time.

In the Basic Information area, you can set the Job Name and Job Priority. Job priority has four levels: L0, L1, L2, and L3, listed from highest to lowest, which affects the scheduling order of training jobs. The higher the priority, the earlier the training job is scheduled and executed.

How to choose between CPT, SFT, and DPO

CPT (Continual Pre-Training) aims to use massive volumes of unlabeled training data to improve model performance in specific industries.

In addition to CPT, SFT, and DPO, RL (Reinforcement Learning) is suitable for scenarios that require continuous improvement of model performance through human feedback or automated scoring, as well as improving reasoning and decision-making capabilities. See Reinforcement Learning. OPD distillation training is suitable for inheriting the capabilities of a stronger teacher model at low cost. See OPD Distillation Training.

SFT (Supervised Fine-Tuning) aims to use targeted datasets and training to enhance model performance for specific business scenarios.

DPO (Direct Preference Optimization) uses training datasets that provide both positive and negative samples. By introducing negative feedback, it reduces hallucinations and optimizes specific bad cases.

The three fine-tuning methods provided by Model Studio are not mutually exclusive but are progressive and complementary.

CPT (optional) → SFT → DPO (optional)

1. CPT (continual pre-training) - Supplements knowledge (the "breadth" and "shallow depth" of general model knowledge cannot meet the "depth" and "precision" requirements of professional fields)

   • Finance model: Learns financial terms
   • Medical model: Memorizes drug pathology
   • Legal model: Understands legal articles and precedents

2. SFT (supervised fine-tuning) - Learns how to perform tasks

   • Customer service bot: Learns customer service procedures
   • Code assistant: Learns programming paradigms
   • Tool calling (Agent): Learns to use MCP

3. DPO (direct preference optimization) - Performs tasks better

   • Safety and responsibility: Rejects harmful suggestions
   • Conciseness and effectiveness: Provides concise answers
   • Objectivity and neutrality: Evaluates fairly and objectively

For data file formats of each training method, see Fine-tuning data upload rules.

For data volume requirements of each training method, see Fine-tuning data upload rules.

Alibaba Cloud Model Studio recommends using model fine-tuning in this order: CPT (optional) first, then SFT, and finally DPO:

1. First, collect a large volume (at least 10 million tokens) of domain-specific unlabeled samples and run CPT continual pre-training to turn the model into an expert in a specific industry or domain.
2. Before your application goes live, run SFT training with sufficient (1,000+) positive samples for your specific scenario or business, that is, collect scenario/business inputs and expected model outputs.
3. After your application enters trial operation or goes live, collect sufficient (100+) user feedback (such as likes, dislikes, or feedback) or bad cases. Convert this data into a DPO training set and run DPO training.

Model selection

If this is your first time fine-tuning a model, select the official model you want.

If you are retraining a model because of poor training results, select My Models > the model you want to retrain.

After selecting a model, some models display a Training Modality option (such as text generation or visual understanding). Choose the modality that matches your business scenario. If the model supports it, a Thinking Mode option (such as Instruct or Thinking) also appears. Select as needed.

In the model drop-down list, search for decision to select the Decision Model (Preview) for decision model fine-tuning. For more information, see Decision model fine-tuning best practices.

Full-parameter training vs. efficient training

• Full-parameter training learns by updating all model parameters.
• Efficient training uses Low-Rank Adaptation (LoRA), which updates the decomposed low-rank parameters through matrix decomposition.

Since both training methods cost the same, Alibaba Cloud Model Studio recommends that you prefer full-parameter training if your model supports it, because full-parameter training delivers better results than efficient training and is more cost-effective.

Step 2: Configure hyperparameters

Training parameter descriptions:

Not all models support adjusting all parameters. Refer to the console display for available options.

The parameter configuration panel contains the following training parameters and default values in addition to batch size:

• learning_rate (learning rate): default 3e-4
• n_epochs (number of epochs): default 3, range [1, 200]
• eval_steps (evaluation steps): default 50
• lora_alpha (LoRA scaling factor): default 16
• lora_dropout: default 0.1, range [0, 0.2]
• lora_rank (LoRA rank): default 8
• lr_scheduler_type (learning rate scheduler type): default linear
• max_length (maximum length): default 8192, range [500, 131072]
• warmup_ratio (warmup ratio): default 0.05, range [0, 1]
• weight_decay (weight decay): default 0.01, range [0, 0.2]

Parameter name	Recommended setting	Function of the hyperparameter
Batch size (batch_size)	Use the default value	Batch size represents the data step size at which the model updates its parameters during training. It can be understood as how many data samples the model processes before updating its parameters once. The generally recommended batch size is 16/32, meaning the model updates its parameters once after processing every 16 or 32 data samples. The valid range varies by model and training method. Refer to the console display.
Learning rate (learning_rate)	Efficient training: 1e-4 magnitude Full-parameter training: 1e-5 magnitude CPT continual pre-training: 1e-5 magnitude	Controls how strongly the model corrects its weights. If the learning rate is set too high, model parameters change drastically, and the fine-tuned model may not perform better or may even perform worse; If the learning rate is too low, the performance of the fine-tuned model will not change much.
Epochs (n_epochs)	Data volume < 10,000, 3~5 epochs Data volume > 10,000, 1~2 epochs Adjust based on experimental results	The number of times the model traverses the training data. Adjust it based on your actual model fine-tuning experience. More training epochs mean longer training time and higher training cost. Range [1, 200].
Evaluation steps (eval_steps)	Use the default value	The validation interval in steps during the training phase, used to periodically evaluate model training accuracy and training loss. This parameter affects how frequently Validation Loss and Validation Token Accuracy are displayed during model fine-tuning.
Learning rate scheduler (lr_scheduler_type)	Use “linear” or “Inverse_sqrt”.	The strategy for dynamically adjusting the learning rate during model training. For details on each strategy, see Learning rate scheduler strategies.
Sequence length (max_length)	Set to the maximum value supported by the model	The maximum token length supported for a single training data entry. If the token length of a single entry exceeds the set value: SFT discards the entry directly and does not train on it; DPO automatically truncates the tokens beyond the configured length, and the truncated data is still trained on. For the relationship between characters and tokens, see How to convert between tokens and strings. Range [500, 32768].
Warmup ratio (warmup_ratio)	Use the default value	The proportion of the total training process used for learning rate warmup. Learning rate warmup means the learning rate linearly increases from a smaller value to the set learning rate after training starts. This parameter mainly limits the magnitude of model parameter changes in the initial stage of training, helping the model train more stably. A ratio that is too large has the same effect as a learning rate that is too low, resulting in little change in the performance of the fine-tuned model. A ratio that is too small has the same effect as a learning rate that is too high, and the fine-tuned model may not perform better or may even perform worse. Range [0, 1]. This parameter has no effect on the “Constant” learning rate scheduler.
Weight decay (weight_decay)	Use the default value	L2 regularization strength. L2 regularization helps maintain the model's general capabilities to some extent. Too high a value makes the fine-tuning effect less noticeable. Range [0, 0.2].
Efficient training parameters
LoRA alpha (lora_alpha)	Use the default value	The scaling coefficient that controls the combination of the original model weights and the LoRA low-rank correction. A larger Alpha value gives more weight to the LoRA correction, making the model rely more on the specific information of the fine-tuning task; a smaller Alpha value makes the model more inclined to retain the knowledge of the original pre-trained model.
LoRA dropout (lora_dropout)	Use the default value	The dropout rate of the low-rank matrix values in LoRA training. Using the recommended value enhances the model's generalization capability. Too high a value makes the fine-tuning effect less noticeable. Range [0, 0.2].
LoRA rank (lora_rank)	Set to the maximum value supported by the model	The rank size of the low-rank matrices in LoRA training. A larger rank yields slightly better fine-tuning results but slower training.
Freeze ViT (freeze_vit)	Use the default value	Used to freeze the parameters of the vision backbone network so that its weights are not updated during training. Applies only to Qwen-VL (visual understanding) models. Only when freeze_vit is set to “true” can the model be billed by Token usage.

Learning rate scheduler strategies

Learning rate scheduler (lr_scheduler_type) is located in the Hyperparameter Configuration area and appears after you click Expand Configuration. It contains eight strategies:

• Linear: The learning rate decreases linearly. Suitable for short training tasks.

  

• Polynomial: The learning rate decreases according to a predefined polynomial function as training iterations or epochs progress, allowing finer control over the decay rate. Suitable for complex tasks. However, the built-in polynomial coefficient is 1, so the curve and effect are the same as Linear. Not recommended.

• Cosine: The learning rate follows a cosine curve. Suitable for long training tasks that require fine adjustments.

  

• Cosine_with_restarts: The learning rate periodically decreases to a minimum following the cosine curve shape and "restarts" to the preset value at the end of each cycle, then starts the next cycle. Suitable for cases where the model needs to escape local optima and search for better global solutions. However, testing shows the learning rate does not actually "restart" to the preset value at the bottom of the curve. Not recommended.

• Constant: The learning rate remains unchanged, and the warmup ratio parameter has no effect. Suitable for initial exploration of model performance.

  

• Constant_with_warmup: The learning rate remains unchanged, but the warmup ratio parameter takes effect. Suitable for initial exploration of model performance.

  

• Inverse_sqrt: The learning rate gradually decreases, with the decrease positively correlated with the reciprocal of the square root of the iteration count. Suitable for SFT fine-tuning, balancing learning efficiency and model convergence well.

  

• reduce_lr_on_plateau: Automatically lowers the learning rate when a monitored metric (validation loss or validation accuracy) shows no significant improvement over several consecutive epochs. When the model struggles to improve further, this strategy helps the model continue to optimize and improve.

  

Step 3: Select training data

For dataset construction tips, see Dataset construction tips. To upload fine-tuning datasets, go to the Data Management page.

For the upload format specifications, packaging rules, and size and quantity limits of fine-tuning data, see Fine-tuning data upload rules.

In the Data Configuration area, you can configure the following:

• Training set: Supports two methods: Dataset Selection and Dataset Mounting. Dataset Selection selects a previously uploaded fine-tuning dataset as training data; Dataset Mounting directly mounts data files in OSS.

  NoteIf you use Dataset Mounting to load a dataset, you must first authorize Model Studio to access your data. To do this: go to the Model Fine-tuning page, click Create Training Task, then click Dataset Mounting in Data Configuration and complete the authorization as prompted. If you can see your file list on that page, authorization has succeeded.

• Mixed training: When enabled, you can additionally add a mixed training dataset. Mixed training data preserves the model's general capabilities during fine-tuning, preventing the model from losing its original general conversational ability by overfitting to specific task data. If you have data from multiple business scenarios, enabling mixed training is recommended.

• Validation set: Supports two methods: Auto Split and Dataset Selection. With Auto Split, the platform randomly extracts 10% of the training dataset as the validation set. You can also upload a separate validation dataset. The validation set evaluates model performance during training, displaying Validation Loss and Validation Token Accuracy.

Step 4: Configure training resources

In the Training Resource Configuration area, select the billing method for the training job. The console supports the following three billing methods:

• Token-based billing: Uses the platform's shared idle resources, billed by the number of tokens actually consumed during training. Training speed depends on resource availability, and queuing may occur.
• Training Unit (Prepaid): Pre-purchase training units, deducted by unit. Resources are guaranteed with no queuing. Suitable for scenarios with stable training demand. You must first purchase training units through Resource Pool Management (Prepaid).
• Training Unit (Postpaid): Uses training unit resources, deducted postpaid by actual usage. Suitable for flexible scenarios where you have training demand but prefer not to pre-purchase.

Step 5: Training output

The following settings apply to SFT, DPO, and CPT training.

In the Training Output area, you can configure:

• Model Name: Set the name of the output model. After training completes, the last checkpoint produced is automatically published to My Models under this name.
• Maximum number of exports: Use the Up to drop-down list to set the number of models finally exported by the training job. Each model save adds extra task duration.
• Checkpoint save interval: Set how often checkpoints are saved. Select Calculated by epoch or Calculated by step.
• Model Encryption (Security Upgrade): When enabled, the platform enables OSS server-side encryption for model files, using fully managed OSS keys for encryption and decryption (SSE-OSS), with AES256 as the encryption algorithm.

Step 6: Train the model

Click Start Training > Confirm the Model Fine-tuning Billing Notice > The model starts training.

If you encounter insufficient permissions, see: What should I do if I get a permission error during model fine-tuning?

During model training, click the "Logs" button to view the logs generated in real time during model training. You can also go to the Metrics tab to view Training Loss, Validation Loss, and Validation Token Accuracy.

After training completes, check the trend of the difference between Training Loss and Validation Loss.

1. If Training Loss gradually decreases while Validation Loss gradually increases, the model has overfit the training data. Training may have failed and the results may be poor. Apply the following recommended methods (in order of recommendation) to optimize and retrain to improve the results:

   1. Use data augmentation to increase training data diversity and volume.
   2. Collect more training data to increase training data diversity and volume.
   3. Adjust hyperparameters, including: reduce "epochs", lower the "learning rate", decrease the "batch size", increase the "weight decay", raise the "LoRA dropout", and increase the "warmup ratio".

2. If Training Loss shows little change or gradually increases (uncommon), the model has underfit the training data. Training failed. Apply the following recommended methods (in order of recommendation) to optimize and continue training:

   1. Check data quality and ensure thorough data cleaning.
   2. Adjust hyperparameters, including: increase "epochs", raise the "learning rate", increase the "batch size", decrease the "weight decay", lower the "LoRA dropout", and reduce the "warmup ratio".

3. If neither of the above occurs, continue with the subsequent steps.

Task list and operations

On the Model Fine-tuning page, you can view all training jobs. The list displays the following columns: job name/ID, priority (L0-L3), base model, training method, training status, billing method, output, job duration, and creation time. The top-right corner of the page provides the following buttons:

• Training Cost Records: View cost details of training jobs.
• Resource Pool Management (Prepaid): Manage prepaid training unit resource pools.
• Create Training Task: Create a new training job.
• Reset: Clear filter conditions.

The list supports filtering by priority (after selecting a priority, only jobs of that level are displayed). Each row's operation column provides Details, Stop, Logs, and Delete buttons, with availability depending on the job status.

Step 7: Publish the model for deployment

Only SFT fine-tuning supports publishing model snapshots of intermediate training states

After model training completes, based on the configuration in Step 5: Training output, the last checkpoint produced is automatically published to the My Models page under the set model name.

To publish checkpoints from other intermediate training stages, go to the Outputs tab on the training job details page to view the list of all saved checkpoints, select the target checkpoint, and click Publish Model.

The Outputs tab contains the following information: Checkpoint ID, training save info, model publish status, model name, remaining retention time, creation time, and actions.

Published models can be viewed on the My Models page and deployed.

Step 8: Deploy the model

Go to the My Models page to quickly find information such as supported deployment modes and model ID. After deployment, you can evaluate the fine-tuned model. For model deployment details, see Model deployment.

Step 9: Evaluate the model

Use the Alibaba Cloud Model Studio Model Evaluation feature to evaluate the training results of your custom model. For related information, see Help Center: Model evaluation overview.

FAQ

When should I use model fine-tuning?

• Although text generation model fine-tuning can achieve excellent results in specific business scenarios, it has the following limitations:

  • Time-consuming, including: building a large-scale (at least 50 million token) CPT dataset, constructing an effective (1,000+) SFT dataset, collecting enough (100+) bad cases to build an effective DPO dataset, and the slow iteration speed of model optimization.
  • High cost. A fine-tuned model can only be used after deployment, and model deployment billing is high.

• Alibaba Cloud Model Studio recommends that, before using text generation model fine-tuning, you first try using Prompt Engineering (Prompt Engineering) or Function Calling (Function Calling) to customize your application. Model fine-tuning is also usually the "last resort" for improving model performance. Because:

  1. In many tasks, the model may initially perform poorly, but applying the correct prompt techniques can improve the results, and model fine-tuning is not necessarily required.
  2. Iteratively optimizing prompts and plugins is more agile and lower-cost than fine-tuning iterations, because fine-tuning iterations may require re-collecting data, cleaning and optimizing data, collecting bad cases, and conducting customer surveys.
  3. Even if model fine-tuning is ultimately required, the initial prompt engineering and plugin iteration work will not be wasted. This preliminary work can be fully reused when building the fine-tuning dataset (as input for building the dataset).

What should I do if I get a permission error during model fine-tuning?

Contact your platform or workspace administrator to verify and grant the following permissions:

1. Your account must have the page permissions for Model Fine-tuning - Operation, Model Deployment - Operation, and Model Evaluation - Operation in the workspace where you run fine-tuning jobs.

   For related information, see: Account permission management. Console link: Model Studio - Account Management.

2. The workspace where you run fine-tuning jobs must have the permission to perform Model Training (fine-tuning) on the specific model.

   For related information, see: Grant sub-workspaces model invocation, training, and deployment permissions. Console link: Model Studio - Workspace Management.

   On the Model List page in Workspace Management, confirm that the Model Training column of the target model (e.g., Qwen3-8B) shows Authorized.

What should I do if the evaluation results are poor after model fine-tuning?

1. If you use manual evaluation, check whether the evaluation dimensions match your business or scenario.
2. Collect the test cases with poor evaluation results, analyze the causes of the poor results, adjust the training dataset based on the analysis, and continue iterating on the fine-tuned model.
3. Generate DPO cases from the poorly evaluated cases and run DPO training on the model.

How are model fine-tuning, model deployment, and model evaluation billed?

Model fine-tuning is billed by the number of tokens actually consumed during training. A trained model incurs only deployment fees after deployment, not invocation fees. Model evaluation is free of charge. For detailed data, see Model training and deployment billing.

Can I download a model fine-tuned on Alibaba Cloud Model Studio for local deployment?

Models fine-tuned on the Alibaba Cloud Model Studio platform do not support export. They can only be tested and invoked after deployment on Alibaba Cloud Model Studio.

What should I do if model fine-tuning fails with a message about insufficient training data?

When model fine-tuning fails due to insufficient training data, you can try adjusting the Sequence Length (max_length) parameter to resolve it.

Explanation

Sequence length determines the maximum token length supported for a single training data entry. In SFT (supervised fine-tuning) training, data exceeding the sequence length is discarded directly. If the sequence length is set too large, a large amount of training data may be discarded for being too long, leaving insufficient data actually participating in training and causing training failure.

Solution

1. When creating a training job, click Expand Configuration in the Hyperparameter Configuration area and find the max_length (sequence length) parameter.
2. Reduce the sequence length to a smaller value (such as 8192) so that more training data meets the length requirement and participates in model training.
3. Resubmit the training job.

What should I do if the fine-tuned model still responds with the base model identity during testing?

Explanation

Identity settings take effect through the system prompt. If the system prompt used during fine-tuning is not passed in when calling, the model may still respond with the base model identity.

Solution

You can verify the identity setting effect of the fine-tuned model in the following ways:

• Method 1: Pass in the system prompt through an API call

  When calling the API, fill in the identity setting content in the system field of the request, and the model will answer according to the fine-tuned settings.

• Method 2: Test in the Experience Center of the console
  1. In the left-side navigation pane, choose Experience Center > Text Model.
  2. Click the model name and select your deployed fine-tuned model.
  3. Click Model Debugging in the upper-right corner and fill in the identity setting content in the System Prompt input box of the configuration panel that opens.
  4. Send a test question in the dialog box and confirm that the model answers with the fine-tuned identity.