Fine-tune models in the console

SFT training set

The SFT training set uses the ChatML (Chat Markup Language) format, which supports multi-turn conversations and various role settings.

The OpenAI parameters name and weight are not supported. All assistant outputs are used for training.

# A typical line of expanded JSON training data:
{"messages": [
  {"role": "system", "content": "System input 1"}, 
  {"role": "user", "content": "User input 1"}, 
  {"role": "assistant", "content": "Desired model output 1"}, 
  {"role": "user", "content": "User input 2"}, 
  {"role": "assistant", "content": "Desired model output 2"}
  ...
]}

For details on the system, user, and assistant roles, see Overview. For training dataset samples, see SFT-ChatML Format Sample.jsonl and SFT-ChatML Format Sample.xlsx. Note that the XLS and XLSX formats support only single-turn conversations.

In a single training example, all assistant messages support the "loss_weight" parameter. This parameter sets the message's relative importance during training. The value ranges from 0.0 to 1.0, where a higher value indicates greater importance.

This is a beta parameter. To use it, please contact your account manager.

 {"role": "assistant", "content": "Desired model output 1", "loss_weight": 1.0}, 
 {"role": "assistant", "content": "Desired model output 2", "loss_weight": 0.5}

SFT thinking model

The training data supports multi-turn conversations and various role settings, but the model trains only on the last assistant output. Each expanded line of training data has the following structure:

The newline characters \n before and after the tag must be preserved.

# A single line of training data (JSON format) has the following typical structure when expanded:
{"messages": [
  {"role": "system", "content": "System input 1"}, 
  {"role": "user", "content": "User input 1"}, 
  {"role": "assistant", "content": "Model output 1"}, --Intermediate assistant outputs must not include <think> tags.
   ...
  {"role": "user", "content": "User input 2"}, 
  {"role": "assistant", "content": "<think>\nDesired thinking content 2\n</think>\n\nDesired output 2"} --Include thinking content only in the final assistant output. 
]}

For details on the system, user, and assistant roles, see Overview. For a training dataset sample, see SFT-Deep Thinking Content Sample.jsonl.

You can also provide training examples without the <think> tag. If you use this method, we do not recommend enabling thinking mode for inference after training.

{"role": "assistant", "content": "Desired model output 2"}  --Instructs the model not to enable thinking.

In a single training example, the final assistant message supports the "loss_weight" parameter. This parameter sets the message's relative importance during training. The value ranges from 0.0 to 1.0, where a higher value indicates greater importance.

This is a beta parameter. To use it, please contact your account manager.

 {"role": "assistant", "content": "<think>\nDesired thinking content 2\n</think>\n\nDesired output 2", "loss_weight": 1.0}

SFT for visual understanding

The OpenAI parameters name and weight are not supported. All assistant outputs are used for training.

For the differences between system, user, and assistant, see Overview. The following is an example of training data in the ChatML format:

If you pass a system message, its content must use the array format [{"text":"..."}]. The string format "content":"string" is not supported.

# A typical line of expanded JSON training data:
{"messages": [
  {"role": "system", "content": [{"text": "System input"}]},
  {"role": "user", "content": [{"text": "User input 1"}, {"image": "image_filename_1.jpg", "resized_width": 200, "resized_height": 200}]},
  {"role": "assistant", "content": [{"text": "Desired model output 1"}]},
  {"role": "user", "content": [{"text": "User input 2"}, {"video": "video_filename_1.mp4", "fps": 3.0, "resized_width": 200, "resized_height": 200, "video_start": 0.0, "video_end": 3.0}]},
  {"role": "assistant", "content": [{"text": "Desired model output 2"}]},
  {"role": "user", "content": [{"text": "User input 2"}, {"video": ["0.jpg", "1.jpg", "2.jpg", "3.jpg"], "sample_fps": 5.0, "resized_width": 200, "resized_height": 200}]},
  {"role": "assistant", "content": [{"text": "Desired model output 2"}]},
  ...
]}

DPO dataset

The DPO dataset uses the ChatML format. Each expanded line of training data has the following structure:

For details on the system, user, and assistant roles, see Overview. For a training dataset sample, see DPO ChatML Format Sample.jsonl.

# A typical line of expanded JSON training data:
{"messages": [
  {"role": "system", "content": "System input"},
  {"role": "user", "content": "User input 1"},
  {"role": "assistant", "content": "Model output 1"},
  {"role": "user", "content": "User input 2"},
  {"role": "assistant", "content": "Model output 2"},
  {"role": "user", "content": "User input 3"}
 ],
 "chosen":
   {"role": "assistant", "content": "The chosen model output for Input 3"},
 "rejected":
   {"role": "assistant", "content": "The rejected model output for Input 3"}}

The model uses all content within messages as input. DPO then trains the model to prefer the chosen response over the rejected response for the final user input.

To include thinking content, wrap it with the <think> tag:

{"role": "assistant", "content": "<think>Desired model thinking content</think>Desired model output"}

The "chosen" module of a single training data sample supports the "loss_weight" parameter, which sets the relative importance of that sample during training. (The valid range is 0.0 to 1.0, where a larger value indicates higher importance.)

This is a beta parameter. To use it, please contact your account manager.

 "chosen":
   {"role": "assistant", "content": "The chosen model output for Input 3", "loss_weight": 1.0},

CPT training set

The CPT training set uses the plain text format. Each line of training data has the following structure:

{"text":"Text content"}

For a training dataset sample, see CPT-Text Generation Training Set Sample.jsonl.