When using for image-to-video, if prompt optimization or calling the official video effects does not meet your custom requirements for specific actions, effects, or styles, use model fine-tuning.
Applicable scope
-
Applicable region: The features described in this document are available only in the China (Beijing) region. You must use an API key created in this region.
-
Grant account permissions: If you use an Alibaba Cloud sub-account (RAM user), you need to grant model invocation, training, and deployment permissions to the sub-account.
-
Fine-tuning method supported: Efficient SFT-LoRA fine-tuning.
-
Models supported for fine-tuning:
-
Image-to-video (first-frame-based): wan2.5-i2v-preview, wan2.2-i2v-flash.
-
Image-to-video (first-and-last-frame-based): wan2.2-kf2v-flash.
-
How to fine-tune a model
Image-to-video (first-frame-based)
Fine-tuning goal: Train a LoRA model for the money rain effect.
Expected result: Input a first-frame image. No prompt is needed. The model generates a video with the money rain effect.
|
Input first-frame image
|
Output video (before fine-tuning) You cannot generate a consistent money rain effect every time using prompts. Motion is not controllable. |
Output video (after fine-tuning) The fine-tuned model reproduces the specific money rain effect from the training dataset without prompts. |
Image-to-video (first-and-last-frame-based)
Fine-tuning goal: Train a LoRA model for the fashion magazine effect.
Expected result: Input a first-frame image and a last-frame image. No prompt is needed. The model generates a video with the fashion magazine effect.
|
Input first-frame image
|
Input last-frame image
|
Output video (before fine-tuning) You cannot generate a consistent fashion magazine effect every time using prompts. Motion is not controllable. |
Output video (after fine-tuning) The fine-tuned model reproduces the specific fashion magazine effect from the training dataset without prompts. |
Before running the code below, retrieve your API key and API host, and configure your API key.
Step 1: Upload your dataset
Upload your local dataset (in .zip format) to Alibaba Cloud Model Studio. Retrieve the file ID (file_id).
Sample training dataset: See the training set for format details.
-
Image-to-video (first-frame-based): wan-i2v-training-dataset.zip.
-
Image-to-video (first-and-last-frame-based): wan-kf2v-training-dataset.zip.
Request example
This example uses the first-frame-based image-to-video model. Only upload the training set. The system automatically splits part of it as the validation set.
curl --location --request POST 'https://dashscope.aliyuncs.com/api/v1/files' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--form 'files=@"./wan-i2v-training-dataset.zip"' \
--form 'purpose="fine-tune"' \
--form 'descriptions="a fine-tune training data file for wan"'Response example
Save the file_id. This is the unique identifier for your uploaded dataset.
{
"data": {
"uploaded_files": [
{
"name": "wan-i2v-training-dataset.zip",
"file_id": "3bff1ef7-f72d-4285-bb75-xxxxxx"
}
],
"failed_uploads": []
},
"request_id": "1f3f1c5b-7418-4976-aaea-xxxxxx"
}Step 2: Fine-tune the model
Step 2.1 Create a fine-tuning job
Start training using the file ID from Step 1.
Request example
Replace <replace with training dataset file ID> fully with the file_id from the previous step.
Image-to-video (first-frame-based)
curl --location 'https://dashscope.aliyuncs.com/api/v1/fine-tunes' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
"model": "wan2.7-i2v",
"training_file_ids": [
"<your-training-dataset-file-id>"
],
"training_type": "efficient_sft",
"hyper_parameters": {
"n_epochs": 50,
"batch_size": 1,
"learning_rate": 2e-5,
"split": 0.9,
"max_split_val_dataset_sample": 5,
"eval_epochs": 20,
"max_pixels": 102400,
"save_total_limit": 10,
"lora_rank": 32,
"lora_alpha": 32
}
}'Image-to-video (first-and-last-frame-based)
curl --location 'https://dashscope.aliyuncs.com/api/v1/fine-tunes' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
"model": "wan2.2-kf2v-flash",
"training_file_ids": [
"<your-training-dataset-file-id>"
],
"training_type": "efficient_sft",
"hyper_parameters": {
"n_epochs": 50,
"batch_size": 4,
"learning_rate": 2e-5,
"split": 0.9,
"max_split_val_dataset_sample": 5,
"eval_epochs": 20,
"max_pixels": 262144,
"save_total_limit": 10,
"lora_rank": 32,
"lora_alpha": 32
}
}'Hyperparameters
Hyperparameters for video generation models
These hyperparameters apply only to video generation models (Wan series). If model performance is poor or training fails to converge, consider adjusting n_epochs or learning_rate. A minimum of 800 total training steps is recommended.
|
Parameter |
Type |
Required |
Description |
Recommended value |
|
batch_size |
int |
Yes |
Batch size. The number of data samples processed in a single training iteration.
|
Varies by model |
|
n_epochs |
int |
Yes |
Number of training epochs. The total number of steps is calculated as: steps = n_epochs × ⌈dataset size / batch_size⌉. A minimum of 800 total steps is recommended. Example: If the dataset has 5 samples and The recommended number of epochs adjusts automatically based on dataset size. Fewer samples require more epochs for sufficient learning, while larger datasets contain more samples per epoch and thus need fewer epochs. The default of 50 epochs is primarily suited for very small datasets (around 2 samples). For datasets of 50-60 videos, training for approximately 3,000-5,000 steps is generally recommended. |
50 |
|
learning_rate |
float |
Yes |
Learning rate. Controls the magnitude of model weight updates during training. A value that is too high can degrade model performance, while a value that is too low may result in insignificant changes. |
2e-5 |
|
eval_epochs |
int |
Yes |
Validation interval. The interval, in epochs, at which to perform validation and save a checkpoint. The value must be ≥ |
20 |
|
max_pixels |
int |
Yes |
Maximum resolution for training videos (total pixels = width × height). The system only resizes videos that exceed this value.
|
Varies by model |
|
split |
float |
No |
Training set split ratio. The proportion of the dataset used for training, with a valid range of (0, 1). This parameter is ignored if |
0.9 |
|
max_split_val_dataset_sample |
int |
No |
Maximum samples for auto-split validation set. The size of the validation set is the smaller of two values: the result of total_samples × (1 − split) or the value of this parameter. |
5 |
|
save_total_limit |
int |
No |
Checkpoint save limit. The maximum number of recent checkpoints to keep. The system deletes older checkpoints once this limit is exceeded. |
10 |
|
lora_rank |
int |
No |
LoRA rank. The rank (dimension) of the LoRA low-rank matrices. Must be a power of 2 (2n), such as 16, 32, or 64. |
32 |
|
lora_alpha |
int |
No |
LoRA alpha. The scaling factor for the LoRA weights. Must be a power of 2 (2n), such as 16, 32, or 64. |
32 |
Response example
Check these three key parameters in output:
-
job_id: Job ID. Use it to check progress. -
finetuned_output: Name of the new fine-tuned model. Use this name for deployment. -
status: Training status. After creating the job, the initial status is PENDING. Training has not started yet.
{
...
"output": {
"job_id": "ft-202511111122-xxxx",
"status": "PENDING",
"finetuned_output": "xxxx-ft-202511111122-xxxx",
...
}
}Step 2.2 Check the fine-tuning job status
Use the job_id from Step 2.1 to check progress. Poll this endpoint until status becomes SUCCEEDED.
This fine-tuning job takes several hours. Exact time depends on the model. Wait patiently.
Request example
Replace <replace with fine-tuning job job_id> in the URL fully with the value of job_id.
curl --location 'https://dashscope.aliyuncs.com/api/v1/fine-tunes/<your-job-id>' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json'Response example
Check these two parameters in the output field:
-
status: When this value becomes SUCCEEDED, training is complete. You can deploy the model. -
usage: Total tokens used for training. Used for billing.
{
...
"output": {
"job_id": "ft-202511111122-xxxx",
"status": "SUCCEEDED",
"usage": 432000,
...
}
}Step 3: Deploy the fine-tuned model
Step 3.1 Deploy the model as an online service
After the fine-tuning job status is SUCCEEDED, deploy the model as an online service.
Request example
Replace <replace with model name model_name> fully with the value of finetuned_output from creating the fine-tuning job.
Image-to-video (first-frame-based)
curl --location 'https://dashscope.aliyuncs.com/api/v1/deployments' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
"model_name": "<your-model-name>",
"aigc_config": {
"use_input_prompt": false,
"prompt": "Provide a video description based on the image content. The description must include the phrase “Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding them. The bills continue to fall, while the camera slowly zooms in, they stretch their arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.”\nOutput template:\nThe video begins with a shot of [subject description]. [Environment description]. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding them. The bills continue to fall, while the camera slowly zooms in, they stretch their arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.\nExample:\nThe video begins with a scene of a young woman on a beach. Her hair is wet, dark brown, curly, and slightly messy. She has a bright smile on her face. The background features crashing waves and distant mountains. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, while the camera slowly zooms in, she stretches her arms upward, neck slightly tilted back, with a surprised expression, completely immersed in this wild money rain.",
"lora_prompt_default": "Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding the main character. The bills continue to fall, while the camera slowly zooms in, the main character stretches their arms upward, neck slightly tilted back, with a surprised expression, completely immersed in this wild money rain."
},
"capacity": 1,
"plan": "lora"
}'Image-to-video (first-and-last-frame-based)
curl --location 'https://dashscope.aliyuncs.com/api/v1/deployments' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
"model_name": "<your-model-name>",
"aigc_config": {
"use_input_prompt": false,
"prompt": "Provide a video description based on the image content. The description must include the phrase “Then they begin the s86b5p transformation.”\nOutput Template:\nThe video begins with a shot of [subject description]. [Environment description]. Then they begin the s86b5p transformation.\nExample:\nThe video begins with a young woman in an outdoor setting. She has short, curly dark brown hair and a friendly smile. She is wearing a black Polo shirt with colorful floral embroidery. The background features green vegetation and distant mountains. Then she begins the s86b5p transformation.",
"lora_prompt_default": "Then they begin the s86b5p transformation."
},
"capacity": 1,
"plan": "lora"
}'Response example
Check these two parameters in output:
-
deployed_model: Name of the deployed model. Use it to check deployment status and call the model. -
status: Deployment status. After deploying the fine-tuned model, the initial status is PENDING. Deployment has not started yet.
{
...
"output": {
"deployed_model": "xxxx-ft-202511111122-xxxx",
"status": "PENDING",
...
}
}Step 3.2 Check deployment status
Poll this endpoint until status becomes RUNNING.
Deploying this fine-tuned model takes about 5–10 minutes.
Request example
Replace <replace with deployed_model> fully with the value of deployed_model from Step 3.1.
curl --location 'https://dashscope.aliyuncs.com/api/v1/deployments/<your-deployed-model-id>' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' curl --location 'https://dashscope.aliyuncs.com/api/v1/deployments/<your-deployed-model-id>' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' Response example
Check these two parameters in the output field:
-
status: When the status becomes RUNNING, the model is deployed successfully. You can start calling it. -
deployed_model: Name of the deployed model.
{
...
"output": {
"status": "RUNNING",
"deployed_model": "xxxx-ft-202511111122-xxxx",
...
}
}Step 4: Call the model to generate videos
After successful model deployment (when the deployment status is RUNNING ), you can start calling the model.
Build a custom dataset
In addition to using the sample datasets in this guide, you can build your own dataset for fine-tuning.
Your dataset must include a training set (required) and may include a validation set (optional, supports automatic splitting from the training set). Package all files into a .zip file. Use only English letters, numbers, underscores, or hyphens in filenames.
Dataset format
Training set: Required
Image-to-video (first-frame-based)
The training set includes first-frame images, training videos, and an annotation file (data.jsonl).
-
Sample training set: wan-i2v-training-dataset.zip.
-
ZIP directory structure:
wan-i2v-training-dataset.zip ├── data.jsonl # Must be named data.jsonl. Max size: 20 MB. ├── image_1.jpeg # Max resolution: 4096×4096. Formats: BMP, JPEG, PNG, WEBP. ├── video_1.mp4 # Max resolution: 4096×4096. Formats: MP4, MOV. ├── image_2.jpeg └── video_2.mp4 -
Annotation file (data.jsonl): Each line is one training sample. It must be a JSON object. Structure:
{ "prompt": "The video begins showing a young woman standing in front of a brick wall covered with ivy. She has long, smooth reddish-brown hair, wearing a white sleeveless dress, a shiny silver necklace, and a smile on her face. The brick wall in the background is covered with green vines, appearing rustic and natural. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, she stretches her arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.", "first_frame_path": "image_1.jpg", "video_path": "video_1.mp4" }
Image-to-video (first-and-last-frame-based)
The training set includes first-frame images, last-frame images, training videos, and an annotation file (data.jsonl).
-
Sample training set: wan-kf2v-training-dataset.zip.
-
ZIP directory structure:
wan-kf2v-training-dataset.zip ├── data.jsonl # Must be named data.jsonl. Max size: 20 MB. ├── image/ # First-frame and last-frame images. │ ├── image_1_first.jpg # Max resolution: 4096×4096. Formats: BMP, JPEG, PNG, WEBP. │ └── image_1_last.png └── video/ # Training videos. ├── video_1.mp4 # Max resolution: 4096×4096. Formats: MP4, MOV. └── video_2.mov -
Annotation file (data.jsonl): Each line is one training sample. It must be a JSON object. Structure:
{ "prompt": "The video begins by showing a young woman in an outdoor setting. She has short, curly dark brown hair, a smile on her face, and looks very friendly. She is wearing a black polo shirt with colorful floral embroidery, with a background of green vegetation and distant mountains. Then she begins the s86b5p transformation.", "first_frame_path": "image/image_1_first.jpg", "last_frame_path": "image/image_1_last.jpg", "video_path": "video/video_1.mp4" }
Validation set: Optional
Image-to-video (first-frame-based)
The validation set includes first-frame images and an annotation file (data.jsonl). Videos are not required. At each evaluation point, the training job automatically calls the model service to generate preview videos using the validation images and prompts.
-
Sample validation set: wan-i2v-valid-dataset.zip.
-
ZIP directory structure:
wan-i2v-valid-dataset.zip ├── data.jsonl # Must be named data.jsonl. Max size: 20 MB. ├── image_1.jpeg # Max resolution: 4096×4096. Formats: BMP, JPEG, PNG, WEBP. └── image_2.jpeg -
Annotation file (data.jsonl): Each line is one validation sample. It must be a JSON object. Structure:
{ "prompt": "The video begins showing a scene of a young man standing in front of a cityscape. He is wearing a black and white checkered jacket over a black hoodie, with a smile on his face and a confident expression. The background is a city skyline at sunset, with a famous domed building and layered roofs visible in the distance, the sky filled with clouds showing warm orange-yellow hues. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding him. The bills continue to fall while the camera slowly zooms in, he stretches his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.", "first_frame_path": "image_1.jpg" }
Image-to-video (first-and-last-frame-based)
The validation set includes first-frame images, last-frame images, and an annotation file (data.jsonl). Videos are not required. At each evaluation point, the training job automatically calls the model service to generate preview videos using the validation images and prompts.
-
Sample validation set: wan-kf2v-valid-dataset.zip.
-
ZIP directory structure:
wan-kf2v-valid-dataset.zip ├── data.jsonl # Must be named data.jsonl. Max size: 20 MB. └── image/ # First-frame and last-frame images. ├── image_1_first.jpg # Max resolution: 4096×4096. Formats: BMP, JPEG, PNG, WEBP. └── image_1_last.jpg -
Annotation file (data.jsonl): Each line is one validation sample. It must be a JSON object. Structure:
{ "prompt": "The video begins showing a scene of a young man standing in front of a cityscape. He is wearing a black and white checkered jacket over a black hoodie, with a smile on his face and a confident expression. The background is a city skyline at sunset, with a famous domed building and layered roofs visible in the distance, the sky filled with clouds showing warm orange-yellow hues. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding him. The bills continue to fall while the camera slowly zooms in, he stretches his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.", "first_frame_path": "image/image_1_first.jpg", "last_frame_path": "image/image_1_last.jpg", }
Data scale and limits
-
Data volume: Provide at least 10 samples. More training data yields better results. We recommend 20–100 samples for stable performance.
-
ZIP archive: When uploading via API, total size ≤ 1 GB.
-
Training image requirements:
-
Formats: BMP, JPEG, PNG, WEBP.
-
Resolution ≤ 4096×4096.
-
No hard limit on individual file size (system pre-processes automatically).
-
-
Training video requirements:
-
Formats: MP4, MOV.
-
Resolution ≤ 4096×4096.
-
No hard limit on individual file size (system pre-processes automatically).
-
Duration per video: wan2.2 models 2–5 seconds recommended; wan2.5 models 2–10 seconds recommended.
-
Data collection and cleaning
1. Define the fine-tuning scenario
Wanxiang supports fine-tuning for image-to-video in these scenarios:
-
Fixed video effects: Teach the model a specific visual change, such as a carousel or magic costume change.
-
Fixed character actions: Improve the model’s ability to reproduce specific body movements, such as dance moves or martial arts techniques.
-
Fixed camera motion: Replicate complex cinematography, such as push, pull, pan, tilt, or orbit shots.
2. Collect raw assets
-
AI generation and filtering: Use the Wanxiang foundation model to batch-generate videos. Manually select high-quality samples that best match your target effect. This is the most common method.
-
Real-world filming: If you need high realism for interactive scenes (e.g., hugging, shaking hands), use real footage.
-
3D software rendering: For effects or abstract animations requiring precise control, use 3D software (e.g., Blender, Cinema 4D).
3. Clean the data
|
Dimension |
Positive Requirements |
Negative example |
|
Consistency |
Core features must be highly uniform. Example: For training “360-degree rotation”, all videos must rotate clockwise at roughly the same speed. |
Mixed directions. The dataset contains both clockwise and counterclockwise rotations. The model does not know which direction to learn. |
|
Diversity |
More variety in subjects and scenes is better. Cover different subjects (men, women, children, animals, buildings) and compositions (close-ups, wide shots, high angles, low angles). Also vary resolution and aspect ratio. |
Single scene or subject. All videos show “a person in red clothes rotating in front of a white wall”. The model may mistakenly treat “red clothes” and “white wall” as part of the effect. It fails when clothes change. |
|
Balance |
Data types should be balanced. If multiple styles exist, their counts should be roughly equal. |
Severe imbalance. 90% are portrait videos, 10% are landscape videos. The model may perform poorly on landscape videos. |
|
Purity |
Clean and clear visuals. Use original assets without interference. |
Interfering elements. Videos contain subtitles, logos, watermarks, obvious black bars, or noise. The model may learn the watermark as part of the effect. |
|
Duration |
Asset duration ≤ target duration. If you want 5-second videos, crop assets to 4–5 seconds. |
Too long assets. You want 5-second videos but feed the model 8-second assets. This leads to incomplete action learning and a choppy feel. |
Video annotation: Write prompts for videos
In the dataset annotation file (data.jsonl), each video has a corresponding prompt. The prompt describes the video content. Prompt quality directly determines what the model learns.
|
Prompt example The video begins with a young woman standing in front of a brick wall covered with ivy. She has long, smooth reddish-brown hair, wearing a white sleeveless dress, a shiny silver necklace, and a smile on her face. The background is a brick wall covered with green vines, appearing rustic and natural. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, she stretches her arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain. |
Prompt writing formula
Prompt = [Subject description] + [Background description] + [Trigger word] + [Motion description]
|
Prompt element |
Description |
Guidance |
Example |
|
Subject description |
Describe people or objects present in the frame |
Required |
The video begins with a young woman... |
|
Background description |
Describe the environment where the subject is located |
Required |
The background is a brick wall covered with green vines... |
|
Trigger word |
A rare, meaningless word |
Recommended |
s86b5p or m01aa |
|
Motion description |
Describe the dynamic changes during the effect in detail |
Recommended |
Countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain... |
How to write good prompts
Follow consistency rules for effect descriptions
For all samples containing the same effect, keep the motion description part as consistent as possible. Apply this rule to both training and validation sets.
-
Purpose: When the model sees
s86b5pfollowed by the same fixed description and always sees money rain, it learns that s86b5p = money rain visual effect. -
Example: Whether the subject is a “young woman” or a “man in a suit”, the prompt ends the same way for the money rain effect: “...then the s86b5p money rain effect begins, countless US dollar bills pour down like a torrential rain...”
Sample type
Prompt content (Note consistency in underlined parts)
Training sample 1
The video begins with a young woman standing in front of a brick wall... (environment description omitted)... Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, she stretches her arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.
Training sample 2
The video begins with a man in a suit in a high-end restaurant... (environment description omitted)... Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding him. The bills continue to fall, he stretches his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.
Validation sample 1
The video begins with a young child standing in front of a cityscape... (environment description omitted)... Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding him. The bills continue to fall while the camera slowly zooms in, he stretches his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.
Use AI to generate prompts
To get high-quality prompts, use multimodal large language models such as Qwen-VL to assist.
-
Use AI to generate initial descriptions
-
Free-form brainstorming (find inspiration): If you’re unsure how to describe the effect, let AI explore freely.
-
Send “
Describe the video content in detail” and observe the model’s output. -
Focus on terms the model uses for motion trajectory (e.g., “pour down like a torrential rain”, “camera slowly zooms in”). These terms can be used for later optimization.
-
-
Fixed-format prompting (standardize output): Once you have a rough idea, design a fixed format to guide AI to generate prompts that match your template.
-
-
Extract effect templates
-
Run the process multiple times on samples with the same effect. Identify high-frequency, accurate phrases used to describe the effect. Extract a generic “effect description”.
-
Copy and paste this standardized effect description into all samples for that effect.
-
Keep unique “subject” and “background” descriptions for each sample. Replace only the “effect description” part with the unified template.
-
-
Manual review
AI may hallucinate or make detection errors. As a final step, perform a manual check. For example, confirm that the descriptions of the entity and background match the actual scene.
Evaluate the model using a validation set
Specify a validation set
A fine-tuning job requires a training set. A validation set is optional. You can choose to let the system split automatically or upload manually. Specify as follows:
Method 1: No validation set uploaded (system splits automatically)
When you create a fine-tuning job, if you do not upload a separate validation set, meaning the validation_file_ids parameter is not passed, the system automatically splits a portion of the training set to use as the validation set based on the following two hyperparameters:
-
split: Proportion of training data to split. For example, 0.9 means 90% for training and 10% for validation. -
max_split_val_dataset_sample: Maximum number of samples for the auto-split validation set.
Validation set split rule: The system selects the smaller value between dataset size × (1 - split) and max_split_val_dataset_sample.
-
Example: Suppose you upload only a training set with 100 samples, split=0.9 (10% for validation), and max_split_val_dataset_sample=5.
-
Theoretical split: 100 × 10% = 10 samples.
-
Actual split: min(10, 5)=5. So the system uses only 5 samples for the validation set.
-
Method 2: Upload a validation set manually (using validation_file_ids)
If you prefer to evaluate checkpoints using your own prepared data instead of relying on random system splits, upload a custom validation set.
Note: Once you choose manual upload, the system ignores the automatic split rules above and validates only with your uploaded data.
Select the best checkpoint for deployment
During training, the system saves periodic “snapshots” (checkpoints). By default, the system outputs the final checkpoint as the fine-tuned model. But intermediate checkpoints may outperform the final version. You can pick the best one for deployment.
The system runs checkpoints on the validation set and generates preview videos at intervals defined by the hyperparameter eval_epochs.
-
Evaluation: Judge by watching the preview videos directly.
-
Selection criteria: Pick the checkpoint with the best visual quality and no motion distortion.
Procedure
Step 1: View preview results for checkpoints
Step 2: Export the checkpoint and get the model name for deployment
Step 3: Deploy and call the model
Go live
In production, if the initial trained model performs poorly (e.g., distorted visuals, weak effects, inaccurate motion), tune along these dimensions:
1. Check data and prompts
-
Data consistency: Consistency is critical. Check for “bad samples” with opposite directions or vastly different styles.
-
Sample count: Increase high-quality data to 20+ samples.
-
Prompt: Ensure trigger words are meaningless, rare terms (e.g., s86b5p). Avoid common words (e.g., running) that cause interference.
2. Adjust hyperparameters: See hyperparameters for details.
-
n_epochs (training epochs)
-
Default: 50. Use the default unless necessary. If adjusting, follow the rule: “Total training steps ≥ 800”.
-
Total steps formula:
steps = n_epochs × ceiling (training set size / batch_size). -
n_epochs minimum formula:
n_epochs = 800 / ceiling (dataset size / batch_size). -
Example: Assume a training set of 5 samples, using the Wan2.5 model (batch_size=2).
-
Steps per epoch: 5 / 2 = 2.5, rounded up to 3. Total epochs: n_epochs = 800 / 3 ≈ 267. This is the recommended minimum. You can increase it as needed.
-
-
-
You can use the default values for learning_rate (learning rate) and batch_size (batch size). These values typically do not require modification.
Billing
-
Model training: Charged.
-
Cost = Total training tokens × Unit price. See model training billing.
-
After training, check the total tokens consumed in the
usagefield of the query fine-tuning job status API.
-
-
Model deployment: Free.
-
Model calls: Charged.
-
Billed according to the standard call price of the fine-tuned foundation model. See model pricing.
-
API reference
FAQ
Q: How are training and validation set data volumes calculated?
A: The training set is required, the validation set is optional. Calculation methods are as follows:
-
When no validation set is provided: The uploaded training set is the "total dataset size". The system automatically splits part of the data for validation.
-
Validation set size =
min(total dataset size × (1 − split), max_split_val_dataset_sample). For an example, see specify a validation set. -
Training set size =
total dataset size − validation set size.
-
-
When a validation set is uploaded manually: The system no longer splits the validation set from the training data.
-
Training set size = Uploaded training set data volume.
-
Validation set size = Uploaded validation set data volume.
-
Q: How do I design a good trigger word?
A: Follow these rules:
-
Use meaningless letter combinations, such as sksstyle, a8z2_bbb.
-
Avoid common English words (e.g., beautiful, fire, dance). This prevents polluting the model's original understanding of these words.
Q: Can fine-tuning change video resolution or duration?
A: No. Fine-tuning learns "content" and "dynamics", not "specifications". Output video format (resolution, frame rate, maximum duration) is still determined by the foundation model.