VideoRetalk API reference-Alibaba Cloud Model Studio(Model Studio)-阿里云帮助中心

Use the VideoRetalk API to generate videos with synchronized lip movements by replacing a person's speech with a provided audio track.

ImportantThis document applies only to the China (Beijing) region. Use an API key from this region.

HTTP API

The VideoRetalk video generation API supports only HTTP calls. To reduce waiting time and avoid request timeouts, an asynchronous processing method is used. Therefore, you need to send two requests to generate a video.

Submit a task: Submit a request to create a video generation task. The API returns a task ID.
Query task status and retrieve results: Use the returned task ID to query the task status and retrieve the generated video.

Prerequisites

You have created an API key and set the API key as an environment variable.

Input limitations

Video requirements:
- File: MP4, AVI, or MOV. Max 300 MB. Duration: 2-120 seconds.
- Properties: Frame rate: 15-60 fps. Encoding: H.264 or H.265 required. Side length: 640-2,048 pixels.
- Content: The video should be a close-up of a person facing the camera. Avoid extreme side angles or faces that are too small.
Audio requirements:
- File: WAV, MP3, or AAC. Max 30 MB. Duration: 2-120 seconds.
- Content: Clear, loud human voice. Remove ambient noise and background music.
Character reference image requirements:
- File: JPEG, JPG, PNG, BMP, or WebP. Max 10 MB.
- Content: Clear, frontal face view. Person must appear in the video. You can use a video screenshot.
File URL requirements:
- Files must be accessible via HTTP links (local paths not supported). Use the platform's temporary storage space to upload local files and create links.
- If you use an oss:// prefix URL obtained after uploading a file to the temporary storage space, you must add the X-DashScope-OssResourceResolve: enable parameter to the HTTP request header. Omitting this parameter causes the error No connection adapters were found for 'oss://...'. For more information, see Upload files to get temporary URLs.

Submit a task

POST https://dashscope.aliyuncs.com/api/v1/services/aigc/image2video/video-synthesis

Request parameters

Field	Type	Location	Required	Description	Example
Content-Type	String	Header	Yes	Request type: application/json	application/json
Authorization	String	Header	Yes	API key, for example: Bearer d1**2a	Bearer d1**2a
X-DashScope-Async	String	Header	Yes	Set to `enable`, indicating that the task is created asynchronously.	enable
model	String	Body	Yes	Specifies the model to call.	videoretalk
input.video_url	String	Body	Yes	URL of the video file uploaded by the user. The URL must be publicly accessible and support the HTTP or HTTPS protocol. You can also get a temporary public URL here. Video file requirements: Size: file ≤ 300 MB Format: MP4, AVI, MOV Duration: 2 seconds < duration < 120 seconds Frame rate: 15 fps ≤ frame rate ≤ 60 fps Encoding: H.264 or H.265 encoding is recommended Side length: 640 ≤ side length ≤ 2048 Content: a close-up of a person facing the camera. Avoid extreme side angles or faces that are too small. If the faces in the video are partially captured or no person appears in the frame, see the FAQ for handling.	http://aaa/bbb.mp4
input.audio_url	String	Body	Yes	URL of the audio file uploaded by the user. The URL must be publicly accessible and support the HTTP or HTTPS protocol. You can also get a temporary public URL here. Audio file requirements: Size: file ≤ 30 MB Format: WAV, MP3, AAC Duration: 2 seconds < duration < 120 seconds. If the video and audio durations do not match, see the FAQ for handling Content: the audio must contain clear, loud human speech, with ambient noise, background music, and other interfering sounds removed.	http://aaa/bbb.wav
input.ref_image_url	String	Body	No	URL of the face reference image uploaded by the user. The URL must be publicly accessible and support the HTTP or HTTPS protocol. You can also get a temporary public URL here. When multiple faces exist in the input video, you can use this parameter to specify the face used for lip-sync matching. If the video contains only one face, no specification is needed. If no face reference image is provided, by default the person with the largest face proportion in the first frame that contains a face is selected as the target. Image file requirements: Content: must contain a clear frontal face of a person who appears in the video File size: file ≤ 10 MB Image size: aspect ratio less than or equal to 2, longest side less than or equal to 4096 Format: JPEG, JPG, PNG, BMP, WebP	http://aaa/bbb.jpg
parameters.video_extension	Boolean	Body	No	Whether to extend the video length when the input audio duration is longer than the video duration. The default value is `false`. You can set it to `true` or `false`. When the value is `true`, the video duration is extended using an alternating "reverse-play, forward-play" pattern of the original video frames until it matches the audio duration. When the value is `false`, the video length is not extended. The generated video has the same duration as the original video, and the audio is truncated.	false
parameters.query_face_threshold	Integer	Body	No	When a face reference image is input, you can adjust the face matching confidence with this value. The value range is 120-200. A smaller value makes matching against the face reference image looser; a larger value makes it stricter. The default value is 170. Note: this parameter does not take effect when input.ref_image_url is empty.	170

Response parameters

Field	Type	Description	Example
output.task_id	String	The job ID of the submitted asynchronous task. The actual job results must be retrieved through the asynchronous task query interface.	a8532587-fa8c-4ef8-82be-0c46b17950d1
output.task_status	String	The job status after the asynchronous task is submitted.	"PENDING"
request_id	String	The unique system code for this request.	7574ee8f-38a3-4b1e-9280-11c33ab46e51

Sample request

curl --location 'https://dashscope.aliyuncs.com/api/v1/services/aigc/image2video/video-synthesis' \
--header 'X-DashScope-Async: enable' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model": "videoretalk",
    "input": {
        "video_url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250717/pvegot/input_video_01.mp4",
        "audio_url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250717/aumwir/stella2-%E6%9C%89%E5%A3%B0%E4%B9%A67.wav",
        "ref_image_url": ""
     },
    "parameters": {
        "video_extension": false
    }
  }'

Sample response

{
    "output": {
	"task_id": "a8532587-fa8c-4ef8-82be-0c46b17950d1",
        "task_status": "PENDING"
    },
    "request_id": "7574ee8f-38a3-4b1e-9280-11c33ab46e51"
}

Query task status and retrieve results

GET https://dashscope.aliyuncs.com/api/v1/tasks/{task_id}

Request parameters

Field	Type	Location	Required	Description	Example
Authorization	String	Header	Yes	API key, for example: Bearer d1**2a	Bearer d1**2a
task_id	String	Url Path	Yes	The task_id of the job to query, which is the value returned by the task submission interface.	a8532587-fa8c-4ef8-82be-0c46b17950d1

Response parameters

Field	Type	Description	Example
output.task_id	String	The task_id of the queried job.	a8532587-fa8c-4ef8-82be-0c46b17950d1
output.task_status	String	The job status of the queried job.	Task statuses: PENDING: queuing PRE-PROCESSING: pre-processing RUNNING: processing POST-PROCESSING: post-processing SUCCEEDED: succeeded FAILED: failed UNKNOWN: the job does not exist or its status is unknown
output.video_url	String	The generated video. The video_url is valid for 24 hours after the job is completed.	https://xxx/1.mp4"
usage.video_duration	Float	The duration of the video generated by this request, in seconds.	"video_duration": 10.23
usage.video_ratio	String	The aspect ratio type of the video generated by this request. The value is standard; by default, the output follows the original video ratio.	"video_ratio": "standard"
usage.size	String	The resolution of the video generated by this request, which is the same as the input video resolution.	"size": "1080*1920"
usage.fps	Integer	The frame rate of the video generated by this request, which is the same as the input video frame rate.	"fps": 25
request_id	String	The unique system code for this request.	7574ee8f-38a3-4b1e-9280-11c33ab46e51

Sample request

curl -X GET 'https://dashscope.aliyuncs.com/api/v1/tasks/<YOUR_TASK_ID>' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY"

Sample response

{
    "request_id": "87b9dce5-7f36-4305-a347-xxxxxx",
    "output": {
        "task_id": "3afd65eb-9604-48ea-8a91-xxxxxx",
        "task_status": "SUCCEEDED",
        "submit_time": "2025-09-11 20:15:29.887",
        "scheduled_time": "2025-09-11 20:15:36.741",
        "end_time": "2025-09-11 20:16:40.577",
        "video_url": "http://dashscope-result-sh.oss-cn-shanghai.aliyuncs.com/xxx.mp4?Expires=xxx"
    },
    "usage": {
        "video_duration": 7.2,
        "size": "1080*1920",
        "video_ratio": "standard",
        "fps": 25
    }
}

Sample error response

{
    "request_id": "7574ee8f-38a3-4b1e-9280-11c33ab46e51",
    "output": {
        "task_id": "a8532587-fa8c-4ef8-82be-0c46b17950d1",
        "task_status": "FAILED",
        "code": "xxx",
        "message": "xxxxxx"
    }
}

Error codes

See Error codes for general status codes.

In addition, this model has the following specific error codes:

HTTP return code	Error code	Error message	Description
400	InvalidParameter	Field required: xxx	Missing or incorrect request parameter.
400	InvalidURL.ConnectionRefused	Connection to ${url} refused, please provide avaiable URL	Download rejected. Provide an available URL.
400	InvalidURL.Timeout	Download ${url} timeout, please check network connection.	Download timed out (timeout: 60s).
400	InvalidFile.Size	Invalid file size. The video/audio/image file size must be less than **MB.	File must be smaller than ** MB.
400	InvalidFile.Format	Invalid file format，the request file format is one of the following types: MP4, AVI, MOV, MP3, WAV, AAC, JPEG, JPG, PNG, BMP, and WEBP.	Invalid file format. Supported: Video (MP4/AVI/MOV), Audio (MP3/WAV/AAC), Image (JPG/JPEG/PNG/BMP/WebP).
400	InvalidFile.Resolution	Invalid video resolution. The height or width of video must be 640 ~ 2048.	Video side length must be 640-2048 pixels.
400	InvalidFile.FPS	Invalid video FPS. The video FPS must be 15 ~ 60.	Video frame rate must be 15-60 fps.
400	InvalidFile.Duration	Invalid file duration. The video/audio file duration must be 2s ~ 120s.	Video/audio duration must be 2-120 seconds.
400	InvalidFile.ImageSize	The size of image is beyond limit.	Image size exceeds the limit. The image aspect ratio must be no greater than 2, and the longest side must be no greater than 4096
400	InvalidFile.Openerror	Invalid file, cannot open file as video/audio/image.	The video/audio/image file cannot be opened.
400	InvalidFile.Content	The input image has no human body or multi human bodies. Please upload other image with single person.	Image contains no person or multiple people.
400	InvalidFile.FaceNotMatch	There are no matched face in the video with the provided reference image.	Reference face does not match any video face.

FAQ

What happens if the input audio and video durations do not match?

By default, the shorter of the audio and video durations is used for truncation. When the input audio duration is longer than the video duration and you want to generate based on the audio length, set the video extension parameter (parameters.video_extension) to true. The algorithm will extend the video duration using an alternating "reverse-play, forward-play" pattern of the original video frames until it matches the audio.
What happens if the input audio contains silence?

During silent segments of the audio, the person in the video is expected to close their mouth.
What happens if the input video contains no person or partially captured faces?

If the audio contains human speech but the frame shows no person or no mouth movements, the original video frames are preserved and the audio plays normally.
What happens if the input video contains multiple people?

Only one person can be replaced. The algorithm identifies the specified face based on the input face reference image (input.ref_image_url). If no face reference image is provided, the largest face in the first frame that contains a face is selected by default.