快速搭建数字人视频生成接口
更新时间：
借助 Live Portrait（实时肖像）在 ComfyUI 中快速搭建文本生成数字人视频（文本生成视频，aigc 生成视频）
方案概览
用户自己手动在智作工坊控制台创建对应的应用，调试需要的工作流，并且发布为一个可以调用的接口，之后使用对应的语言的 SDK 接入，并且调用获取到 ComfyUI 任务的结果。基于智作工坊的控制台和提供的各语言版本 SDK，可以快速调试自己需要的工作流，并且部署为API接口供调用。
部署准备
开始部署前，请按以下指引完成账号申请、账号充值。
准备账号
如果您还没有阿里云账号，请访问阿里云账号注册页面，根据页面提示完成注册。阿里云账号是您使用云资源的付费实体，因此是部署方案的必要前提。
开通智作工坊
参考智作工坊服务开通文档：服务开通
快速体验
准备工作流
保存下面的内容到t2v.json 文件：
{
"last_node_id": 199,
"last_link_id": 31,
"nodes": [
{
"id": 8,
"type": "VHS_LoadVideo",
"pos": {
"0": 70,
"1": 0
},
"size": [
250,
620
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [
{
"name": "meta_batch",
"type": "VHS_BatchManager",
"link": null,
"label": "批次管理"
},
{
"name": "vae",
"type": "VAE",
"link": null
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
21
],
"shape": 3,
"label": "图像"
},
{
"name": "frame_count",
"type": "INT",
"links": null,
"slot_index": 1,
"shape": 3,
"label": "帧计数"
},
{
"name": "audio",
"type": "AUDIO",
"links": [
31
],
"slot_index": 2,
"shape": 3,
"label": "音频"
},
{
"name": "video_info",
"type": "VHS_VIDEOINFO",
"links": null,
"shape": 3,
"label": "视频信息"
}
],
"properties": {
"Node name for S&R": "VHS_LoadVideo"
},
"widgets_values": {
"video": "_sp_auto_upload_01j7fhv11v4vmfydkhnpz8h67w.mp4",
"force_rate": 0,
"force_size": "Disabled",
"custom_width": 512,
"custom_height": 512,
"frame_load_cap": 0,
"skip_first_frames": 0,
"select_every_nth": 1,
"choose video to upload": "image",
"videopreview": {
"hidden": false,
"paused": false,
"params": {
"filename": "_sp_auto_upload_01j7fhv11v4vmfydkhnpz8h67w.mp4",
"type": "input",
"format": "video/mp4",
"frame_load_cap": 0,
"skip_first_frames": 0,
"force_rate": 0,
"select_every_nth": 1
},
"muted": false
}
}
},
{
"id": 196,
"type": "LoadImage",
"pos": {
"0": 30,
"1": 680
},
"size": {
"0": 320,
"1": 310
},
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
13
],
"shape": 3,
"label": "图像"
},
{
"name": "MASK",
"type": "MASK",
"links": null,
"shape": 3,
"label": "遮罩"
}
],
"properties": {
"Node name for S&R": "LoadImage"
},
"widgets_values": [
"_sp_auto_upload_01j7fhtp8s3k82dyma6479jk9b.png",
"image"
]
},
{
"id": 165,
"type": "ImageResizeKJ",
"pos": {
"0": 390,
"1": 680
},
"size": {
"0": 320,
"1": 270
},
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 13,
"label": "图像"
},
{
"name": "get_image_size",
"type": "IMAGE",
"link": null,
"label": "参考图像大小"
},
{
"name": "width_input",
"type": "INT",
"link": null,
"widget": {
"name": "width_input"
},
"label": "宽度"
},
{
"name": "height_input",
"type": "INT",
"link": null,
"widget": {
"name": "height_input"
},
"label": "高度"
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
17,
22
],
"shape": 3,
"label": "图像"
},
{
"name": "width",
"type": "INT",
"links": null,
"shape": 3,
"label": "宽度"
},
{
"name": "height",
"type": "INT",
"links": null,
"shape": 3,
"label": "高度"
}
],
"properties": {
"Node name for S&R": "ImageResizeKJ"
},
"widgets_values": [
512,
512,
"lanczos",
true,
2,
0,
0,
"disabled"
]
},
{
"id": 198,
"type": "LivePortraitLoadMediaPipeCropper",
"pos": {
"0": 350,
"1": 320
},
"size": {
"0": 320,
"1": 80
},
"flags": {},
"order": 2,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "cropper",
"type": "LPCROPPER",
"links": [
16
],
"shape": 3,
"label": "裁剪框架"
}
],
"properties": {
"Node name for S&R": "LivePortraitLoadMediaPipeCropper"
},
"widgets_values": [
"CUDA",
true
]
},
{
"id": 190,
"type": "LivePortraitProcess",
"pos": {
"0": 1120,
"1": 110
},
"size": {
"0": 430,
"1": 330
},
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "pipeline",
"type": "LIVEPORTRAITPIPE",
"link": 18,
"label": "LivePortrait管线"
},
{
"name": "crop_info",
"type": "CROPINFO",
"link": 19,
"label": "裁剪信息"
},
{
"name": "source_image",
"type": "IMAGE",
"link": 20,
"label": "原图像"
},
{
"name": "driving_images",
"type": "IMAGE",
"link": 21,
"label": "驱动图像"
},
{
"name": "opt_retargeting_info",
"type": "RETARGETINGINFO",
"link": null,
"label": "重定向信息(可选)"
}
],
"outputs": [
{
"name": "cropped_image",
"type": "IMAGE",
"links": [
23
],
"shape": 3,
"label": "裁剪图像"
},
{
"name": "output",
"type": "LP_OUT",
"links": [
24
],
"shape": 3,
"label": "LivePOrtrait输出"
}
],
"properties": {
"Node name for S&R": "LivePortraitProcess"
},
"widgets_values": [
false,
0.03,
true,
1,
"constant",
"relative",
0.000003,
false,
1
]
},
{
"id": 191,
"type": "LivePortraitComposite",
"pos": {
"0": 1600,
"1": 130
},
"size": {
"0": 360,
"1": 90
},
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{
"name": "source_image",
"type": "IMAGE",
"link": 22,
"label": "原图像"
},
{
"name": "cropped_image",
"type": "IMAGE",
"link": 23,
"label": "裁剪图像"
},
{
"name": "liveportrait_out",
"type": "LP_OUT",
"link": 24,
"label": "LivePOrtrait输出"
},
{
"name": "mask",
"type": "MASK",
"link": null,
"label": "遮罩"
}
],
"outputs": [
{
"name": "full_images",
"type": "IMAGE",
"links": [
14
],
"shape": 3,
"label": "图像"
},
{
"name": "mask",
"type": "MASK",
"links": null,
"shape": 3,
"label": "遮罩"
}
],
"properties": {
"Node name for S&R": "LivePortraitComposite"
}
},
{
"id": 189,
"type": "LivePortraitCropper",
"pos": {
"0": 740,
"1": 350
},
"size": {
"0": 330,
"1": 240
},
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "pipeline",
"type": "LIVEPORTRAITPIPE",
"link": 15,
"label": "LivePortrait管线"
},
{
"name": "cropper",
"type": "LPCROPPER",
"link": 16,
"label": "裁剪框架"
},
{
"name": "source_image",
"type": "IMAGE",
"link": 17,
"label": "原图像"
}
],
"outputs": [
{
"name": "cropped_image",
"type": "IMAGE",
"links": [
20
],
"shape": 3
},
{
"name": "crop_info",
"type": "CROPINFO",
"links": [
19
],
"shape": 3,
"label": "裁剪信息"
}
],
"properties": {
"Node name for S&R": "LivePortraitCropper"
},
"widgets_values": [
512,
2.3000000000000003,
0,
-0.125,
0,
"large-small",
true
]
},
{
"id": 168,
"type": "VHS_VideoCombine",
"pos": {
"0": 1660,
"1": 300
},
"size": [
210,
500
],
"flags": {},
"order": 8,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 14,
"label": "图像"
},
{
"name": "audio",
"type": "AUDIO",
"link": 31,
"label": "音频"
},
{
"name": "meta_batch",
"type": "VHS_BatchManager",
"link": null,
"label": "批次管理"
},
{
"name": "vae",
"type": "VAE",
"link": null
}
],
"outputs": [
{
"name": "Filenames",
"type": "VHS_FILENAMES",
"links": null,
"shape": 3,
"label": "文件名"
}
],
"properties": {
"Node name for S&R": "VHS_VideoCombine"
},
"widgets_values": {
"frame_rate": 30,
"loop_count": 0,
"filename_prefix": "LivePortrait/",
"format": "video/h264-mp4",
"pix_fmt": "yuv420p",
"crf": 19,
"save_metadata": true,
"pingpong": false,
"save_output": true,
"videopreview": {
"hidden": false,
"paused": false,
"params": {
"filename": "_sp_auto_upload_https://sd-fc-prod.oss-cn-shanghai.aliyuncs.com/comfy%2Foutput%2FLivePortraitd869915d5e824dedaf9feed80af48eb4_00001-audio.mp4?Expires=1728618753&OSSAccessKeyId=LTAI5tBsDGhKWWwT6sC79mEH&Signature=5tFCnnJIj8aENes8435sLmKbFwM%3D",
"format": "video/h264-mp4",
"frame_rate": 30,
"object_key": "comfy/output/LivePortraitd869915d5e824dedaf9feed80af48eb4_00001-audio.mp4",
"subfolder": "",
"type": "output"
},
"muted": false
}
}
},
{
"id": 1,
"type": "DownloadAndLoadLivePortraitModels",
"pos": {
"0": 80,
"1": -150
},
"size": {
"0": 320,
"1": 80
},
"flags": {},
"order": 3,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "live_portrait_pipe",
"type": "LIVEPORTRAITPIPE",
"links": [
15,
18
],
"shape": 3,
"label": "LivePortrait管线"
}
],
"properties": {
"Node name for S&R": "DownloadAndLoadLivePortraitModels"
},
"widgets_values": [
"fp16",
"human"
]
}
],
"links": [
[
13,
196,
0,
165,
0,
"IMAGE"
],
[
14,
191,
0,
168,
0,
"IMAGE"
],
[
15,
1,
0,
189,
0,
"LIVEPORTRAITPIPE"
],
[
16,
198,
0,
189,
1,
"LPCROPPER"
],
[
17,
165,
0,
189,
2,
"IMAGE"
],
[
18,
1,
0,
190,
0,
"LIVEPORTRAITPIPE"
],
[
19,
189,
1,
190,
1,
"CROPINFO"
],
[
20,
189,
0,
190,
2,
"IMAGE"
],
[
21,
8,
0,
190,
3,
"IMAGE"
],
[
22,
165,
0,
191,
0,
"IMAGE"
],
[
23,
190,
0,
191,
1,
"IMAGE"
],
[
24,
190,
1,
191,
2,
"LP_OUT"
],
[
31,
8,
2,
168,
1,
"*"
]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 0.5559917313492244,
"offset": [
249.9019008932993,
305.3913911427086
]
}
},
"version": 0.4
}新建工作流
点击新建工作流
填写工作流信息，以及选择需要的工作流模板，这里选择本地工作流文件导入，点击确认
点击之后将会跳转到工作流编辑界面，等待页面加载完毕之后，显示如下：
运行查看效果
点击右上角运行工作流查看效果
等待工作流执行
执行结束，查看结果如下图
部署工作流为接口
发布工作流
确认工作流可以执行得到结果后，点击发布，进入发布菜单
填入版本描述：填入文生视频测试
设置工作流入参：在这里可以设置接口服务对外暴露的参数，其他未暴露的参数值默认使用工作流中的值。若调用时为传递设置的参数，也默认是工作流中的参数。
点击下方添加按钮，可以添加新的参数
添加之后确认设置好的参数，含义如下，注意
#3会显示在工作流右上角，方便对应关系
如果需要修改参数
点击字段对应的编辑，可以编辑参数别名
编辑态：
编辑好之后，回车保存
发布工作流：确认无误之后，点击提交发布。
发布之后可以看到生成对应的接口版本管理信息
为工作流指定别名
点击别名管理tab，点击新建别名按钮，填写接口调用别名名称，选择别名当前对应的主版本，后续更新接口版本只需要在此替换版本即可，无需修改代码中的版本ID。
点击确认，得到如下效果：
复制对应的别名备用，这里是
t2v_0912
应用创建获取AKSK
接口发布之后，想要调用接口，需要先新建应用，使用应用的AKSK通过SDK调用。打开智作工坊控制台应用管理tab页
点击右上角创建应用
填写应用名称以及应用类型，点击确认
确认后会自动跳转应用详情页，复制对应的AK、SK备用
代码调用
获取到应用的 AKSK、以及工作流别名，需要使用对应语言的SDK调用，这里以Python为例，其余语言参考各语言SDK参考
参考指南完成接入：Python接入指南
替换 main.py 其中的 AKSK、工作流ID、工作流别名和参数如下：
from client import Client from proto import ComfyRequest, ComfyResponse, PredictResultResponse, ProgressResponse import time import json cli = Client( endpoint="openai.edu-aliyun.com", app_key="替换AK", app_secret="替换SK" ) # 原始调用方法 def call(url, body, method='POST', headers=None): # 忽略 def comfy_prompt(prompt: ComfyRequest, custom_resource_config_id='default') -> ComfyResponse: # 忽略内容 if __name__ == '__main__': begin = time.time() # 工作流别名 alias_id = "替换为工作流别名" workflow_id = "替换为控制台上的工作流 ID" params = { "prompt": "提示词" } result = comfy_prompt(ComfyRequest(alias_id=alias_id, workflow_id=workflow_id, inputs=params)) print("生图结果：" + str(result)) print("时间消耗： %.2fs" % (time.time() - begin))
查看效果
执行之后可以获取结果，示例结果为：
{
"status": 10,
"err_code": null,
"err_message": null,
"sub_err_code": null,
"sub_err_message": null,
"api_invoke_id": "i_66e2605219c46d002564ff0f",
"data": {
"task_id": "01j7j4epagfdab24jbsknt9q97",
"images": [
"http://xxxxx"
],
"info": {},
"parameters": null,
"status": "succeeded",
"imgs_bytes": null
}
}文档内容是否对您有帮助？