数字人离线合成openAPI

版本变更

概览

交互示例

接入准备

1. 需要接入方提前准备阿里云账号，并利用阿里云子账号生成对应的AK/SK；

2. 阿里云主账号需要对生成AK/SK的子账号进行RAM授权；

3. 使用阿里云主账号登录平台，签署相关法务协议。

API详情

1. 提交数字人视频合成任务

SubmitProjectTask

1.1. 入参

OpenFrame

OpenVideoScript

OpenLayer

背景图层必须铺满整个画面，默认最底层

• 背景图层的width，height默认等于画面的width，height

• 背景图层的positionX，positionY默认等于0

OpenMaterial 主播/素材

OpenSubtitle 字幕

1.1.1. 字体说明

• Alibaba PuHuiTi - 阿里巴巴普惠体

• FZKai-Z03S - 方正楷体简体

• FZHei-B01S - 方正黑体简体

• FZShuSong-Z01S - 方正书宋简体

• FZFangSong-Z02S - 方正仿宋简体

• SimSun - 宋体

• WenQuanYi MicroHei - 文泉驿微米黑

• WenQuanYi Zen Hei Mono - 文泉驿等宽正黑

• WenQuanYi Zen Hei Sharp - 文泉驿点阵正黑

• Microsoft YaHei - 微软雅黑

• Yuanti SC - 圆体-简常规体

• Roboto Bold - Google Robota字体，不支持中文

• Roboto - Google Robota字体，不支持中文

• KaiTi - 楷体

• SiYuan Heiti - 思源黑体

• SiYuan SimSun - 思源宋体

• HappyZcool-2016 - 站酷快乐体

1.1.2. 坐标说明

• 主播、素材、水印坐标参数均为画面左上角对接图片左上角的距离

• 坐标如果存在小数，舍弃小数， 如下图：X=470, Y=700

1.1.3. 图层超出画面外

• 超出画面的图片需要进行裁剪

• 图片的宽高使用裁剪后的图片宽高

• X,Y轴的坐标超出部分为0

原图：

裁剪后图片：

画面内的效果：

1.1.4. 字幕说明

下图参数示例：

{
    "positionX": 68, // 字幕不允许超出画面外，最小0，最大不超过画面宽度
    "positionY": 392,// 字幕不允许超出画面外，最小0，最大不超过画面高度
    "maxCharLength": 7, // 一行可显示最大字数，建议文本宽度/字体宽度
    "textWidth": 800, // 文本宽度，文本框不能超出画面外
    "textHeight": 110, // 文本高度，必须大于字体大小
    "fontSize": 100, // 字体大小
    "font": "SiYuan SimSun",
    "fontColor": "#090707",
    "alignment": "BottomRight", // 右对齐
    "backgroundColor": "#fefefe"
}

1.1.5. 背景示例

背景图层提供的图片需要充满整个画布，无需传递坐标值，系统默认X=0, Y=0

1.1.6. 变音规则介绍

OpenVideoScript对象的类型为文本（TEXT）的时候可以支持变音配置，需要变音的时候OpenVideoScript的参数speechOpen需设置为true，textContent字段需要设置在变音标签下面，具体规则如下：

完整示例：

String textContent = "<speak> <sub alias="替换">我是文本替换</sub> <phoneme alphabet="ipa"ph="æ">a</phoneme>，apple.是苹果的意思<phoneme alphabet="py"ph="xi1">茜</phoneme>公主 , 南京市<word>长江大桥</word>。今天天气<break time="2.5s" />很好 </speak>"

1.1.7. 学术音标表

1.2. 出参

2. 查询数字人视频合成信息

GetProjectTask

2.1. 入参

2.2. 出参

3. 批量查询数字人视频合成信息

BatchGetProjectTask

3.1. 入参

3.2. 出参

Object说明

4. 合成任务停止

StopProjectTask

4.1. 入参

4.2. 出参

5. 查询资源信息

5.1. 入参

无

5.2. 出参

ResourceInfo

6. 数字人形象模型查询

ListAnchor

6.1. 入参

6.2. 出参

AnchorResponse

7. 声音模板查询

ListVoiceModels

7.1. 入参

7.2. 出参

VoiceModelResponse

对接详情

PHP

对接示例

require 'vendor/autoload.php';
use AlibabaCloud\SDK\Imarketing\V20220704\Models\GetOssUploadSignatureRequest;
use AlibabaCloud\SDK\IntelligentCreation\V20240313\IntelligentCreation;
use Darabonba\OpenApi\Models\Config as AlibabaConfig;

$config = new AlibabaConfig();
$config->accessKeyId = '****';
$config->accessKeySecret = '****';
$config->endpoint = "intelligentcreation.cn-zhangjiakou.aliyuncs.com";

$intelligentCreationClient = new IntelligentCreation($config);

$request = new GetProjectTaskRequest();
$request->taskId = '1111';
try {
  $response = $intelligentCreationClient->getProjectTask($request);
  var_dump($response->toMap());
} catch (TeaError $e) {
  Log::error($e);
}

2.6.0

u-2d06dbf9-cce5-4bcf-91b6-750b97fee3bf-composer-tea.zip

composer require alibabacloud/intelligentcreation-20240313 2.6.0

2.4.0

u-17374653-852f-4536-9c8f-61f96b9890ef-composer-tea.zip

composer require alibabacloud/intelligentcreation-20240313 2.4.0

Java

对接示例

package com.aliyun.intelligentcreation20240313;

import com.aliyun.intelligentcreation20240313.models.*;
import com.aliyun.tea.TeaException;
import com.aliyun.teaopenapi.models.Config;
import com.google.gson.Gson;
import com.google.gson.reflect.TypeToken;

import java.util.HashMap;
import java.util.Map;

/**
 * @author fangming.yu
 * @since 2024/7/5
 */
public class TestProjectTest {

    public static void main(String[] args) throws Exception {
        TestProjectTest testProjectTest = new TestProjectTest();

        try {
           String taskId = testProjectTest.submitProjectTaskRequest();
           testProjectTest.stopProjectTaskRequest(taskId);
           testProjectTest.getProjectTaskRequest(taskId);
           testProjectTest.selectResourceRequest(null);
            testProjectTest.listVoiceModelsRequest();
            System.out.println(taskId);
        } catch (TeaException e) {
            Gson gson = new Gson();
            System.out.println(e.getMessage());
            System.out.println(gson.toJson(e.getData()));
        } catch (Exception e) {

            e.printStackTrace();
        }


    }

    String url = "intelligentcreation.cn-zhangjiakou.aliyuncs.com";
    //初始化配置
    Config config = new Config().setAccessKeyId("**")
            .setAccessKeySecret("**")
            .setEndpoint(url);
    // 创建客户端
    Client client = new Client(config);

    public TestProjectTest() throws Exception {
    }


    String submitProjectTaskRequest() throws Exception {
        Gson gson = new Gson();
        String para = "{\"scaleType\":\"16:9\",\"subtitleTag\":1,\"frames\":[{\"index\":1,\"layers\":[{\"type\":\"BACKGROUND\",\"material\":{\"format\":\"image/png\",\"url\":\"https://yic-pre.oss-cn-hangzhou.aliyuncs.com/common/image/backgroundImage/shu/1.png\"}},{\"type\":\"MATERIAL\",\"material\":{\"format\":\"image/png\",\"url\":\"https://yic-pre.oss-cn-hangzhou.aliyuncs.com/1539704706413278/image/material/1716348615967-%E5%86%B0%E6%B7%87%E6%B7%8B.png\"},\"index\":2,\"positionX\":76,\"positionY\":1169,\"width\":549,\"height\":750},{\"type\":\"ANCHOR\",\"material\":{\"id\":\"759459670880206851\",\"url\":\"https://yic-pre.oss-cn-hangzhou.aliyuncs.com/1539704706413278/image/anchor/1715148681933-55254f81420c4c82954042414136e52d.png\"},\"index\":3,\"positionX\":0,\"positionY\":0,\"width\":1080,\"height\":1920},{\"type\":\"WATERMARK\",\"material\":{\"type\":\"WATERMARK\",\"format\":\"image/png\",\"url\":\"https://yic-pre.oss-cn-hangzhou.aliyuncs.com/common/image/material/%E6%B0%B4%E5%8D%B003%403x.png"},\"index\":4,\"positionX\":248,\"positionY\":280,\"width\":191,\"height\":195}],\"videoScript\":{\"textContent\":\"让我们一同投身于这场激动人心的呼吸之旅，共同见证科技与医学的完美融合，携手共进\",\"type\":\"TEXT\",\"voiceTemplateId\":82,\"volume\":50,\"speedRate\":1.5},\"subtitle\":{\"positionX\":0,\"positionY\":545,\"maxCharLength\":6,\"textWidth\":1080,\"fontSize\":60,\"font\":\"Microsoft YaHei\",\"fontColor\":\"#2a313d\",\"textHeight\":70,\"alignment\":\"BottomCenter\"}}]}";

        Map<String, Object> map = gson.fromJson(para, new TypeToken<Map<String, Object>>() {
        }.getType());

        SubmitProjectTaskRequest request = SubmitProjectTaskRequest.build(map);
        // 请求接口
        SubmitProjectTaskResponse response = client.submitProjectTask(request);

        System.out.println(gson.toJson(response));

        if (response.getStatusCode().equals(200)) {
            System.out.println("submitProjectTaskRequest 请求成功");
            return response.getBody().getTaskId();
        }
        return null;
    }

    void getProjectTaskRequest(String taskId) throws Exception {
        Gson gson = new Gson();

        Map<String, Object> map = new HashMap<>();
        map.put("taskId", taskId);

        GetProjectTaskRequest request = GetProjectTaskRequest.build(map);
        // 请求接口
        GetProjectTaskResponse response = client.getProjectTask(request);

        System.out.println(gson.toJson(response));

        if (response.getStatusCode().equals(200)) {
            System.out.println("getProjectTaskRequest 请求成功");
        }
    }

    void stopProjectTaskRequest(String taskId) throws Exception {
        Gson gson = new Gson();

        Map<String, Object> map = new HashMap<>();
        map.put("taskId", taskId);

        StopProjectTaskRequest request = StopProjectTaskRequest.build(map);
        // 请求接口
        StopProjectTaskResponse response = client.stopProjectTask(request);

        System.out.println(gson.toJson(response));

        if (response.getStatusCode().equals(200)) {
            System.out.println("stopProjectTaskRequest 请求成功");
        }
    }

    void selectResourceRequest(String projectId) throws Exception {
        Gson gson = new Gson();
        Map<String, Object> map = new HashMap<>();
        SelectResourceRequest request = SelectResourceRequest.build(map);
        SelectResourceResponse response = client.selectResource(request);

        System.out.println(gson.toJson(response));
        if (response.getStatusCode().equals(200)) {
            System.out.println("selectResourceRequest 请求成功");
        }
    }

    void listVoiceModelsRequest() throws Exception {
        Gson gson = new Gson();
        Map<String, Object> map = new HashMap<>();
        map.put("useScene", "offlineSynthesis");
        map.put("pageSize", 1);
        map.put("pageNumber", 10);
        ListVoiceModelsRequest request = ListVoiceModelsRequest.build(map);
        ListVoiceModelsResponse response = client.listVoiceModels(request);

        System.out.println(gson.toJson(response));
        if (response.getStatusCode().equals(200)) {
            System.out.println("listVoiceModelsRequest 请求成功");
        }
    }
}

2.6.0

u-2d06dbf9-cce5-4bcf-91b6-750b97fee3bf-java-tea.zip

<dependency>
  <groupId>com.aliyun</groupId>
  <artifactId>intelligentcreation20240313</artifactId>
  <version>2.6.0</version>
</dependency>

2.4.0

u-17374653-852f-4536-9c8f-61f96b9890ef-java-tea.zip

<dependency>
  <groupId>com.aliyun</groupId>
  <artifactId>intelligentcreation20240313</artifactId>
  <version>2.4.0</version>
</dependency>