数字人实时互动openAPI

版本变更

概览

交互示例

接入准备

1. 需要接入方提前准备阿里云账号，并利用阿里云子账号生成对应的AK/SK；

2. 阿里云主账号需要对生成AK/SK的子账号进行RAM授权；

3. 使用阿里云主账号登录平台，签署相关法务协议。

API详情

1. 批量查询数字人项目信息

入参 ListAvatarProject

出参

Object对象字段说明

2. 查询数字人项目详情

2.1. 入参 QueryAvatarProjectRequest

2.2. 出参 QueryAvatarProjectResponse

2.2.1. frame参数说明

2.2.2. layer参数说明

2.2.3. material参数说明

2.2.4. videoScript参数说明

3. 启动会话

入参 StartAvatarSessionRequest

出参 StartAvatarSessionResponse

channelToken 字段解析

{
  "channelId":"123",//频道ID
  "token":"", // 令牌
  "expireTime":600,//过期时间（单位秒）
  "nonce":"",//随机数
  "userId":"",//用户ID
  "appId":""//应用ID
}

4. 4. 停止会话

入参 StopAvatarSessionRequest

出参 StopAvatarSessionResponse

5. 有效资源查询

入参 QueryAvatarResourceRequest

出参 QueryAvatarResourceResponse

QueryResourceInfo

6. 保存数字人项目

6.1. 入参 SaveAvatarProjectRequest

6.1.1. frame参数说明

6.1.2. layer参数说明

6.1.3. material参数说明

6.1.4. videoScript参数说明

6.1.5. 坐标说明

• 坐标参数均为画面左上角对接图片左上角的距离

• 坐标如果存在小数，舍弃小数， 如下图：X=470, Y=700

6.2. 出参

7. 操作数字人项目

7.1. 入参 OperateAvatarProjectRequest

7.2. 出参

8. 查询项目会话状态

8.1. 入参 QuerySessionInfoRequest

8.2. 出参

会话信息

9. 查询数字人形象模型

ListAnchor

9.1. 入参

9.2. 出参

AnchorResponse

10. 查询声音模板

ListVoiceModels

10.1. 入参

10.2. 出参

VoiceModelResponse

WebSocket实时互动

WebSocket对接整体流程

1. 开启会话，获取websocket链接和RTC channel信息；

2. 建立websocket连接；

3. 循环发送"通道准备"数据包，直至收到"通道准备完成"；

4. 使用RTC SDK和channel信息拉取视频流；

5. （可选）发送开场白驱动数据包，进行开场白互动；

6. （可选）发送文本驱动数据包进行文本驱动数字人；

7. （可选）发送音频驱动头包+数据包+尾包进行音频识别后驱动数字人；

8. （可选）发送音频驱动数据包进行音频驱动数字人；

9. 停止会话

10. 关闭websocket连接；

建立websocket连接代码示例：

@ClientEndpoint
public class WebSocketClient {

    private Session userSession = null;

    public WebSocketClient(URI endpointURI) {
        try {
            WebSocketContainer container = ContainerProvider.getWebSocketContainer();
            container.connectToServer(this, endpointURI);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @OnOpen
    public void onOpen(Session userSession) {
        this.userSession = userSession;
        System.out.println("Connected to server");
    }

    @OnClose
    public void onClose(Session userSession, CloseReason reason) {
        this.userSession = null;
        System.out.println("Disconnected from server: " + reason);
    }

    @OnMessage
    public void onMessage(byte[] message) {

        //1. 获取第1字节帧号
        int frame = Byte.toUnsignedInt(message[0]);

        //2. 获取第2字节数据类型
        int dataType = Byte.toUnsignedInt(message[1]);

        //3. 获取第3字节业务类型
        int bizType = Byte.toUnsignedInt(message[2]);

        //4. 获取第4-11字节序列号
        byte[] serialNumberBytes = Arrays.copyOfRange(message, 3, 11);
        String serialNumberString = new String(serialNumberBytes, StandardCharsets.UTF_8);

        //5. 获取剩余内容数据，数据结构需要结合具体业务类型文档
        byte[] contentJsonBytes = Arrays.copyOfRange(message, 11, message.length);
        String contentJsonString = new String(contentJsonBytes, StandardCharsets.UTF_8);

        //6. 执行相应业务逻辑......
    }

    public void sendMessage(byte[] message) {
        try {
            if (this.userSession != null) {
                this.userSession.getBasicRemote().sendBinary(ByteBuffer.wrap(message));
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) throws InterruptedException {

        // 1. 建立ws连接
        WebSocketClient client = new WebSocketClient(URI.create("ws://127.0.0.1:7005/v1/interaction?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzZXNzaW9uSWQiOiI4ZjM3YmVhOC1lNDYxLTRhNzktODczZS00Yzg2ZTI2OWU4YTAiLCJleHAiOjE3MjU1MTY3NTIsImFsaXl1bk1haW5JZCI6IjE1Mzk3MDQ3MDY0MTMyNzgifQ.PAvCT7gY1VWNGJQc1cmubVLd76INRVZnhHdoBRV9-Rc&sessionId=8f37bea8-e461-4a79-873e-4c86e269e8a0"));

        // 2. 通道准备业务
        //      2.1 组装数据包
        byte frameId = (byte) 0;
        byte dataType = (byte) 1;
        byte bizType = (byte) 131;
        // 正常业务使用随机生成8位序列号即可
        byte[] serialNumber = new byte[8];
        byte[] contentBytes = new byte[0];

        byte[] message = new byte[11 + contentBytes.length];
        ByteBuffer bf = ByteBuffer.wrap(message)
        .put(frameId)
        .put(dataType)
        .put(bizType)
        .put(serialNumber)
        .put(contentBytes);
        message = bf.array();

        // 3. 发送数据包
        client.sendMessage(message);

    }

}

WebSocket消息结构

1. 整体结构

整体数据包格式为： 11字节包头+N字节包内容

前11字节包头格式为：1字节帧类型+1字节数据类型+1字节业务类型+8字节序列号（调用端随机生成，排查问题使用）

public class InteractionWebSocketProtocol {

    // 1字节帧类型
    private InteractionProtocolFrameEnum interactionProtocolFrameEnum;

    // 1字节数据类型
    private DataTypeEnum dataType;

    // 1字节业务类型
    private BusinessTypeEnum businessType;

    // 8字节序列号
    private String serialNumber;

    // N字节消息 二进制格式
    private byte[] messageBytes;

}

2. 2. 帧类型

3. 数据类型

4. 业务类型

业务类型：

业务内容出入参格式：

4.1. 查询开场白

请求：

返回：

4.2. 打断数字人播报

请求：

返回：暂无

4.3. 文本驱动数字人

请求：

返回：

4.4. 音频识别并驱动【开始】

请求：

返回：

4.5. 音频识别并驱动【数据】

请求：二进制数据，pcm格式

返回：提问模式的音频识别结果返回见接口说明 第3点接收识别结果；播报模式暂无返回。

数字人播报的文本返回见4.3文本驱动数字人返回；

4.6. 音频识别并驱动【结束】

请求：

返回：暂无

4.7. 通道准备

请求：

返回：

4.8. 音频驱动

请求：

返回：暂无

4.9. 通用业务类型

消息格式：

错误码：

WebSocket对接示例

1. 开场白

客户端发送：

    public void sendOpenning() {
        byte frameId = (byte) 0; //帧类型：控制帧
        byte dataType = (byte) 1; //数据类型：json
        byte bizType = (byte) 0; //开场白业务类型：0
        byte[] serialNumber = new byte[8]; // 正常业务使用随机生成8位序列号即可

        String content = "{\"sessionId\":\"xxx\"}"; //需替换sessionId
        byte[] contentBytes = content.getBytes(); 

        byte[] message = new byte[11 + contentBytes.length];
        ByteBuffer bf = ByteBuffer.wrap(message)
        .put(frameId)
        .put(dataType)
        .put(bizType)
        .put(serialNumber)
        .put(contentBytes);
        message = bf.array();

        // 3. 发送数据包
        client.sendMessage(message);
    }

客户端接收：

    @OnMessage
    public void onMessage(byte[] message) {

        //1. 获取第1字节帧号
        int frame = Byte.toUnsignedInt(message[0]);

        //2. 获取第2字节数据类型
        int dataType = Byte.toUnsignedInt(message[1]);

        //3. 获取第3字节业务类型
        int bizType = Byte.toUnsignedInt(message[2]);

        //4. 获取第4-11字节序列号
        byte[] serialNumberBytes = Arrays.copyOfRange(message, 3, 11);
        String serialNumberString = new String(serialNumberBytes, StandardCharsets.UTF_8);

        //5. 获取剩余内容数据，数据结构需要结合具体业务类型文档
        // 如果是开场白业务
        if (bizType == 0) {
            byte[] contentJsonBytes = Arrays.copyOfRange(message, 11, message.length);
            String contentJsonString = new String(contentJsonBytes, StandardCharsets.UTF_8);
            //6. 执行相应业务逻辑......
        }
        
    }

2. 打断

客户端发送：

    public void sendOpenning() {
        byte frameId = (byte) 0; //帧类型：控制帧
        byte dataType = (byte) 1; //数据类型：json
        byte bizType = (byte) 2; //打断业务类型：2
        byte[] serialNumber = new byte[8]; // 正常业务使用随机生成8位序列号即可

        String content = "{\"sessionId\":\"xxx\"}"; //需替换sessionId
        byte[] contentBytes = content.getBytes(); 

        byte[] message = new byte[11 + contentBytes.length];
        ByteBuffer bf = ByteBuffer.wrap(message)
        .put(frameId)
        .put(dataType)
        .put(bizType)
        .put(serialNumber)
        .put(contentBytes);
        message = bf.array();

        // 3. 发送数据包
        client.sendMessage(message);
    }

客户端接收：无需

3. 文本驱动

客户端发送：

    public void sendText() {
        byte frameId = (byte) 0; //帧类型：控制帧
        byte dataType = (byte) 1; //数据类型：json
        byte bizType = (byte) 128; //文本驱动业务类型：128
        byte[] serialNumber = new byte[8]; // 正常业务使用随机生成8位序列号即可

        String content = "{\n" +
                "  \"sessionId\":\"xxx\",\n" + //需替换sessionId
                "  \"text\":\"xxx\",\n" + //需替换text文本
                "  \"askType\":1 \n" + // 1：提问；2：播报
                "}"; 
        byte[] contentBytes = content.getBytes(); 

        byte[] message = new byte[11 + contentBytes.length];
        ByteBuffer bf = ByteBuffer.wrap(message)
        .put(frameId)
        .put(dataType)
        .put(bizType)
        .put(serialNumber)
        .put(contentBytes);
        message = bf.array();

        // 3. 发送数据包
        client.sendMessage(message);
    }

客户端接收：

    @OnMessage
    public void onMessage(byte[] message) {

        //1. 获取第1字节帧号
        int frame = Byte.toUnsignedInt(message[0]);

        //2. 获取第2字节数据类型
        int dataType = Byte.toUnsignedInt(message[1]);

        //3. 获取第3字节业务类型
        int bizType = Byte.toUnsignedInt(message[2]);

        //4. 获取第4-11字节序列号
        byte[] serialNumberBytes = Arrays.copyOfRange(message, 3, 11);
        String serialNumberString = new String(serialNumberBytes, StandardCharsets.UTF_8);

        //5. 获取剩余内容数据，数据结构需要结合具体业务类型文档
        // 文本驱动业务
        if (bizType == 128) {
            byte[] contentJsonBytes = Arrays.copyOfRange(message, 11, message.length);
            String contentJsonString = new String(contentJsonBytes, StandardCharsets.UTF_8);
            //6. 执行相应业务逻辑......
        }
        
    }

4. 音频识别并驱动

客户端发送：

    public void sendAudio() {

        // 1. 发送音频驱动头数据包
        byte frameId = (byte) 0; //帧类型：控制帧
        byte dataType = (byte) 1; //数据类型：json
        byte bizType = (byte) 130; //音频驱动头尾业务类型：130
        byte[] serialNumber = new byte[8]; // 正常业务使用随机生成8位序列号即可
        String content = "{\n" +
                "\t\"sessionId\":\"xxx\",\n" +
                "\t\"type\": \"startAsrAudio\",\n" +
                "\t\"askType\":1 " +
                "}";
        byte[] contentBytes = content.getBytes(); 
        byte[] message = new byte[11 + contentBytes.length];
        ByteBuffer bf = ByteBuffer.wrap(message)
        .put(frameId)
        .put(dataType)
        .put(bizType)
        .put(serialNumber)
        .put(contentBytes);
        message = bf.array();
        client.sendMessage(message);

        // 2. 多次发送音频二进制数据
        byte frameId = (byte) 0; //帧类型：控制帧
        byte dataType = (byte) 0; //数据类型：binary
        byte bizType = (byte) 129; //音频驱动数据业务类型：129
        byte[] serialNumber = new byte[8]; // 正常业务使用随机生成8位序列号即可
        byte[] audioData = new Byte[1024]; //需替换音频数据
        byte[] contentBytes = audioData;
        byte[] message = new byte[11 + contentBytes.length];
        ByteBuffer bf = ByteBuffer.wrap(message)
        .put(frameId)
        .put(dataType)
        .put(bizType)
        .put(serialNumber)
        .put(contentBytes);
        message = bf.array();
        client.sendMessage(message);

        // 3. 发送音频驱动尾包
        byte frameId = (byte) 0; //帧类型：控制帧
        byte dataType = (byte) 1; //数据类型：json
        byte bizType = (byte) 130; //音频驱动头尾业务类型：130
        byte[] serialNumber = new byte[8]; // 正常业务使用随机生成8位序列号即可
        String content = "{\n" +
                "\t\"sessionId\":\"xxx\",\n" +
                "\t\"type\":\"endAsrAudio\",\n" +
                "\t\"askType\": 1 " +
                "}";
        byte[] contentBytes = content.getBytes(); 
        byte[] message = new byte[11 + contentBytes.length];
        ByteBuffer bf = ByteBuffer.wrap(message)
        .put(frameId)
        .put(dataType)
        .put(bizType)
        .put(serialNumber)
        .put(contentBytes);
        message = bf.array();
        client.sendMessage(message);
    }

客户端接收：

    @OnMessage
    public void onMessage(byte[] message) {

        //1. 获取第1字节帧号
        int frame = Byte.toUnsignedInt(message[0]);

        //2. 获取第2字节数据类型
        int dataType = Byte.toUnsignedInt(message[1]);

        //3. 获取第3字节业务类型
        int bizType = Byte.toUnsignedInt(message[2]);

        //4. 获取第4-11字节序列号
        byte[] serialNumberBytes = Arrays.copyOfRange(message, 3, 11);
        String serialNumberString = new String(serialNumberBytes, StandardCharsets.UTF_8);

        //5. 获取剩余内容数据，数据结构需要结合具体业务类型文档
        // 获取音频驱动返回
        if (bizType == 129) {
            byte[] contentJsonBytes = Arrays.copyOfRange(message, 11, message.length);
            String contentJsonString = new String(contentJsonBytes, StandardCharsets.UTF_8);
            //6. 执行相应业务逻辑......
        }
        
    }

5. 通道准备

客户端发送：

    public void channelReady() {
        byte frameId = (byte) 0; //帧类型：控制帧
        byte dataType = (byte) 1; //数据类型：json
        byte bizType = (byte) 131; //通道准备业务类型：131
        byte[] serialNumber = new byte[8]; // 正常业务使用随机生成8位序列号即可

        String content = "{\"sessionId\":\"xxx\"}"; //需替换sessionId
        byte[] contentBytes = content.getBytes(); // 正常业务使用随机生成8位序列号即可

        byte[] message = new byte[11 + contentBytes.length];
        ByteBuffer bf = ByteBuffer.wrap(message)
        .put(frameId)
        .put(dataType)
        .put(bizType)
        .put(serialNumber)
        .put(contentBytes);
        message = bf.array();

        // 3. 发送数据包
        client.sendMessage(message);
    }

客户端接收：

    @OnMessage
    public void onMessage(byte[] message) {

        //1. 获取第1字节帧号
        int frame = Byte.toUnsignedInt(message[0]);

        //2. 获取第2字节数据类型
        int dataType = Byte.toUnsignedInt(message[1]);

        //3. 获取第3字节业务类型
        int bizType = Byte.toUnsignedInt(message[2]);

        //4. 获取第4-11字节序列号
        byte[] serialNumberBytes = Arrays.copyOfRange(message, 3, 11);
        String serialNumberString = new String(serialNumberBytes, StandardCharsets.UTF_8);

        //5. 获取剩余内容数据，数据结构需要结合具体业务类型文档
        // 通道准备
        if (bizType == 131) {
            byte[] contentJsonBytes = Arrays.copyOfRange(message, 11, message.length);
            String contentJsonString = new String(contentJsonBytes, StandardCharsets.UTF_8);
            //6. 执行相应业务逻辑......
        }
        
    }

6. 心跳

客户端发送：

    public void sendPing() {
        byte frameId = (byte) 0; //帧类型：控制帧
        byte dataType = (byte) 1; //数据类型：json
        byte bizType = (byte) 255; //通用心跳业务类型：255
        byte[] serialNumber = new byte[8]; // 正常业务使用随机生成8位序列号即可

        String content = "{}"; //无需
        byte[] contentBytes = content.getBytes(); 

        byte[] message = new byte[11 + contentBytes.length];
        ByteBuffer bf = ByteBuffer.wrap(message)
        .put(frameId)
        .put(dataType)
        .put(bizType)
        .put(serialNumber)
        .put(contentBytes);
        message = bf.array();

        // 3. 发送数据包
        client.sendMessage(message);
    }

客户端接收：无需

对接详情

PHP

对接示例

require 'vendor/autoload.php';
use AlibabaCloud\SDK\Imarketing\V20220704\Models\GetOssUploadSignatureRequest;
use AlibabaCloud\SDK\IntelligentCreation\V20240313\IntelligentCreation;
use Darabonba\OpenApi\Models\Config as AlibabaConfig;

$config = new AlibabaConfig();
$config->accessKeyId = '****';
$config->accessKeySecret = '****';
$config->endpoint = "intelligentcreation.cn-zhangjiakou.aliyuncs.com";

$intelligentCreationClient = new IntelligentCreation($config);

$request = new QueryAvatarProjectRequest();
$request->projectId = '111';
try {
  $response = $intelligentCreationClient->queryAvatarProject($request);
  var_dump($response->toMap());
} catch (TeaError $e) {
  Log::error($e);
}

2.7.0

composer require alibabacloud/intelligentcreation-20240313 2.7.0

Java

对接示例

package com.aliyun.intelligentcreation20240313;

import com.aliyun.intelligentcreation20240313.models.*;
import com.aliyun.tea.TeaException;
import com.aliyun.teaopenapi.models.Config;
import com.google.gson.Gson;

import java.util.HashMap;
import java.util.Map;

public class TestAvatarTest {

    public TestAvatarTest() throws Exception {
    }

    public static void main(String[] args) throws Exception {
        TestAvatarTest avatarTest = new TestAvatarTest();

        try {
            String projectId = "780931376329506816";
            avatarTest.queryAvatarProjectTest(projectId);
            String sessionId = avatarTest.startAvatarSessionRequest(projectId);
            avatarTest.checkSessionRequest(projectId,sessionId);
            avatarTest.sendTextMsgRequest(projectId,sessionId);
            avatarTest.stopAvatarSessionRequest(projectId,sessionId);
            avatarTest.queryAvatarResourceRequest(projectId);
        } catch (TeaException e) {
            Gson gson = new Gson();
            System.out.println(e.getMessage());
            System.out.println(gson.toJson(e.getData()));
        }
        
    }

    String url = "intelligentcreation.cn-zhangjiakou.aliyuncs.com";
    //初始化配置
    String ak = "**";
    String sk = "**";
    Config config = new Config().setAccessKeyId(ak)
            .setAccessKeySecret(sk)
            .setEndpoint(url);
    // 创建客户端
    Client client = new Client(config);

    void queryAvatarProjectTest(String projectId) throws Exception {
        Gson gson = new Gson();
        Map<String, Object> map = new HashMap<>();
        map.put("projectId",projectId);
        QueryAvatarProjectRequest request = QueryAvatarProjectRequest.build(map);
        // 请求接口
        QueryAvatarProjectResponse response = client.queryAvatarProject(request);

        System.out.println(gson.toJson(response));

        if (response.getStatusCode().equals(200)) {
            System.out.println("queryAvatarProjectTest 请求成功");
        }
    }

    String startAvatarSessionRequest(String projectId) throws Exception {
        Gson gson = new Gson();
        Map<String, Object> map = new HashMap<>();
        map.put("projectId",projectId);
        StartAvatarSessionRequest startAvatarSessionRequest = StartAvatarSessionRequest.build(map);
        StartAvatarSessionResponse response = client.startAvatarSession(startAvatarSessionRequest);

        System.out.println(gson.toJson(response));
        if (response.getStatusCode().equals(200)) {
            System.out.println("startAvatarSessionRequest 请求成功");
            return response.getBody().getSessionId();
        }
        return null;
    }

    void stopAvatarSessionRequest(String projectId, String sessionId) throws Exception {
        Gson gson = new Gson();
        Map<String, Object> map = new HashMap<>();
        map.put("projectId",projectId);
        map.put("sessionId",sessionId);
        StopAvatarSessionRequest request = StopAvatarSessionRequest.build(map);
        StopAvatarSessionResponse response = client.stopAvatarSession(request);

        System.out.println(gson.toJson(response));
        if (response.getStatusCode().equals(200)) {
            System.out.println("stopAvatarSessionRequest 请求成功");
        }
    }

    void queryAvatarResourceRequest(String projectId) throws Exception {
        Gson gson = new Gson();
        Map<String, Object> map = new HashMap<>();
        QueryAvatarResourceRequest request = QueryAvatarResourceRequest.build(map);
        QueryAvatarResourceResponse response = client.queryAvatarResource(request);

        System.out.println(gson.toJson(response));
        if (response.getStatusCode().equals(200)) {
            System.out.println("queryAvatarResourceRequest 请求成功");
        }
    }

}

2.7.0

<dependency>
  <groupId>com.aliyun</groupId>
  <artifactId>intelligentcreation20240313</artifactId>
  <version>2.7.0</version>
</dependency>

安卓

实时音视频SDK接入文档: Android平台

※ mLocalSurfaceContainer 是 视频流的展示容器

※ projectId 是项目Id

1. 添加maven仓库

maven(url = "https://maven.aliyun.com/nexus/content/repositories/releases")

2. 添加推流SDK依赖

implementation("com.aliyun.aio:AliVCSDK_ARTC:6.8.7")
implementation("com.aliyun:intelligentcreation20240313:2.1.0")
implementation("com.aliyun:tea-openapi:0.3.4")

3. 初始化拉流引擎

private fun createEngine() {
    mAliRtcEngine = AliRtcEngine.getInstance(applicationContext)
    mAliRtcEngine?.setRtcEngineEventListener(object : AliRtcEngineEventListener() {
        /* SDK与服务器的链接状态通知，务必处理链接失败的情况 */
        override fun onConnectionStatusChange(
            aliRtcConnectionStatus: AliRtcEngine.AliRtcConnectionStatus,
            aliRtcConnectionStatusChangeReason: AliRtcEngine.AliRtcConnectionStatusChangeReason
        ) {
            super.onConnectionStatusChange(
                aliRtcConnectionStatus,
                aliRtcConnectionStatusChangeReason
            )
            if (aliRtcConnectionStatus == AliRtcEngine.AliRtcConnectionStatus.AliRtcConnectionStatusFailed) {
                /* TODO: 务必处理；建议业务提示客户，此时SDK内部已经尝试了各种恢复策略已经无法继续使用时才会上报 */
            } else {
                /* TODO: 可选处理；增加业务代码，一般用于数据统计、UI变化 */
            }
        }

        /* SDK尝试控制本地设备异常 */
        override fun OnLocalDeviceException(
            aliRtcEngineLocalDeviceType: AliRtcEngine.AliRtcEngineLocalDeviceType,
            aliRtcEngineLocalDeviceExceptionType: AliRtcEngine.AliRtcEngineLocalDeviceExceptionType,
            s: String
        ) {
            //TODO 发生该异常时App需要检测权限、设备硬件是否正常。
        }

        override fun onJoinChannelResult(
            result: Int,
            channel: String,
            userId: String,
            elapsed: Int
        ) {
            super.onJoinChannelResult(result, channel, userId, elapsed)
            Log.i(TAG, "onJoinChannelResult result=$result,channel=$channel,userId=$userId,elapsed=$elapsed")
        }

        override fun onLeaveChannelResult(result: Int, stats: AliRtcEngine.AliRtcStats) {
            super.onLeaveChannelResult(result, stats)
            Log.i(TAG, "onLeaveChannelResult result=$result")
        }
    })

    mAliRtcEngine?.setRtcEngineNotify(object : AliRtcEngineNotify() {
        /* 鉴权距离过期还有30s时会回调，务必进行鉴权时间刷新 */
        override fun onAuthInfoWillExpire() {
            super.onAuthInfoWillExpire()
            /* TODO: 务必处理；业务触发重新获取当前channel，user的鉴权信息，然后设置refreshAuthInfo即可 */

        }

        /* 业务可能会触发踢人的动作，所以这个地方也需要处理 */
        override fun onBye(code: Int) {
            super.onBye(code)
            /* TODO: 建议业务根据自己的场景，进行对应的处理 */
        }

        override fun onRemoteUserOnLineNotify(uid: String, elapsed: Int) {
            super.onRemoteUserOnLineNotify(uid, elapsed)
            Log.i(TAG, "onRemoteUserOnLineNotify uid=$uid,elapsed=$elapsed")
        }

        override fun onRemoteUserOffLineNotify(
            uid: String,
            aliRtcUserOfflineReason: AliRtcEngine.AliRtcUserOfflineReason
        ) {
            super.onRemoteUserOffLineNotify(uid, aliRtcUserOfflineReason)
            Log.i(TAG, "onRemoteUserOnLineNotify uid=$uid,aliRtcUserOfflineReason=$aliRtcUserOfflineReason")
        }

        override fun onRemoteTrackAvailableNotify(
            uid: String,
            aliRtcAudioTrack: AliRtcEngine.AliRtcAudioTrack,
            aliRtcVideoTrack: AliRtcEngine.AliRtcVideoTrack
        ) {
            super.onRemoteTrackAvailableNotify(uid, aliRtcAudioTrack, aliRtcVideoTrack)
            Log.i(TAG, "onRemoteUserOnLineNotify uid=$uid,aliRtcAudioTrack=$aliRtcAudioTrack,aliRtcVideoTrack=$aliRtcVideoTrack")
            mLocalSurfaceContainer.post{
                if (aliRtcVideoTrack == AliRtcEngine.AliRtcVideoTrack.AliRtcVideoTrackCamera
                    || aliRtcVideoTrack == AliRtcEngine.AliRtcVideoTrack.AliRtcVideoTrackBoth
                ) {
                    val remote_canvas = AliRtcVideoCanvas()
                    val remoteView = mAliRtcEngine?.createRenderSurfaceView(this@PreviewActivity)
                    if (remoteView != null) {
                        remoteView.setZOrderOnTop(true)
                        remoteView.setZOrderMediaOverlay(true)
                    }
                    remote_canvas.view = remoteView
                    mLocalSurfaceContainer.addView(
                        remote_canvas.view, FrameLayout.LayoutParams(
                            mLocalSurfaceContainer.width,
                            mLocalSurfaceContainer.height
                        )
                    )
                    mAliRtcEngine?.setRemoteViewConfig(
                        remote_canvas,
                        uid,
                        AliRtcEngine.AliRtcVideoTrack.AliRtcVideoTrackCamera
                    )
                } else {
                    mAliRtcEngine?.setRemoteViewConfig(
                        null,
                        uid,
                        AliRtcEngine.AliRtcVideoTrack.AliRtcVideoTrackCamera
                    )
                }
            }
        }
    })
}

4. 入会前引擎参数配置

private fun initEngineBeforeJoin() {
    /* 可选：入会前的参数设置 */
    mAliRtcEngine?.setChannelProfile(AliRtcEngine.AliRTCSdkChannelProfile.AliRTCSdkInteractiveLive)
    mAliRtcEngine?.setClientRole(AliRtcEngine.AliRTCSdkClientRole.AliRTCSdkLive)
    /* 设置音频的属性 */
    mAliRtcEngine?.setAudioProfile(
        AliRtcEngine.AliRtcAudioProfile.AliRtcEngineStereoHighQualityMode,
        AliRtcEngine.AliRtcAudioScenario.AliRtcSceneMusicMode
    )

    /* 可选：摄像头预览,不设置也会进行推流 */
    val canvas = AliRtcVideoCanvas()
    canvas.view = mAliRtcEngine?.createRenderSurfaceView(this)
    mLocalSurfaceContainer.removeAllViews()
    if (canvas.view != null) {
        mLocalSurfaceContainer.addView(
            canvas.view,
            FrameLayout.LayoutParams(
                FrameLayout.LayoutParams.MATCH_PARENT,
                FrameLayout.LayoutParams.MATCH_PARENT
            )
        )
    }
    mAliRtcEngine?.setLocalViewConfig(
        canvas,
        AliRtcEngine.AliRtcVideoTrack.AliRtcVideoTrackCamera
    )
}

5. 初始化pop接口SDK

private fun initPopApiClient(){
    val config = Config()
        .setAccessKeyId(KEY_ID)
        .setAccessKeySecret(KEY_SECRET)
        .setEndpoint(HOST)
    client = Client(config)
}

6. 获取项目信息

private suspend fun getProjectInfo() = withContext(Dispatchers.IO) {
    val map: Map<String , String> = mutableMapOf<String, String>().apply {
        put(PROJECT_ID , projectId?:"")
    }
    client?.queryAvatarProject(QueryAvatarProjectRequest.build(map))?.apply {
        Log.e(TAG , "statusCode:${statusCode}")
        if (statusCode == 200) {
            Log.e(TAG , "projectName:${body.projectName}")
            tvProjectName.post {
                tvProjectName.text = body.projectName
            }
        }
    }
}

7. 获取会议信息

private suspend fun getMeetingInfo() = withContext(Dispatchers.IO){
    val map: Map<String , String> = mutableMapOf<String, String>().apply {
        put(PROJECT_ID , projectId?:"")
    }
    val response: StartAvatarSessionResponse? = client?.startAvatarSession(StartAvatarSessionRequest.build(map))

    response?.apply {
        Log.e(TAG , "statusCode:${statusCode}")
        if (statusCode == 200) {
            Log.e(TAG , "sessionId:${body.sessionId},channelToken:${body.channelToken}")
            sessionId = body.sessionId
            val meetingJsonObj = JSONObject(body.channelToken)
            var channelId = ""
            var token = ""
            var nonce = ""
            var userId = ""
            var appId = ""
            var expireTime = 0L
            var gslbList:MutableList<String> = mutableListOf()
            if (meetingJsonObj.has(CHANNEL_ID)){
                channelId = meetingJsonObj.optString(CHANNEL_ID)
            }
            if (meetingJsonObj.has(TOKEN)){
                token = meetingJsonObj.optString(TOKEN)
            }
            if (meetingJsonObj.has(NONCE)){
                nonce = meetingJsonObj.optString(NONCE)
            }
            if (meetingJsonObj.has(USER_ID)){
                userId = meetingJsonObj.optString(USER_ID)
            }
            if (meetingJsonObj.has(APP_ID)){
                appId = meetingJsonObj.optString(APP_ID)
            }
            if (meetingJsonObj.has(EXPIRE_TIME)){
                expireTime = meetingJsonObj.optLong(EXPIRE_TIME)
            }
            if (meetingJsonObj.has(GSLB_LIST)) {
                val gslbListJson = meetingJsonObj.optJSONArray(GSLB_LIST)
                for (i in 0 until gslbListJson.length()) {
                    gslbList.add(gslbListJson.optString(i))
                }
            }
            join(appId , channelId , userId , nonce,expireTime , token , gslbList.toTypedArray())
        }
    }
}

8. 入会

private fun join(appId:String,channelId:String , userId:String ,
                     nonce:String , expireTime:Long,token:String,gslbList:Array<String>){
    val userInfo = AliRtcAuthInfo()
    userInfo.setAppId(appId)
    userInfo.setChannelId(channelId)
    userInfo.setUserId(userId)
    userInfo.setNonce(nonce)
    userInfo.setTimestamp(expireTime)
    userInfo.setGslb(gslbList)
    userInfo.setToken(token)
    mAliRtcEngine?.joinChannel(userInfo, "testUserName")

    resetCountDown()
}

9. 发送问题

private fun send(){
    val question = etInputQuestion.text.toString().trim()
    if (TextUtils.isEmpty(question)) {
        Toast.makeText(this , "question content cannot be null" , Toast.LENGTH_SHORT).show()
        return
    }
    GlobalScope.launch {
        sendText(question)
    }
}

private suspend fun sendText(text:String) = withContext(Dispatchers.IO){

    val map: Map<String , String> = mutableMapOf<String, String>().apply {
        put(PROJECT_ID , projectId?:"")
        put(SESSION_ID , sessionId)
        put(REQUEST_ID , (System.currentTimeMillis()/1000).toString())
        put(TEXT , text)
        put(TYPE , "1")
    }
    client?.sendTextMsg(SendTextMsgRequest.build(map))?.apply {
        onSendResponse(statusCode == 200 && body.status == "SUCCESS" , text)
    }
}

private fun onSendResponse(result:Boolean , text:String){
    tvExpireHint.post {
        if (result) {
            etInputQuestion.setText("")
            questionAdapter?.data?.add(0 , text)
            questionAdapter?.notifyItemInserted(0)
            rvQuestion.scrollToPosition(0)
            resetCountDown()
            Toast.makeText(this@PreviewActivity , "发送成功" , Toast.LENGTH_SHORT).show()
        }else {
            Toast.makeText(this@PreviewActivity , "发送失败" , Toast.LENGTH_SHORT).show()
        }
    }
}

10. 离会

private fun leave(){
    mAliRtcEngine?.leaveChannel()
    findViewById<LinearLayout>(R.id.llEnd).visibility = View.VISIBLE

    val map: Map<String , String> = mutableMapOf<String, String>().apply {
        put(PROJECT_ID , projectId?:"")
        put(SESSION_ID , sessionId)
    }

    GlobalScope.launch {
        async {
            client?.stopAvatarSession(StopAvatarSessionRequest.build(map))?.apply {
                Log.e(TAG , "statusCode = ${statusCode}, status = ${body.status}")
            }
        }
    }
}

11. 资源回收

override fun onDestroy() {
    super.onDestroy()
    mAliRtcEngine?.destroy()
    mAliRtcEngine = null
    handler.removeCallbacksAndMessages(null)
}

12. 10min无提问，自动离会

private var handler: Handler = Handler(Looper.getMainLooper())

private val mRunnable = Runnable {
    dealCountDown()
}

 private fun dealCountDown(){
    if (messageCount == 0) {
        //离会
        leave()
        return
    }
    //小于1min ，提示
    val timeStr = when {
        60 == messageCount-> "01:00"
        (messageCount in 1..9) -> "00:0${messageCount}"
        else -> "00:${messageCount}"
    }
    llExpireHint.visibility = View.VISIBLE
    tvExpireHint.text = "检测到您无操作，预览将于${timeStr}自动关闭，以避免占用路数"
    messageCount--
    handler.postDelayed(mRunnable , 1000)
}

//在入会和发送问题之后调用
private fun resetCountDown(){
    handler.removeCallbacksAndMessages(null)
    messageCount = 60
    handler.postDelayed(mRunnable , 1000 * 60 * 9)
}