AI real-time chat agent

Introduction

DingRTC provides an AI real-time chat agent service. You can use a server-side API to start an agent and add it to a specified Real-Time Communication (RTC) channel. To other users in the channel, the agent appears as any other user. The Web software development kit (SDK) provides interfaces to listen for messages and status updates from agents that join the RTC channel. You can also use the SDK to send messages or notifications to the agent.

Concepts

Agent ID: The user ID of the agent.

Transcription message: The transcribed text of the conversation with the agent.

Natural conversation/Push-to-talk mode: The conversation mode for an agent, which is set when the agent is created using the OpenAPI.

Integration method

1. Follow the Quick Start guide to integrate the RTC Web SDK and implement basic features.

2. Create an instance of `AiAgentClient` and register it with the `rtcClient` object:

   import DingRTC from 'dingrtc';
   import AiAgentClient from 'dingrtc-aiagent';
   
   const rtcClient = DingRTC.createClient();
   // Replace with the actual agent user ID.
   const aiAgentClient = new AiAgentClient('xxx'); 
   rtcClient.register(aiAgentClient);

3. Add the `rtcClient` instance to a channel:

   await rtcClient.join({
     appId: '',
     userName: '',
     channel: '',
     uid: '',
     token: '',
   });

4. Use the `AiAgentClient` instance to listen for relevant messages:

   // Listen for transcription messages from the agent.
   aiAgentClient.on('message', (message: AIAgentMessage) => {
     console.log("Message received:", message);
     /**
      * Example of the message structure:
      * - message: The text content.
      * - uid: The sender ID.
      * - reasoning: Indicates whether this is part of the thinking process.
      * - userType: 'user' or 'agent'
      * - extra: Custom content for inserted messages. This is not currently in use.
      * - end: Indicates whether this is the end marker.
      */
   });
   
   // Listen for agent status changes.
   aiAgentClient.on('agent-status', (newStatus: AgentStatus) => {
     console.log("Agent status changed:", newStatus);
     // The status can be: 'listening' | 'processing' | 'responding'
   });

5. From your application server, call the StartAgent - Start the agent operation of the agent service.

6. If you start the agent in natural conversation mode, you can interact with it normally.

7. If you start the agent in push-to-talk mode, use the following APIs to interact with it:

   // Notify the agent that the local client starts to push-to-talk.
   aiAgentClient.startPushToTalk();
   
   // Notify the agent that the local client stops the push-to-talk.
   aiAgentClient.stopPushToTalk();
   
   // Notify the agent that the local client cancels the push-to-talk.
   aiAgentClient.cancelPushToTalk();

8. In either mode, you can use the following `AiAgentClient` API to interrupt the agent:

   // Interrupt the agent.
   aiAgentClient.interrupt(): void;

Notes

1. To create an agent and add it to a channel, call the server-side OpenAPI StartAgent - Start the agent operation.

2. To avoid missing callback messages, set a message listener for the `AIAgentClient` instance before you join the channel.