Transferring a call to a human agent

Web client takeover workflow

Joining the channel (agent client)

1. The agent client calls the Enable human takeover mode API.

2. The agent client obtains information for joining the channel, including an RTC token.

3. The agent client uses the channel information to join the RTC channel, publish and subscribe to streams, and receive the transcript from the AI agent. For more information about RTC integration, see Get started with ARTC SDK for Web.

The following sample code shows how to receive the transcript from the AI agent:

1. Listen for the transcript event.

   aliRtcEngine.on('dataChannelMsg', (uid, msg) => {
     try {
       const dataChannelMsg = JSON.parse(new TextDecoder().decode(msg.data));
       if (dataChannelMsg.type === 1002) {
         // Received transcript from the AI agent. Data format:
         // text: The transcribed text.
         // sentenceId: The ID of the sentence. Used to group multiple results for the same sentence in a stream.
         // end: Indicates if this is the end of the current sentence.
         console.log('agentSubtitleNotify', dataChannelMsg.data);
       } else if (dataChannelMsg.type === 1003) {
         // Received transcript from the user.
         // text: The transcribed text.
         // sentenceId: The ID of the sentence. Used to group multiple results for the same sentence in a stream.
         // end: Indicates if this is the end of the current sentence.
         console.log('userSubtitleNotify', dataChannelMsg.data);
       } else {
         // Other types can be ignored.
       }
     } catch(error) {
       console.error(error)
     }
   });

2. Before joining the channel, call the following method to receive transcripts.

   aliRtcEngine.setParameter(
     JSON.stringify({
       data: {
         enableSubDataChannel: true,
       },
     }),
   );

On-demand subscription for human agents

By default, a human agent subscribes to all users in the channel, including the end user and the AI agent. This means the human agent receives the user's audio and video, as well as the speech-to-text transcript from the AI agent.

If you do not need the AI agent's audio or transcript, use on-demand subscription to subscribe only to the user's audio and video streams.

1. Before calling joinChannel, call methods such as setDefaultSubscribeAllRemoteAudioStreams to disable automatic subscription.

   // Disable automatic subscription to audio streams.
   aliRtcEngine.setDefaultSubscribeAllRemoteAudioStreams(false);
   // Disable automatic subscription to video and screen sharing streams.
   aliRtcEngine.setDefaultSubscribeAllRemoteVideoStreams(false);

2. After disabling automatic subscription, use the subscribeRemoteMediaStream method to subscribe to a specific user.

   // Import the AliRtcVideoTrack enum by using npm.
   import { AliRtcVideoTrack } from 'aliyun-rtc-sdk';
   // Alternatively, import it from the window object.
   // const AliRtcVideoTrack = window.AliRtcEngine.AliRtcVideoTrack;
   // The ID of the user to subscribe to.
   const subscribeUserId = 'xxxx';
   // This example subscribes to a user by listening for the remoteUserOnLineNotify event.
   // In a real-world scenario, you might decide whether to subscribe based on your business logic.
   aliRtcEngine.on('remoteUserOnLineNotify', (userId) => {
     if (userId === subscribeUserId) {
       // Subscribe to all streams (video, audio, screen) of this user. No error occurs if the user is not publishing certain streams.
       // If the user starts publishing a new stream later, it will be subscribed to automatically.
       aliRtcEngine.subscribeRemoteMediaStream(
         userId,
         AliRtcVideoTrack.AliRtcVideoTrackBoth,
         true,
         true
       );
     }
   });
   // Handle cases where the user is already in the channel.
   if (aliRtcEngine.getOnlineRemoteUsers()?.includes(subscribeUserId)) {
     // Subscribe to all streams (video, audio, screen) of this user. No error occurs if the user is not publishing certain streams.
     // If the user starts publishing a new stream later, it will be subscribed to automatically.
     aliRtcEngine.subscribeRemoteMediaStream(
       userId,
       AliRtcVideoTrack.AliRtcVideoTrackBoth,
       true,
       true
     );
   }
   

Inform users about the takeover process

Use callback interfaces to monitor the human agent's status during the takeover and provide real-time feedback to the user.

Implement the onHumanTakeoverWillStart and onHumanTakeoverConnected callback interfaces to monitor the takeover process and notify users. The following is sample code:

// Callback handling (example of core callback operations)
ARTCAICallEngine.IARTCAICallEngineCallback mCallEngineCallbackWrapper = new ARTCAICallEngine.IARTCAICallEngineCallback() {
	@Override
    public void onHumanTakeoverWillStart(String takeoverUid, int takeoverMode) {
        // A human agent is about to take over the current AI agent.
    }
    @Override
    public void onHumanTakeoverConnected(String takeoverUid) {
        // The human agent has successfully connected.
    }
    // Other callbacks.
    ...
};

extension AUIAICallStandardController: ARTCAICallEngineDelegate {
	public func onHumanTakeoverWillStart(takeoverUid: String, takeoverMode: Int) {
        // A human agent is about to take over the current AI agent.
        debugPrint("AUIAICallStandardController onHumanTakeoverWillStart:\(takeoverUid) , takeoverMode:\(takeoverMode)")
    }
    public func onHumanTakeoverConnected(takeoverUid: String) {
        // The human agent has successfully connected.
        debugPrint("AUIAICallStandardController onHumanTakeoverConnected:\(takeoverUid)")
    }
    // Other callbacks.
    ...
}

engine.on('humanTakeoverWillStart', (uid: string, mode: number) => {
  // A human agent is about to take over the current AI agent.
  console.log('AICallHumanTakeoverWillStart', uid, mode);
});
engine.on('humanTakeoverConnected', (uid: string) => {
  // The human agent has successfully connected.
  console.log('AICallHumanTakeoverConnected', uid);
});