Real-time Conversational AI FAQ

Features

• Do I need to deploy an agent on my origin server?

• How do I connect an agent to a large language model deployed on Alibaba Cloud Model Studio?

Integration

• Errors occur when starting a call

• The client reports an "AgentNotFound" error when starting a messaging conversation

• The client reports an "UnsupportedWorkflowType" error when starting a messaging conversation

• How do I adjust the client audio capture sample rate?

• How do I adjust the agent audio playback sample rate?

Agent deployment on origin server

No. The agent is a public cloud service from Alibaba Cloud, so no cloud deployment is needed. You can build and use agents by configuring them in the console and calling the OpenAPI.

Connect agent to LLM on Model Studio

The agent is integrated with Alibaba Cloud Model Studio by design. You can call the large language models on Alibaba Cloud Model Studio by simply specifying the required parameters in the console. For more information, see Large Language Models (LLMs).

Errors occur when starting a call

"AgentNotFound" error when starting messaging

In your client code, verify that the agentId and region are configured correctly.

On the Intelligent Media Services console, go to Real-time Conversational AI > Agent Management in the left-side navigation pane. Confirm that the region selected at the top of the page matches the region configured in your code, and then get the correct agentId from the Agent ID/Name column in the agent list.

Client code:

String mAgentId = "XXX";          // Agent ID. Use the ID of the messaging conversation agent created on the console.
String mRegion = "cn-shanghai";   // The region where the agent is located. Use the region of the messaging conversation agent displayed on the console.
ARTCAIChatAgentInfo agentInfo = new ARTCAIChatEngine.ARTCAIChatAgentInfo(mAgentId, mRegion)

// Agent ID. Use the ID of the messaging conversation agent created on the console.
let agentInfo = ARTCAIChatAgentInfo(agentId: "xxxx")
// The region where the agent is located. Use the region of the messaging conversation agent displayed on the console.
agentInfo.region = "cn-shanghai"

"UnsupportedWorkflowType" error when starting messaging

If the error details are "The specified workflowType \"VoiceChat\" is not supported by this interface. Please use a compatible workflowType: [\"MessageChat\"]", check the agentId and verify that its associated workflow type is MessageChat.

On the Agent Management page, find the target agent and check the Agent Workflow ID column to confirm that the associated workflow type is MessageChat.

Adjust client audio capture sample rate

Real-time Conversational AI supports the following settings for client-side audio capture:

• 48 kHz sample rate, mono channel

• 16 kHz sample rate, mono channel

By default, the AICallKit SDK uses a 48 kHz sample rate. To switch to 16 kHz, see the following code examples:

self.engine.audioConfig = ARTCAICallAudioConfig(audioProfile: .BasicQualityMode, audioScenario: .MusicMode)
// Call other APIs
...  
// Initiate a call
self.engine.call(...)

ARTCAICallEngine.ARTCAICallConfig artcaiCallConfig = new ARTCAICallEngine.ARTCAICallConfig();
artcaiCallConfig.audioConfig.audioProfile = ARTCAICallAudioBasicQualityMode;
engine.init(artcaiCallConfig);

Adjust agent audio playback sample rate

Real-time Conversational AI supports the following audio settings for agent playback:

• 48 kHz sample rate, mono or stereo channel

• 16 kHz sample rate, mono channel

By default, Real-time Conversational AI uses a 48 kHz mono channel sample rate. You must adjust this setting in the API call that starts the conversation. The method varies by how you initiate the call:

Method 1: Using the server-side GenerateAIAgentCall API

You can add the AgentConfig parameter and construct the ExperimentalConfig field in AIAgentConfig.

// AudioQualityMode: Integer    
// The sample rate mode for RTC output.
// 0: 48 kHz mono channel 
// 1: 48 kHz stereo channel
// 2: 16 kHz mono channel
// The value of ExperimentalConfig must be a JSON string.
{
    "ExperimentalConfig":"{\"AudioQualityMode\":2}"
}

Method 2: Using the server-side StartAIAgentInstance API

You can add the AgentConfig parameter and construct the ExperimentalConfig field in AIAgentConfig.

// AudioQualityMode: Integer    
// The sample rate mode for RTC output.
// 0: 48 kHz mono channel 
// 1: 48 kHz stereo channel
// 2: 16 kHz mono channel
// The value of ExperimentalConfig must be a JSON string.
{
    "ExperimentalConfig":"{\"AudioQualityMode\":2}"
}

Method 3: Using the client-side call() API

If you create and start a call using the call(xxx) API with the callConfig parameter, you can use this method to modify the agent's playback sample rate.

let callConfig = ...   // Create and initialize ARTCAICallConfig. For details, see the user guide.
let agentConfig = ARTCAICallAgentConfig()         // Create an ARTCAICallAgentConfig object.
agentConfig.experimentalConfig = [
    "AudioQualityMode": 2
]
...                                                  // Set parameters based on your business requirements.
callConfig.agentConfig = agentConfig
self.engine.call(config: callConfig)                 // Initiate the call.

ARTCAICallEngine.ARTCAICallConfig artcaiCallConfig = new ARTCAICallEngine.ARTCAICallConfig();
artcaiCallConfig.agentConfig.experimentalConfig = new JSONObject();
try {
    artcaiCallConfig.agentConfig.experimentalConfig.put("AudioQualityMode", 2);
} catch (JSONException e) {
    e.printStackTrace();
}
engine.init(artcaiCallConfig);