Harmony data structures-Intelligent Media Services(IMS)-阿里云帮助中心

This topic describes the data structures used in the Harmony software development kit (SDK).

Data structure overview

Structure type	Data type	Description
*Enum*	ARTCAICallAgentType	The agent type.
	ARTCAICallAgentState	The agent state.
	ARTCAICallNetworkQuality	Network Status
	ARTCAICallSpeakingInterruptedReason	The reason why the agent's current speech was interrupted.
	ARTCAICallConnectionStatus	The network connectivity status during a call.
	ARTCAICallAudioProfile	The audio encoding configuration.
	ARTCAICallAudioScenario	The audio scenario configuration.
	ARTCAICallAgentViewMode	The agent view rendering mode.
	ARTCAICallAgentViewMirrorMode	The agent view mirror mode.
	ARTCAICallAgentViewRotationMode	The agent view rotation mode.
	ARTCAICallAudioAccompanyState	The music accompaniment playback state.
	ARTCAICallAudioAccompanyErrorCode	The music accompaniment playback error code.
	ARTCAICallVoiceprintResult	The Voice Activity Detection (VAD) feedback result.
	ARTCAICallTurnDetectionMode	The mode for determining whether the user has finished speaking.
Class	ARTCAICallAgentInfo	The agent runtime information.
	ARTCAICallConfig	The agent call configuration.
	ARTCAICallAudioConfig	The local audio configuration for calls.
	ARTCAICallVideoConfig	The local video configuration for calls.
	ARTCAICallViewConfig	The configuration for rendering views. Use this class to configure rendering when an agent or local camera needs to be rendered, such as for a digital human.
	ARTCAICallChatSyncConfig	The configuration parameters for an associated chat agent session.
	ARTCAICallVisionCustomCaptureRequest	The request model for enabling custom frame capture for a visual understanding agent.
	ARTCAICallSendTextToAgentRequest	Send a message to the agent to request the Model.
	ARTCAICallUserSubtitleResult	The user caption notification result.
	ARTCAICallAgentSubtitleResult	The agent caption notification result.
	ARTCAICallAgentVcrFrameMotionResult	The detection result for silent or invalid frames from the VCR detection results.
	ARTCAICallAgentVcrPeopleCountResult	The detection result for the real-time number of people in the frame from the VCR detection results.
	ARTCAICallAgentVcrEquipmentResult	The detection result for electronic devices from the VCR detection results.
	ARTCAICallAgentVcrHeadMotionResult	The detection result for head movements from the VCR detection results.
	ARTCAICallAgentVcrLookAwayResult	The detection result for gaze deviation from the VCR detection results.
	ARTCAICallAgentVcrResult	The VCR detection result.
	ARTCAICallAgentShareConfig	The configuration for sharing an agent.
	ARTCAICallAgentShareGenerateResult	The result of generating a shared agent.
	ARTCAICallAgentConfig	The startup and runtime configuration for a call agent.
	ARTCAICallAgentAsrConfig	The speech recognition configuration.
	ARTCAICallAgentTtsConfig	The speech synthesis configuration.
	ARTCAICallAgentLlmConfig	The large language model (LLM) configuration.
	ARTCAICallAgentAvatarConfig	The digital human configuration.
	ARTCAICallAgentInterruptConfig	The interruption configuration.
	ARTCAICallAgentVoiceprintConfig	The voiceprint denoising configuration.
	ARTCAICallAgentTurnDetectionConfig	The turn detection configuration.
	ARTCAICallAgentVcrConfig	The VCR configuration.
	ARTCAICallAgentVcrBaseConfig	The basic VCR detection configuration.
	ARTCAICallAgentVcrFrameMotionConfig	The VCR video frame detection configuration.
	ARTCAICallExperimentalConfig	The experimental configuration.
	ARTCAICallAgentAmbientConfig	The call environment parameter.
	ARTCAICallAgentAutoSpeechContent	The content that the agent speaks automatically in scenarios such as back-channeling and proactive inquiries.
	ARTCAICallAgentAutoSpeechLlmPending	The configuration for the agent to automatically speak when there is a delay in the LLM response.
	ARTCAICallAgentAutoSpeechUserIdle	The configuration for the agent to make an inquiry when the user is silent.
	ARTCAICallAgentBackChanneling	The configuration module for the back-channeling feature. When enabled, the agent randomly plays short back-channeling phrases at specific trigger points.

Data structure details

Enumerations

ARTCAICallAgentType

The agent type.

Enumeration	Value	Description
VoiceAgent	0	Voice only.
AvatarAgent	1	Digital human.
VisionAgent	2	Visual understanding.
VideoAgent	3	Video call.

ARTCAICallAgentState

The agent state.

Enumeration	Value	Description
Listening	1	Listening.
Thinking	2	Thinking.
Speaking	3	Speaking.

ARTCAICallNetworkQuality

Network status

Enumeration	Value	Description
Excellent	0	The network quality is excellent. The stream is smooth and the definition is high.
Good	1	The network quality is good. The stream smoothness and definition are similar to excellent quality.
Poor	2	The network quality is poor. The audio and video may have minor issues with smoothness and definition, but communication is not affected.
Bad	3	The network quality is bad. The video experiences severe stuttering, but audio communication is normal.
VeryBad	4	The network quality is very bad. Communication is nearly impossible.
Disconnect	5	The network is disconnected.
Unknow	6	Unknown.

ARTCAICallSpeakingInterruptedReason

The reason the agent's current speech was interrupted.

Enumeration	Value	Description
unknown	0	Unknown reason.
byWords	1	Interrupted because a specific word was detected.
byVoice	2	Interrupted by voice.
byInterruptSpeaking	3	Interrupted by a call to the interruptSpeaking API.
bySpeechBroadCast	4	Interrupted by a proactive voice broadcast.
byLlmQuery	5	Interrupted by a proactive LLM query.

ARTCAICallConnectionStatus

The network connectivity status during a call.

Enumeration	Value	Description
Init	0	Initialization is complete.
Disconnected	1	The network connection is disconnected.
Connecting	2	Establishing a network connection.
Connected	3	The network is connected.
Reconnecting	4	Re-establishing the network connection.
Failed	5	The network connection failed.

ARTCAICallAudioProfile

The audio encoding configuration.

Enumeration	Value	Description
LowQualityMode	0x0000	Low-quality audio mode. The default sample rate is 8000 Hz, single-channel, with a maximum encoding bitrate of 12 kbps.
BasicQualityMode	0x0001	Standard-quality audio mode. The default sample rate is 16000 Hz, single-channel, with a maximum encoding bitrate of 24 kbps.
HighQualityMode	0x0010	(Default) High-quality audio mode. The default sample rate is 48000 Hz, single-channel, with a maximum encoding bitrate of 64 kbps.
StereoHighQualityMode	0x0011	Stereo high-quality audio mode. The default sample rate is 48000 Hz, dual-channel, with a maximum encoding bitrate of 80 kbps.
SuperHighQualityMode	0x0012	Super high-quality audio mode. The default sample rate is 48000 Hz, single-channel, with a maximum encoding bitrate of 96 kbps.
StereoSuperHighQualityMode	0x0013	Stereo super high-quality audio mode. The default sample rate is 48000 Hz, dual-channel, with a maximum encoding bitrate of 128 kbps.

ARTCAICallAudioScenario

The audio scenario configuration.

Enumeration	Value	Description
DefaultMode	0x0000	This is the recommended choice for general Real-Time Communication scenarios.
MusicMode	0x0300	The default scenario. Recommended for music scenarios that require high-fidelity audio, such as musical instrument lessons.

ARTCAICallAgentViewMode

The agent view rendering mode.

Enumeration	Value	Description
Auto	0	Auto mode.
Stretch	1	Stretch mode.
Fill	2	Fill mode.
Crop	3	Crop mode.

ARTCAICallAgentViewMirrorMode

The agent view mirror mode.

Enumeration	Value	Description
OnlyFrontCameraPreviewEnabled	0	Mirroring is enabled only for the front camera preview. Other views are not mirrored.
AllEnabled	1	Mirroring is enabled for all views.
AllDisabled	2	Mirroring is disabled for all views.

ARTCAICallAgentViewRotationMode

The agent view rotation mode.

Enumeration	Value	Description
Rotation_0	0	Video view rotation angle: 0 degrees.
Rotation_90	1	Video view rotation angle: 90 degrees.
Rotation_180	2	Video view rotation angle: 180 degrees.
Rotation_270	3	Video view rotation angle: 270 degrees.

ARTCAICallAudioAccompanyState

The music accompaniment playback state.

Enumeration	Value	Description
ARTCAICallAudioAccompanyStarted	100	Playback has started.
ARTCAICallAudioAccompanyStopped	101	Playback has stopped.
ARTCAICallAudioAccompanyPaused	102	Playback is paused.
ARTCAICallAudioAccompanyResumed	103	Playback has resumed.
ARTCAICallAudioAccompanyEnded	104	Playback is complete.
ARTCAICallAudioAccompanyBuffering	105	Buffering.
ARTCAICallAudioAccompanyBufferingEnd	106	Buffering has ended.
ARTCAICallAudioAccompanyFailed	107	Playback has failed.

ARTCAICallAudioAccompanyErrorCode

The music accompaniment playback error code.

Enumeration	Value	Description
ARTCAICallAudioAccompanyNoError	0	No error.
ARTCAICallAudioAccompanyUnknowError	-1	Unknown error.
ARTCAICallAudioAccompanyOpenFailed	-100	Failed to open the file.
ARTCAICallAudioAccompanyDecodeFailed	-101	Failed to decode the file.

ARTCAICallVoiceprintResult

VAD feedback

Enumeration value	Value	Description
Off	0	Voiceprint denoising VAD is not enabled, and AI VAD is disabled.
Unregister	1	Voiceprint denoising VAD is enabled, but voiceprint registration is not complete.
DetectedSpeaker	2	Voiceprint denoising VAD is enabled, and the main speaker is detected.
UndetectedSpeaker	3	Voiceprint denoising VAD is enabled, but the main speaker is not detected.
DetectedSpeakerWithAIVad	4	AI VAD is enabled, and the main speaker is detected.
UndetectedSpeakerWithAIVad	5	AI VAD is enabled, but the main speaker is not detected.
Unknown	100	Unknown.

ARTCAICallTurnDetectionMode

The mode for determining whether the user has finished speaking.

Enumeration	Value	Description
Normal	0	Normal mode. AI is not used to determine whether the user has finished speaking based on semantics. The determination is based on the Automatic Speech Recognition (ASR) silence duration.
Semantic	1	Semantic mode. AI is used to determine whether the user has finished speaking based on contextual semantics.

Class

ARTCAICallAgentInfo

The agent runtime information.

Property name	Type	Description
agentId	string	The current agent ID.
agentType	ARTCAICallAgentType	The agent type.
channelId	string	The ID of the RTC channel that contains the agent.
uid	string	The unique identifier used by the agent to join the RTC channel.
instanceId	string	The instance ID where the current agent is running.
requestId	string	The request ID for starting the current agent.
region	string	The region where the agent is located.

ARTCAICallConfig

The agent call configuration.

Property	Type	Description
agentId	string	The agent ID.
agentType	ARTCAICallAgentType	The agent type. This must match the type of the specified agentId. Otherwise, an error occurs when the agent starts.
agentUserId	string	The user ID (UID) of the agent. If this is empty, the agent service assigns a UID.
region	string	The region where the agent service is located. This must be the same region as the specified agentId. Otherwise, an error occurs when the agent starts.
userId	string	The current user ID.
userJoinToken	string	The token for the current user to join the session.
userData	Record<string, RecordValueType>	Custom user information. This information is passed to the agent.
agentConfig	ARTCAICallAgentConfig	The agentConfig parameters used to start the call.
audioConfig	ARTCAICallAudioConfig	The local audio configuration.
videoConfig	ARTCAICallVideoConfig	The local video configuration. This takes effect only for VisionAgent or `VideoAgent` types.
chatSyncConfig	ARTCAICallChatSyncConfig	The configuration for the associated chat agent. If configured, the call history is synchronized to the chat agent during the call.

ARTCAICallAudioConfig

The local audio configuration for calls.

Property name	Type	Description
audioProfile	ARTCAICallAudioProfile	The audio encoding configuration. The default is HighQualityMode.
audioScenario	ARTCAICallAudioScenario	The audio scenario configuration. The default is ARTCAICallAudioSceneMusicMode.
ignoreSetAudioProfile	boolean	Specifies whether to skip calling the setAudioProfile interface when the SDK initializes the ARTC engine. By default, the interface is called.

ARTCAICallVideoConfig

The local video configuration for calls.

Property Name	Type	Description
dimensions	SizeOptions	The stream ingest resolution.
frameRate	number	The stream ingest frame rate.
bitrate	number	The stream ingest bitrate.
keyFrameInterval	number	The stream ingest Group of Pictures (GOP) interval in milliseconds.
useHighQualityPreview	boolean	Specifies whether to use high-definition preview. If not, the SDK automatically adjusts the resolution based on the stream ingest resolution.
cameraCaptureFrameRate	number	The preview resolution.
useFrontCameraDefault	boolean	Specifies whether to start the front camera by default.

ARTCAICallViewConfig

The configuration for rendering views. Use this class to configure the rendering of an agent or local camera, such as a digital human.

Property Name	Type	Description
controller	XComponentController	The rendering view controller.
viewMode	ARTCAICallAgentViewMode	The image rendering mode.
viewMirrorMode	ARTCAICallAgentViewMirrorMode	The image mirror mode.
viewRotationMode	ARTCAICallAgentViewRotationMode	The image rotation mode.

ARTCAICallChatSyncConfig

The configuration parameters for an associated chat agent session.

Property name	Type	Description
sessionId	string	The ID of the associated chat agent session.
agentId	string	The ID of the associated chat agent. The agent must be under the same account and in the same region. If configured, the call history is synchronized to the chat agent during the call.
receiverId	string	The user ID for the associated chat agent session.

ARTCAICallVisionCustomCaptureRequest

The request model for enabling custom frame capture for a visual understanding agent.

Property Name	Type	Description
text	string	The text parameter for requests to the multimodal large language model.
enableASR	boolean	Specifies whether to use the ASR result of human speech as input for the large language model. If true, the ASR result and the captured frame are sent to the large language model. If false, the text field and the captured frame are sent. Note: This parameter is effective only during continuous frame capture.
isSingle	boolean	If set to true, a single frame is captured, and the system exits the custom frame capture state immediately. If set to false, frames are captured continuously until the specified duration is reached, at which point the system automatically exits the custom frame capture state.
eachDuration	number	The interval for frame capture in seconds.
num	number	The number of images to capture each time.
duration	number	The duration for continuous frame capture in seconds. If isSingle is true, this parameter is ignored. Otherwise, the custom frame capture state ends when this duration is reached.
userData	string	A JSON string for custom business information. This is passed to the large language model along with the text and image frames for business processing.

ARTCAICallSendTextToAgentRequest

The request model for sending a text message to an agent.

Property name	Type	Description
text	string	The text message to send to the agent, such as "What is this?"

ARTCAICallUserSubtitleResult

The user caption notification result.

Property name	Type	Description
text	string	The query text recognized by the agent.
isSentenceEnd	boolean	Specifies whether the current text is the final result for this sentence.
sentenceId	number	The ID of the sentence to which the current text belongs.
voiceprintResult	ARTCAICallVoiceprintResult	The feedback on the current voiceprint denoising detection result.

ARTCAICallAgentSubtitleResult

The agent caption notification result.

Property name	Type	Description
text	string	The text of the agent's response.
isSentenceEnd	boolean	Specifies whether the current text is the last sentence of this response.
userAsrSentenceId	number	The ID of the sentence that answers the user's question.

ARTCAICallAgentVcrFrameMotionResult

The VCR detection result for silent or invalid frames.

Property Name	Type	Description
duration	number	How long ago the frame was sent, in milliseconds.

ARTCAICallAgentVcrPeopleCountResult

The VCR detection result for the real-time number of people in the frame.

Property Name	Type	Description
count	number	The result of the people count.

ARTCAICallAgentVcrEquipmentResult

The VCR detection result for electronic devices.

Property name	Type	Description
mobilePhoneCount	number	The number of mobile phones.
watchCount	number	The number of watches.
headPhoneCount	number	The number of headphones.

ARTCAICallAgentVcrHeadMotionResult

The VCR detection result for head movements.

Property name	Type	Description
nodDetected	boolean	Nodding.
shakeDetected	boolean	Shaking head.

ARTCAICallAgentVcrLookAwayResult

The VCR detection result for gaze deviation.

Property name	Type	Description
count	number	The total number of gaze deviations up to the current frame.
duration	number	The total duration of gaze deviation up to the current frame, in milliseconds.

ARTCAICallAgentVcrResult

The VCR detection result.

Property Name	Type	Description
resultData	Record<string, RecordValueType>	All VCR detection results returned by the agent.
stillFrameMotionResult	ARTCAICallAgentVcrFrameMotionResult	The VCR detection result for silent frames.
invalidFrameMotionResult	ARTCAICallAgentVcrFrameMotionResult	The VCR detection result for invalid frames.
peopleCountResult	ARTCAICallAgentVcrPeopleCountResult	The VCR detection result for the real-time number of people in the frame.
equipmentResult	ARTCAICallAgentVcrEquipmentResult	The VCR detection result for electronic devices.
headMotionResult	ARTCAICallAgentVcrHeadMotionResult	The VCR detection result for head movements.
lookAwayResult	ARTCAICallAgentVcrLookAwayResult	The VCR detection result for head movements.

ARTCAICallAgentShareConfig

The configuration for sharing an agent.

Property name	Type	Description
shareId	string	The agent share ID.
agentType	ARTCAICallAgentType	The agent workload type.
expireTime	string	The time-to-live (TTL).
region	string	The region where the agent is located.

ARTCAICallAgentShareGenerateResult

Sharing the agent-generated result

Property name	Type	Description
agentInfo	ARTCAICallAgentInfo	You can share agent information.
token	string	The startup token for the shared agent.

ARTCAICallAgentConfig

The startup and runtime configuration for a call agent.

Property name	Type	Description
agentGreeting	string	The agent's welcome message. If this is empty, the value configured for the agent is used.
wakeUpQuery	string	An instruction from the user before the call starts, which the agent responds to immediately after the call begins. For example: "What's the weather like today?"
agentMaxIdleTime	number	The maximum idle time for the agent in seconds. If the time is exceeded, the agent automatically goes offline. The default value is 600s. A value of -1 disables automatic shutdown, meaning the agent will never go offline automatically.
userOnlineTimeout	number	The timeout period for the agent to close the task if the user does not join the session. The default value is 60s. If you set this to a value less than 0, the setting is not sent to the server, and the server-side default of 60s is used.
userOfflineTimeout	number	The timeout period for the agent to close the task after the user leaves the session. The default value is 5s. If you set this to a value less than 0, the setting is not sent to the server, and the server-side default of 5s is used.
enablePushToTalk	boolean	Specifies whether to enable push-to-talk mode.
agentGracefulShutdown	boolean	Specifies whether to enable graceful shutdown. With graceful shutdown, when the agent is stopped, it finishes its current speech before stopping. The speech can last for a maximum of 10 seconds.
volume	number	The agent's speaking volume. The value ranges from 0 to 400. The output volume is calculated as: (Voice output volume in the workflow) × volume/100. The default value is 100. If you set this parameter to a value less than 0, the setting is not sent to the server, and the server-side default value of 100 is used.
workflowOverrideParams	Record<string, RecordValueType>	The workflow override parameters.
enableIntelligentSegment	boolean	The switch for intelligent sentence segmentation. When enabled, sentence breaks in the user's speech are intelligently merged into a single sentence.
preConnectAudioUrl	string	The sound effect to play after the connection is established but before the welcome message. You can provide a URL. The welcome message is still played after the sound effect finishes.
asrConfig	ARTCAICallAgentAsrConfig	The speech recognition configuration.
ttsConfig	ARTCAICallAgentTtsConfig	The speech synthesis configuration.
llmConfig	ARTCAICallAgentLlmConfig	The large language model (LLM) configuration.
avatarConfig	ARTCAICallAgentAvatarConfig	The digital human configuration.
interruptConfig	ARTCAICallAgentInterruptConfig	The interruption configuration.
voiceprintConfig	ARTCAICallAgentVoiceprintConfig	The voiceprint denoising configuration.
turnDetectionConfig	ARTCAICallAgentTurnDetectionConfig	The turn detection configuration.
experimentalConfig	ARTCAICallExperimentalConfig	The non-production, custom configuration.
vcrConfig	ARTCAICallAgentVcrConfig	The VCR configuration.
ambientConfig	ARTCAICallAgentAmbientConfig	The environment configuration.
backChannelingConfigs	ARTCAICallAgentBackChanneling[]	The configuration module for the back-channeling feature. When configured, the system randomly plays short back-channeling phrases at specific trigger points.
autoSpeechForLlmPendingConfig	ARTCAICallAgentAutoSpeechLlmPending	The configuration for the agent to automatically speak when there is a delay in the LLM response.
autoSpeechForUserIdleConfig	ARTCAICallAgentAutoSpeechUserIdle	The configuration for the agent to make an inquiry when the user is silent.

ARTCAICallAgentAsrConfig

The speech recognition configuration.

Property Name	Type	Description
asrLanguageId	string	The ASR language ID. If this is empty, the value configured for the agent is used.
asrMaxSilence	number	The threshold for voice activity detection to determine sentence breaks. A period of silence longer than this threshold is considered a sentence break. The value ranges from 200 ms to 1200 ms. The default value is -1. A value less than 0 indicates that the agent's default configuration (the value set in the console) is used.
asrHotWords	string[]	The list of ASR hot words.
vadLevel	number	Sets the sensitivity parameter for AI VAD to mitigate human voice interference. A value of 0 disables AI VAD. The default value is 11. If you set this to a value less than 0, the setting is not sent to the server.
vadDuration	number	The minimum duration threshold for voice activity detection, used to control interruption sensitivity. The default value of 0 disables this feature. The valid range is 200 to 2000 milliseconds. A common range is [200, 500], which corresponds to 1-4 words. If you set this to a value less than 0, the setting is not sent to the server, and the server-side default is to disable this feature.
customParams	string	For custom ASR integration, pass runtime parameters in URL parameter format. For example: "mode=fast&sample=16000&format=wav".

ARTCAICallAgentTtsConfig

The speech synthesis configuration.

Property name	Type	Description
agentVoiceId	string	The voice ID for the agent's speech. If this is empty, the value configured for the agent is used.
pronunciationRules	Array<Record<string, RecordValueType>>	An array of pronunciation rules, with a maximum of 20 rules. If nil or an empty array, no rules are used.
speechRate	number	The Text-to-Speech (TTS) playback speed. This is supported for all TTS types. The value ranges from [0.5, 2.0]. The default value is -1. A value less than 0 indicates that the agent's default configuration (the value set in the console) is used.
languageId	string	The language code for TTS playback. This is effective when the TTS type is MiniMax.
emotion	string	The emotion type for TTS playback. This is effective when the TTS type is MiniMax.
modelId	string	The TTS model ID. Currently, only MiniMax is supported. Possible values are speech-01-turbo and speech-02-turbo.

ARTCAICallAgentLlmConfig

The large language model (LLM) configuration.

Property Name	Type	Description
llmHistoryLimit	number	The maximum number of conversation turns to retain in the LLM/MLLM history context. The default value is -1. A value less than 0 indicates that the agent's default configuration (the value set in the console) is used.
llmSystemPrompt	string	The system prompt for the LLM. For example: "You are a friendly and helpful assistant, focused on providing users with accurate information and advice." Note: This is not supported if the LLM node is of the Alibaba Cloud Model Studio workflow type.
bailianAppParams	Record<string, RecordValueType>	Parameters for the Model Studio Application Center.
llmCompleteReply	boolean	Specifies whether to send the complete LLM result. When enabled, the complete LLM result is returned through the onLLMReplyCompleted event callback after it is generated.
openAIExtraQuery	string	Additional query parameters for an OpenAI protocol LLM. Parameters must be in key=value format, with multiple parameters separated by &. All values must be strings.
outputMinLength	number	The minimum length of the text output in characters. Text shorter than this length is cached for concatenation. The range is [0, 100]. A value less than or equal to 0 means no limit. The default is no limit.
outputMaxDelay	number	The maximum delay time for text output in milliseconds. Cached text is forcibly output if this time is exceeded. The range is [1000, 10000]. A value less than or equal to 0 means no limit. The default is no limit.
historySyncWithTTS	boolean	Specifies whether the LLM message history should be consistent with the TTS playback content. The default is false. When enabled, the saved LLM messages are consistent with the TTS playback content, with minor discrepancies allowed.

ARTCAICallAgentAvatarConfig

The digital human configuration.

Property Name	Type	Description
agentAvatarId	string	The digital human model ID. If this is empty, the value configured for the agent is used.

ARTCAICallAgentInterruptConfig

The interruption configuration.

Property Name	Type	Description
enableVoiceInterrupt	boolean	Specifies whether to enable intelligent interruption.
interruptWords	string[]	Specific words or phrases that trigger a conversation interruption, such as "Hold on" or "I got it."

ARTCAICallAgentVoiceprintConfig

The voiceprint denoising configuration.

Property name	Type	Description
useVoiceprint	boolean	Specifies whether to use voiceprint denoising for the current sentence break.
voiceprintId	string	The voiceprint ID. If not empty, voiceprint denoising is enabled for the current call. If empty, voiceprint denoising is not enabled.
registrationMode	string	The voiceprint registration method. Explicit: pre-registration mode. Implicit: seamless registration mode. An empty or other value is treated as Explicit.

ARTCAICallAgentTurnDetectionConfig

The turn detection configuration.

Property name	Type	Description
turnEndWords	string[]	Specific words that indicate the end of a sentence, such as "Over" or "I'm done." If these words are set and triggered during speech, the current speech content is sent to the LLM.
mode	ARTCAICallTurnDetectionMode	The mode for determining whether the user has finished speaking. The default is Semantic, which uses AI to determine the end of speech based on semantics.
semanticWaitDuration	number	The custom waiting time for semantic sentence breaking, in milliseconds. The range is [0, 10000]. If you set this to a value less than 0, the setting is not sent to the server, and the server-side default of -1 is used, where AI automatically determines the appropriate waiting time.

ARTCAICallAgentVcrConfig

The VCR configuration.

Property name	Type	Description
stillFrameMotion	ARTCAICallAgentVcrFrameMotionConfig	The VCR detection configuration for silent frames.
invalidFrameMotion	ARTCAICallAgentVcrFrameMotionConfig	The VCR detection configuration for invalid frames.
peopleCount	ARTCAICallAgentVcrBaseConfig	The VCR detection configuration for the real-time number of people in the frame.
equipment	ARTCAICallAgentVcrBaseConfig	The VCR detection configuration for electronic devices.
headMotion	ARTCAICallAgentVcrBaseConfig	The VCR detection configuration for head movements.
lookAway	ARTCAICallAgentVcrBaseConfig	The VCR detection configuration for gaze deviation.

ARTCAICallAgentVcrBaseConfig

The basic VCR detection configuration.

Property name	Type	Description
enable	boolean	Specifies whether to enable the feature. Enabled by default.

ARTCAICallAgentVcrFrameMotionConfig

The VCR video frame detection configuration.

Property name	Type	Description
enable	boolean	Specifies whether to enable the feature. Enabled by default.
callbackDelay	number	The delay in milliseconds before the callback is triggered. The default is 3000 ms.

ARTCAICallExperimentalConfig

The experimental configuration.

Property name	Type	Description
isEnableBurstSend	boolean	Specifies whether to enable non-streaming audio sending. (Internal SDK interface: This value is enabled when the SDK starts a call, the agent is a voice agent, and `ARTCAICallBase.isEnableBurst=true`).
rtcSdkParams	Record<string, RecordValueType>	rtcsdk parameter.
commonParams	Record<string, RecordValueType>	The common parameters.

ARTCAICallAgentAmbientConfig

The call environment parameters.

Property Name	Type	Description
volume	number	The background sound volume. The default is 100.
resourceId	string	The resource ID after registering the background sound in the console. An empty string disables it.

ARTCAICallAgentAutoSpeechContent

The content that the agent speaks automatically in scenarios such as back-channeling and proactive inquiries.

Property Name

Type

Description

probability

number

The trigger probability. The range is 0.0 to 1.0.

text

string

The prompt text in UTF-8 encoding. Example: "Are you still there?" The text is limited to a maximum of 20

The maximum length for the Auto-reply scenario is 100 characters.

ARTCAICallAgentAutoSpeechLlmPending

The configuration for the agent to automatically speak when there is a delay in the LLM response.

Property name	Type	Description
waitTime	number	The waiting time threshold in milliseconds. A prompt is triggered if this time is exceeded. The range is 500 to 10000 ms. This cannot be empty.
messages	ARTCAICallAgentAutoSpeechContent[]	A collection of waiting prompts, with a maximum of 10 items. Each item can be up to 100 characters long. The sum of probabilities must be 1.0.

ARTCAICallAgentAutoSpeechUserIdle

The configuration for the agent to make an inquiry when the user is silent.

Property name	Type	Description
waitTime	number	The silence duration threshold in milliseconds. An inquiry is triggered if this time is exceeded. The range is 5000 to 600000 ms. A value of 10000 is recommended.
maxRepeats	number	The maximum number of inquiries. The range is 0 to 10. A value of 5 is recommended. After the maximum is reached, no more inquiries are triggered, and the call is disconnected.
messages	ARTCAICallAgentAutoSpeechContent[]	A collection of waiting prompts, with a maximum of 10 items. Each item can be up to 100 characters long. The sum of probabilities must be 1.0.

ARTCAICallAgentBackChanneling

The configuration module for the back-channeling feature. When enabled, the agent randomly plays short back-channeling phrases at specific trigger points.

Property name	Type	Description
enabled	boolean	Specifies whether to enable the back-channeling feature.
triggerStage	string	The stage at which back-channeling is triggered.
probability	number	The trigger probability. The range is 0.0 to 1.0.
words	ARTCAICallAgentAutoSpeechContent[]	A collection of waiting prompts, with a maximum of 10 items. Each item can be up to 100 characters long. The sum of probabilities must be 1.0.