Integrate AICallKit SDK for web-Intelligent Media Services(IMS)-阿里云帮助中心

Use the AICallKit SDK to add an AI messaging agent to your web application.

Environment requirements

Node 18 or later.
NPM 10 or later.
Webpack 5 or later.

Integrate the SDK

Install the SDK.

npm install aliyun-auikit-aicall --save

SDK usage example

import { AIChatEngine, AIChatAgentInfo, AIChatUserInfo } from 'aliyun-auikit-aicall';

const chatEngine = new AIChatEngine();

chatEngine.on('requestAuthToken', async (userId, callback) => {
  //Required. Call generateMessageChatToken via the app server or business server to get the token.
  

  // role and nonce are optional parameters.
  return new AIChatAuthToken(appId, appSign, token, timestamp, role, nonce);

  // If the returned value is a JSON object containing app_id and app_sign, you can simplify it to: 
  // return AIChatAuthToken.fromData(data);
});

chatEngine.on('engineStateChange', (state) => {
  // Callback for engine status.
});
chatEngine.on('receivedMessage', (message) => {
  // Callback for received messages.
});
chatEngine.on('errorOccurs', (error) => {
  // Callback for errors.
});
chatEngine.on('receivedCustomMessage', (message) => {
  // Callback for received custom messages.
});
chatEngine.on('messagePlayStateChange', (message, state) => {
  // Callback for message readout status.
});

const userId = 'xxxx'; // The unique identifier for the user who joins the conversation. It is recommended to use the user ID of the business system.
const agentId = 'xxxx'; // The ID of the AI agent created in the console. 
const sessionId = ''; // The ID of the conversation session. If empty, the session ID defaults to a concatenation of userId and agentId. 
const region = 'cn-xxxx'; // The region where the AI agent is deployed. The region defaults to cn-shanghai.
await chatEngine.startChat(new AIChatUserInfo(userId), new AIChatAgentInfo(agentId, region), sessionId);

API reference

Overview

Class/Protocol	API	Description
AIChatEngine Messaging engine interface	state	Gets the current engine state.
	userInfo	Gets the current session user info.
	sessionId	Gets the current session ID.
	agentInfo	Gets the current session AI agent info.
	agentResponseState	Gets the current response status of the agent.
	voiceIdList	Retrieves the available voices.
	startChat	Starts a chat session.
	endChat	Ends a chat session.
	sendMessage	Sends a message. This will interrupt the ongoing agent response.
	queryMessageList	Requests historical messages.
	interruptAgentResponse	Interrupts the current reply.
	deleteMessage	Deletes a message.
	startPlayMessage	Starts reading out a message.
	stopPlayMessage	Stops reading out a message.
	isPlayingMessage	Checks if a message is being read out.
	startPushVoiceMessage	Starts push-to-talk recording. Records and sends voice input in real time.
	finishPushVoiceMessage	Ends push-to-talk recording. Called on button release to signal that the voice message has been sent.
	cancelPushVoiceMessage	Cancels push-to-talk recording.
	createAttachmentUploader	Creates an attachment uploader for sending messages with images, videos, or other files.
	destroy	Releases resources.
	parseShareAgentChat	Parses shared agent info.
	generateShareAgentChat	Starts a conversation with a shared agent.
AIChatEngineEvents Messaging engine callbacks	errorOccurs	Triggered when an error occurs.
	requestAuthToken	Triggered when ChatEngine requires IM login or when AuthToken expires.
	engineStateChange	Triggered when the agent connection state changes.
	receivedMessage	Triggered when an agent reply is received or its status changes.
	userMessageUpdated	Triggered when a user-sent message needs updating.
	agentResponseStateChange	Triggered when the agent response status changes.
	messagePlayStateChange	Triggered when the message readout status changes.
AIChatAttachmentUploader Manages attachment uploads	addAttachment	Adds an attachment object.
	removeAttachment	Deletes an attachment object.
	allUploadSussess	Checks if all attachments have been uploaded.
AIChatAttachmentUploadEvents Attachment upload callbacks	uploadSuccess	Triggered on successful upload.
	uploadFailure	Triggered on upload failure.
AIChatAttachmentEvents Attachment events	progress	Triggered on upload progress update.
AIChatAttachmentEvents Attachment events	stateChange	Triggered on upload status change.

AIChatEngineInterface

State

Gets the current engine state.

state: AIChatEngineState;

Userinfo

Gets the current session user info.

userInfo: AIChatUserInfo;

Sessionid

Gets the current session ID.

sessionId: string;

Agentinfo

Gets the current session AI agent info.

agentInfo: AIChatAgentInfo;

Agentresponsestate

Gets the current response status of the agent.

agentResponseState: AIChatAgentResponseState;

VoiceIdList

Retrieves the available voices.

voiceIdList: string[];

Startchat

Starts a chat session.

async startChat(userInfo: AIChatUserInfo, agentInfo: AIChatAgentInfo, sessionId?: string);

Parameter	Type	Description
userInfo	AIChatUserInfo	User joining the session.
agentInfo	AIChatAgentInfo	AI agent joining the session.
sessionId	string (optional)	Session ID. Defaults to a concatenation of `userId` and `agentId` if empty.

Endchat

Ends a chat session.

async endChat(needLogout: boolean);

Parameter	Type	Description
needLogout	boolean	Whether to log out. Set to false to reduce reconnection time on the next startChat call.

Sendmessage

Sends a message. This will interrupt the ongoing agent response.

async sendMessage(message: AIChatMessage): Promise<AIChatMessage>;

Parameter	Type	Description
message	AIChatMessage	The message content.

Querymessagelist

Requests historical messages.

async queryMessageList(request: AIChatMessageListRequest): Promise<AIChatMessage[]>;

Parameter	Type	Description
request	AIChatMessageListRequest	The request model.

InterruptAgentResponse

Interrupts the current reply.

interruptAgentResponse();

Deletemessage

Deletes a message.

async deleteMessage(dialogueIdOrMessage: string | AIChatMessage);

Parameter	Type	Description
dialogueIdOrMessage	string \| AIChatMessage	The message ID or the message object.

Startplaymessage

Starts reading out a message.

async startPlayMessage(message: AIChatMessage, voiceId?: string);

Parameter	Type	Description
message	AIChatMessage	The message to read out.
voiceId	string?	The voice ID.

Stopplaymessage

Stops reading out a message.

stopPlayMessage();

Isplayingmessage

Checks if a message is being read out.

isPlayingMessage(dialogueId: string): boolean;

Startpushvoicemessage

Starts push-to-talk recording. Records and sends voice input in real time.

async startPushVoiceMessage(request: AIChatSendMessageRequest): Promise<AIChatMessage | undefined>;

Finishpushvoicemessage

Ends push-to-talk recording. Called on button release to signal that the voice message has been sent.

async finishPushVoiceMessage(): Promise<AIChatMessage | undefined>;

Cancelpushvoicemessage

Cancels push-to-talk recording.

cancelPushVoiceMessage();

Createattachmentuploader

Creates an attachment uploader for sending messages with images, videos, or other files.

/**
  * Creates an attachment uploader object. You can create only one.
  * @return
*/
public async createAttachmentUploader(): Promise<AIChatAttachmentUploader>;

Destroy

Releases resources.

destroy();

ParseShareAgentChat

Parses shared agent info.

parseShareAgentChat(shareInfo: string): AIChatAgentShareConfig | undefined;

Parameter	Type	Description
shareInfo	string	The shared information.

GenerateShareAgentChat

Starts a conversation with a shared agent.

async generateShareAgentChat(shareConfig: AIChatAgentShareConfig, userId: string);

Parameter	Type	Description
shareConfig	AIChatAgentShareConfig	The shared information.
userId	string	The user ID.

AIChatEngineEvents

ErrorOccurs

Triggered when an error occurs.

errorOccurs(error: AIChatError): void;

Parameter	Type	Description
error	Error	The error message.

RequestAuthToken

Triggered when ChatEngine requires IM login or when AuthToken expires.

requestAuthToken(userId: string, responseBlock: (authToken: AIChatAuthToken | undefined, error: Error | undefined) => void): void;

Parameter	Type	Description
userId	string	The user ID
responseBlock	(authToken: AIChatAuthToken \| undefined, error: Error \| undefined) => void	The token callback.

Enginestatechange

Triggered when the agent connection state changes.

engineStateChange(state: AIChatEngineState): void;

Parameter	Type	Description
state	AIChatEngineState	The current connection state of the agent.

Receivedmessage

Triggered when an agent reply is received or its status changes.

receivedMessage(message: AIChatMessage): void;

Parameter	Type	Description
message	AIChatMessage	The message object.

Usermessageupdated

Triggered when a user-sent message needs updating.

userMessageUpdated(message: AIChatMessage): void;

Parameter	Type	Description
message	AIChatMessage	The message object.

AgentResponseStateChange

Triggered when the agent response status changes.

agentResponseStateChange(state: AIChatAgentResponseState, requestId?: string): void;

Parameter	Type	Description
state	AIChatAgentResponseState	The response status.
requestId	string?	The ID of the current session.

MessagePlayStateChange

Triggered when the message readout status changes.

messagePlayStateChange(message: AIChatMessage, state: AIChatMessagePlayState): void;

Parameter	Type	Description
message	AIChatMessage	The message object.
state	AIChatMessagePlayState	The readout status.

AIChatAttachmentUploader

Addattachment

Adds an attachment object.

/**
   * Adds an attachment.
   * @param attachment
   */
public async addAttachment(attachment: AIChatAttachment);

Parameter	Type	Description
attachment	AIChatAttachment	The attachment object.

Removeattachment

Deletes an attachment object.

 /**
   * Deletes an attachment.
   * @param attachmentId
   */
public async removeAttachment(attachmentId: string)

Parameter	Type	Description
attachmentId	string	The attachment ID.

AllUploadSussess

Checks if all attachments have been uploaded.

 /**
   * Checks if all attachments have been uploaded.
   */
const uploadSussess = uploader.allUploadSussess

AIChatAttachmentUploadEvents

uploadSuccess

Triggered on successful upload.

uploadSuccess: (attachment: AIChatAttachment): void;

Parameter	Type	Description
attachment	ARTCAIChatAttachment	The attachment object.

Uploadfailure

Triggered on upload failure.

uploadFailure: (attachment: AIChatAttachment, error: Error): void;

Parameter	Type	Description
attachment	ARTCAIChatAttachment	The attachment object.
error	Error	The reason for upload failure.

AIChatAttachmentEvents

Progress

Triggered on upload progress update.

progress: (progress: number) => void;

Parameter	Type	Description
progress	number	The current upload progress.

Statechange

Triggered on upload status change.

stateChange: (state: AIChatAttachmentState) => void;

Parameter

Type

Description

state

AIChatAttachmentState

The upload status. Valid values:

Init: initializing
Uploading
Success: uploaded
Failed: failed to upload