Large model gateway protocol-Artificial Intelligence Cloud Call Service(AICCS)-阿里云帮助中心

This document describes the protocol for integrating large models with the communication engine gateway. Your large model must comply with this protocol to connect to the gateway.

Request parameters

Parameter	Type	Description	Example
top_p	String	The nucleus sampling parameter of the model.	0.1
stream	Boolean	Specifies whether to use streaming. This value must be set to `true`.	true
top_k	String	The top-k sampling parameter of the model.	1
temperature	String	Controls the randomness of the model's output.	0.1
session_id	String	The unique ID for the call. This ID is returned when the call starts.	147417592084^1341********
messages	Array[Object]	An array of objects containing the conversation history.	-
role	String	The role of the message sender. Valid values: `assistant`: Indicates a message from the bot. `user`: Indicates a message from the customer.	assistant
content	String	The message content. This is the bot's response for the `assistant` role or the customer's input for the `user` role.	Hello, valued Alibaba Cloud customer. This is a test of the opening message. We apologize for any inconvenience.
model	String	The model version to use for the request. For example, Qwen provides versions such as `qwen-plus`, `qwen-turbo`, and `qwen-max`.	qwen-plus
biz_params	Object	Custom business parameters.	{"city": "Hangzhou"}
out_id	String	A caller-provided ID that corresponds to the `OutId` parameter in the initial call request.	222356****

Example

{
  "top_p": 0.1,
  "stream": true,
  "top_k": 1,
  "temperature": 0.1,
  "session_id": "147417592084^1341********",
  "messages": [
    {
      "role": "assistant",
      "content": "Hello, valued Alibaba Cloud customer. This is a test of the opening message. We apologize for any inconvenience."
    },
    {
      "role": "user",
      "content": "What is this about?"
    }
  ],
  "model": "qwen-plus",
  "biz_params": {
    "city": "Hangzhou"
  },
  "out_id": "222356****"
}

Return values

Parameter	Type	Example	Description
choices	Array[Object]	-	An array of response objects.
delta	Object	-	The message delta generated for a streaming response.
content	String	I am Qwen	The content of the message delta.

Example

{
  "choices": [
    {
      "delta": {
        "content": "I am Qwen"
      }
    }
  ]
}

Validate integration with SSE streaming

Before integrating your large model, you can use SSE streaming to validate its compatibility with the communication engine gateway.

The integration uses HTTP, and the connection must support SSE streaming output.

Run the following cURL command to test the integration.

Note

Before running the command, replace the following placeholders: https://********** with your URL (the Model Address on the configuration page), $AUTH with your authorization code (the Authorization Code on the configuration page), and the model value with your model (the Base Model in Model Configuration settings).

Request:

curl --location "https://**********" \
--header "Authorization: $AUTH" \
--header "Content-Type: application/json" \
--data '{
    "model": "*****",
    "messages": [
        {
            "role": "assistant",
            "content": "This is a test opening message."
        },
        {
            "role": "user", 
            "content": "Who are you?"
        }
    ],
    "stream":true
}'

Response:

data: {"choices":[{"delta":{"content":"I am"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":173****523,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-********-7ae6-95f2-****-4b26****90c5"}