Connect to an image generation agent over HTTP
Features
The image generation agent provides text-to-image, image-to-image, and scribble-to-image capabilities via HTTP. Currently, only pass-through links are supported.
This service is ideal for use cases such as generating wallpapers, creating scribble art, enhancing photos, and stylizing images.
A pass-through link sends a request directly to the image generation agent and returns the resulting image, bypassing nodes such as automatic speech recognition (ASR), intent recognition, and text-to-speech (TTS).
Integration
Prerequisites
Activate Alibaba Cloud Model Studio (Bailian) and obtain an API key.
The API key authenticates your access to Alibaba Cloud Model Studio (Bailian). For instructions, see Obtain an API Key.
Console configuration
-
In the multi-modal development kit, create a multimodal interactive application. Select the All-in-one Edition (not the Vision Edition), and disable voice interaction. Keep intent recognition and the text model enabled.
For the text model, select Multimodal Interaction Proprietary Model - High-speed Edition.
-
Disable dialogue carry-over, knowledge base, internet search, and long-term memory.
Set Number of Context Rounds to
5. -
Clear all skills and MCP services. Keep only the image generation agent.
-
Configure the image generation agent, which supports three modes: scribble-to-image, image assistant, and text-to-image.
- Scribble-to-image: generates images based on hand-drawn scribble line art.
- Image assistant: generates new images based on an uploaded image.
- Text-to-image: generates images based on text instructions.
-
Each mode supports options such as model selection, prompts, intelligent optimization of positive prompts, and negative prompts. You can add variables to prompts to dynamically pass different prompts.
For scribble-to-image, the Model Selection option provides three choices: Image-to-Image Advanced Edition, Image-to-Image Balanced Edition, and Image-to-Image Lightweight Edition.
-
After configuration, click the publish button in the upper-right corner to publish (you must publish before you can test).
HTTP access
Request parameters
Parameter | Sub-parameter | Type | Required | Description |
model | string | Yes | The name of the Alibaba Cloud Model Studio (Bailian) model. The value is fixed to "multimodal-dialog". Copy it directly. | |
input | directive | string | Yes | The directive name. Set this to Request. |
app_id | string | Yes | Your application ID. You can find it on the "My Applications" page of the Multimodal Interaction Development Kit console. For more information, see Get an Application ID. | |
dialog_id | string | No | The dialog ID. If you omit this parameter, the server starts a new dialog and returns a dialog ID. The ID is a 36-character string, such as "12345678-1234-1234-1234-1234567890ab". To continue a previous conversation, pass the dialog_id returned by the server for that conversation. Image generation modes do not use context, so you can leave this empty. | |
text | string | Yes | The text to process. Note: in this use case, if there is no text, set it to "" (empty string). | |
parameters | client_info | object | Yes | For parameter details, see the parameters.client_info parameter table below. |
images | list[] | No | The reference images. Required in image-to-image scenarios, not required for text-to-image. For parameter details, see the parameters.images parameter table. | |
biz_params | object | No | Configure this parameter as needed. For parameter details, see the parameters.biz_params parameter table below. |
Parameters: parameters.client_info
Parameter | Sub-parameter | Type | Required | Description |
|---|---|---|---|---|
user_id | string | Yes | The end user ID. Generate this ID based on your business rules. It enables customized features for different end users. Maximum length: 36 characters. | |
device | uuid | string | No | A globally unique ID for the client. Generate this ID and pass it to the SDK. Maximum length: 40 characters. An end user can have multiple devices, each with a different UUID but the same user_id. |
Parameters: parameters.images
Parameter | Type | Required | Description |
|---|---|---|---|
type | string | Yes | The type of the image. Two types are supported: base64/url. |
value | string | Yes | The image content.
|
Parameters: parameters.biz_params
Parameter | Sub-parameter | Type | Required | Description |
|---|---|---|---|---|
commands[i] | name | string | Yes | Indicates a direct pass-through to the agent. For this use case, the value is fixed to "agent_command". Copy it directly. |
exec_params | object | Yes | Specifies the agent. This value is fixed for this use case. For details, see the commands.exec_params parameter description. | |
user_defined_params | image_to_image | object | No | User-defined agent parameters. image_to_image indicates the parameters passed to the image generation agent. For this use case, populate this only when passing custom prompt variables to the image generation agent. |
Parameters: parameters.biz_params.commands.exec_params
Parameter | Sub-parameter | Type | Required | Description |
|---|---|---|---|---|
exec_params | app_id | string | Yes | Specifies the agent. For this use case, the value is fixed to "image_to_image". Copy it directly. |
intent | string | Yes | Specifies the action of the agent. For this use case, the value is fixed to "open_image_to_image". Copy it directly. | |
slots | list[] | Yes | The slot information required by the agent. |
Parameters: parameters.biz_params.commands.exec_params.slots
name | string | Yes | The name of the slot. |
norm_value | string | Yes | The value of the slot. |
For the image generation agent, you must populate the function and size slots as shown in the following example.
{
"slots": [
{
"name": "function",
"norm_value" :"imageAssistant"
},
{
"name":"size",
"norm_value":"768*768"
}
]
}
The function parameter specifies the image generation feature to use. Supported values:
textToImage: text-to-imageimageAssistant: Image AssistantsketchToImage: Sketch-to-image
The size parameter specifies the resolution of the generated image. The supported resolution range depends on the selected feature and model. See the following table for supported resolutions.
Image generation mode | Model configuration | Supported resolutions |
text-to-image | Lightweight Edition | The image width and height range is [200, 2048], and the aspect ratio range is [1:4, 4:1]. For example, 256x2048 is not allowed because the aspect ratio is 1:8. |
Balanced Edition | ||
Advanced Edition | Image resolution: total pixels between [1280*1280, 1440*1440] Image aspect ratio: [1:4, 4:1] | |
Image Assistant/Sketch-to-image | Lightweight Edition | Image resolution: the width and height are both within [512, 4096] pixels. |
Balanced Edition | Image resolution: the width and height are both within [512*512, 2048*2048] pixels. | |
Advanced Edition | Image resolution: total pixels within [768*768, 2048*2048] pixels. |
Parameters: parameters.biz_params.user_defined_params.image_to_image.user_prompt_params
ImportantPopulate this parameter only when passing custom variables for prompts to the image generation agent.
In the prompt area of the Sketch-to-image configuration panel, click the {x} Custom Variable button in the upper-right corner to add custom variables. ${style} is a default, built-in variable.
Parameter | Sub-parameter | Type | Required | Description |
|---|---|---|---|---|
user_prompt_params | positive | object | No | Custom variables for the prompt. |
negative | object | Yes | Custom variables for the negative prompt. |
"user_defined_params": {
"image_to_image": {
"user_prompt_params": {
"positive": {
"hair": "长头发",
"name": "张三"
},
"negative": {
"style": "低分辨率"
}
}
}
}
Text-to-image request example
{
"model": "multimodal-dialog",
"input": {
"directive": "Request",
"app_id": "xxxxxx",
"text": ""
},
"parameters": {
"client_info": {
"user_id": "test-251222",
"device": {
"uuid": "test-251222-123"
}
},
"biz_params": {
"commands": [
{
"name": "agent_command",
"exec_params": {
"app_id": "image_to_image",
"intent": "open_image_to_image",
"slots": [
{
"name": "function",
"norm_value" :"textToImage"
},
{
"name":"size",
"norm_value":"768*768"
}
]
}
}
],
"user_defined_params": {
"image_to_image": {
"user_prompt_params":{
"positive":{
"hair": "长头发",
"name": "张三"
},
"negative":{
"style": "低分辨率"
}
}
}
}
}
}
}
Text-to-image cURL request example
curl --location "https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation" \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer your_api_key' \
--header 'X-DashScope-SSE: enable' \
--data '{
"model": "multimodal-dialog",
"input": {
"directive": "Request",
"app_id": "xxxxxx",
"text": ""
},
"parameters": {
"client_info": {
"user_id": "test-251222",
"device": {
"uuid": "test-251222-123"
}
},
"biz_params": {
"commands": [
{
"name": "agent_command",
"exec_params": {
"app_id": "image_to_image",
"intent": "open_image_to_image",
"slots": [
{
"name": "function",
"norm_value": "textToImage"
},
{
"name": "size",
"norm_value": "768*768"
}
]
}
}
],
"user_defined_params": {
"image_to_image": {
"user_prompt_params": {
"positive": {
"hair": "长头发",
"name": "张三"
},
"negative": {
"style": "低分辨率"
}
}
}
}
}
}
}'
Important
- Replace
your_api_key(API_KEY),app_id, andimg_urlin the request. - Set the
textfield to an empty string (""). Otherwise, the request enters intent recognition. - Pass
user_defined_paramsonly when you need to pass custom Agent prompt variables in image generation modes.
Image assistant request example
{
"model": "multimodal-dialog",
"input": {
"directive": "Request",
"app_id": "xxxxxx",
"text": ""
},
"parameters": {
"images": [
{
"type": "url",
"value": "img_url"
}
],
"client_info": {
"user_id": "test-251222",
"device": {
"uuid": "test-251222-123"
}
},
"biz_params": {
"commands": [
{
"name": "agent_command",
"exec_params": {
"app_id": "image_to_image",
"intent": "open_image_to_image",
"slots": [
{
"name": "function",
"norm_value": "imageAssistant"
},
{
"name": "size",
"norm_value": "768*768"
}
]
}
}
],
"user_defined_params": {
"image_to_image": {
"user_prompt_params": {
"positive": {
"hair": "长头发",
"name": "张三"
},
"negative": {
"style": "低分辨率"
}
}
}
}
}
}
}
Image assistant cURL request example
curl --location "https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation" \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer your_api_key' \
--header 'X-DashScope-SSE: enable' \
--data '{
"model": "multimodal-dialog",
"input": {
"directive": "Request",
"app_id": "xxxxxx",
"text": ""
},
"parameters": {
"images": [
{
"type": "url",
"value": "img_url"
}
],
"client_info": {
"user_id": "test-251222",
"device": {
"uuid": "test-251222-123"
}
},
"biz_params": {
"commands": [
{
"name": "agent_command",
"exec_params": {
"app_id": "image_to_image",
"intent": "open_image_to_image",
"slots": [
{
"name": "function",
"norm_value": "imageAssistant"
},
{
"name": "size",
"norm_value": "768*768"
}
]
}
}
],
"user_defined_params": {
"image_to_image": {
"user_prompt_params": {
"positive": {
"hair": "长头发",
"name": "张三"
},
"negative": {
"style": "低分辨率"
}
}
}
}
}
}
}'
Important
- Replace
your_api_keywith your API key,app_idwith your app ID, andimg_urlwith the image URL. - Set the
textfield to an empty string (""). Otherwise, the request is routed for intent recognition. - Use the
user_defined_paramsparameter only when passing custom variables to the agent prompt for image generation.
Sketch-to-image request
Use the Image Assistant request example, changing the value of the function slot in parameters.biz_params.commands.exec_params.slots from imageAssistant to sketchToImage.
Text response events
Top-level parameter | Second-level parameter | Type | Required | Description |
|---|---|---|---|---|
output | event | string | Yes | Event name: RespondingContent |
dialog_id | string | Yes | A unique identifier for the dialog. | |
round_id | string | Yes | A unique identifier for the current round of interaction. | |
llm_request_id | string | Yes | The request ID for the call to the large language model (LLM). | |
text | string | Yes | The URL of the generated image. The value is RUNNING while generation is in progress and becomes the image URL after generation completes. | |
spoken | string | Yes | Content for Text-to-Speech (TTS) synthesis. Ignore this field if TTS is not enabled. In this use case, the content is the same as the text field. | |
finished | bool | Yes | Indicates whether the output stream has finished. | |
finish_reason | string | No | The reason for finishing. Currently, only one value is supported:
| |
extra_info | object | No | Other extended information. Currently supported:
|
Parameters for output.extra_info.agent_info
Top-level parameter | Second-level parameter | Type | Required | Description |
|---|---|---|---|---|
round | string | Yes | The dialog round number. | |
device | device_id | string | Yes | The device.uuid used in the request. |
intent_infos | intent | string | Yes | The agent used. For this use case, the value is fixed to "open_image_to_image". |
domain | string | Yes | The agent used. For this use case, the value is fixed to "image_to_image". |
Response example
{
"output": {
"round_id": "xxx",
"llm_request_id": "xxx",
"extra_info": {
"agent_info": {
"round": 1,
"device": {
"device_id": "test-251222-123"
},
"intent_infos": [
{
"intent": "open_image_to_image",
"domain": "image_to_image"
}
]
},
"query": ""
},
"dialog_id": "6c4340eb-xxx-4055-8d66-0794df7986e0",
"spoken": "img_url",
"finished": true,
"text": "img_url",
"event": "RespondingContent"
},
"request_id": "xxx"
}
Important
- During image generation, the service sends a heartbeat every 3–5 seconds. While in progress,
output.textisRUNNINGandoutput.finishedisfalse. - When image generation completes,
output.finishedbecomestrue, andoutput.textcontains the image URL.
id:1 合成过程中
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbbfe84a8xxx","llm_request_id":"f22feed889994756b3787de8b08bxxx","extra_info":{"tool_infos":[{"output":"","tool_name":"image_to_image","arguments":{},"type":"agent","success":true}]},"dialog_id":"00cd961c-bf58-49f5-b3bb-7c5ca9xxx","spoken":"","finished":false,"text":"","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fcb7xxx"}
id:2 合成过程中
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbbfe84xxx","llm_request_id":"f22feed889994756b3787de8b0xxx","extra_info":{"agent_info":{"round":1,"device":{"device_id":"xxx-test-251222-123"},"intent_infos":[{"intent":"open_image_to_image","domain":"image_to_image"}]},"query":""},"dialog_id":"00cd961c-bf58-49f5-b3bb-7c5xxx","spoken":"RUNNING","finished":false,"text":"RUNNING","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fcb7xxx"}
id:3 合成过程中
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbbfe84xxx","llm_request_id":"f22feed889994756b3787de8bxxx","extra_info":{"agent_info":{"round":1,"device":{"device_id":"xxx-test-251222-123"},"intent_infos":[{"intent":"open_image_to_image","domain":"image_to_image"}]},"query":""},"dialog_id":"00cd961c-bf58-49f5-b3bb-7c5cxxx","spoken":"RUNNING","finished":false,"text":"RUNNING","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fcb71xxx"}
id:4 合成过程中
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbb1xxx","llm_request_id":"f22feed889994756b3787xxx","extra_info":{"agent_info":{"round":1,"device":{"device_id":"xxx-test-251222-123"},"intent_infos":[{"intent":"open_image_to_image","domain":"image_to_image"}]},"query":""},"dialog_id":"00cd961c-bf58-49f5-b3bb-xxx","spoken":"RUNNING","finished":false,"text":"RUNNING","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fclxxx"}
id:5 合成结束
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbbfxxx","llm_request_id":"f22feed889994756b3787de8xxx","extra_info":{"tool_infos":[{"output":"","tool_name":"image_to_image","arguments":{},"type":"agent","success":true}],"agent_info":{"round":1,"device":{"device_id":"xxx-test-251222-123"},"intent_infos":[{"intent":"open_image_to_image","domain":"image_to_image"}]},"query":""},"finish_reason":"stop","dialog_id":"00cd961c-bf58-49f5-b3bb-7cxxx","spoken":"https://dashscope-result-sh.oss-cn-shanghai.aliyuncs.com/1d/03/20260107/c6e3cb4b/eb8e554d-66ff-44ca-ba0d-f05481405d2c-1-1.png?Expires=1767854752&OSSAccessKeyId=YOUR_ACCESS_KEY_ID&Signature=YOUR_SIGNATURE","finished":true,"text":"https://dashscope-result-sh.oss-cn-shanghai.aliyuncs.com/1d/03/20260107/c6e3cb4b/eb8e554d-66ff-44ca-ba0d-f05481405d2c-1-1.png?Expires=1767854752&OSSAccessKeyId=YOUR_ACCESS_KEY_ID&Signature=YOUR_SIGNATURE","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fcbxxx"}