Connect to an image generation agent over HTTP

Updated at:

Features

The image generation agent provides text-to-image, image-to-image, and scribble-to-image capabilities via HTTP. Currently, only pass-through links are supported.

This service is ideal for use cases such as generating wallpapers, creating scribble art, enhancing photos, and stylizing images.

A pass-through link sends a request directly to the image generation agent and returns the resulting image, bypassing nodes such as automatic speech recognition (ASR), intent recognition, and text-to-speech (TTS).

Integration

Prerequisites

Activate Alibaba Cloud Model Studio (Bailian) and obtain an API key.

The API key authenticates your access to Alibaba Cloud Model Studio (Bailian). For instructions, see Obtain an API Key.

Console configuration

  1. In the multi-modal development kit, create a multimodal interactive application. Select the All-in-one Edition (not the Vision Edition), and disable voice interaction. Keep intent recognition and the text model enabled.

    For the text model, select Multimodal Interaction Proprietary Model - High-speed Edition.

  2. Disable dialogue carry-over, knowledge base, internet search, and long-term memory.

    Set Number of Context Rounds to 5.

  3. Clear all skills and MCP services. Keep only the image generation agent.

  4. Configure the image generation agent, which supports three modes: scribble-to-image, image assistant, and text-to-image.

    1. Scribble-to-image: generates images based on hand-drawn scribble line art.
    2. Image assistant: generates new images based on an uploaded image.
    3. Text-to-image: generates images based on text instructions.
  5. Each mode supports options such as model selection, prompts, intelligent optimization of positive prompts, and negative prompts. You can add variables to prompts to dynamically pass different prompts.

    For scribble-to-image, the Model Selection option provides three choices: Image-to-Image Advanced Edition, Image-to-Image Balanced Edition, and Image-to-Image Lightweight Edition.

  6. After configuration, click the publish button in the upper-right corner to publish (you must publish before you can test).

HTTP access

Request parameters

Parameter

Sub-parameter

Type

Required

Description

model

string

Yes

The name of the Alibaba Cloud Model Studio (Bailian) model. The value is fixed to "multimodal-dialog". Copy it directly.

input

directive

string

Yes

The directive name. Set this to Request.

app_id

string

Yes

Your application ID. You can find it on the "My Applications" page of the Multimodal Interaction Development Kit console. For more information, see Get an Application ID.

dialog_id

string

No

The dialog ID. If you omit this parameter, the server starts a new dialog and returns a dialog ID. The ID is a 36-character string, such as "12345678-1234-1234-1234-1234567890ab". To continue a previous conversation, pass the dialog_id returned by the server for that conversation. Image generation modes do not use context, so you can leave this empty.

text

string

Yes

The text to process. Note: in this use case, if there is no text, set it to "" (empty string).

parameters

client_info

object

Yes

For parameter details, see the parameters.client_info parameter table below.

images

list[]

No

The reference images. Required in image-to-image scenarios, not required for text-to-image. For parameter details, see the parameters.images parameter table.

biz_params

object

No

Configure this parameter as needed. For parameter details, see the parameters.biz_params parameter table below.

Parameters: parameters.client_info

Parameter

Sub-parameter

Type

Required

Description

user_id

string

Yes

The end user ID. Generate this ID based on your business rules. It enables customized features for different end users. Maximum length: 36 characters.

device

uuid

string

No

A globally unique ID for the client. Generate this ID and pass it to the SDK. Maximum length: 40 characters. An end user can have multiple devices, each with a different UUID but the same user_id.

Parameters: parameters.images

Parameter

Type

Required

Description

type

string

Yes

The type of the image. Two types are supported: base64/url.

value

string

Yes

The image content.

  • When type is base64, this is the base64 string of the image.

  • When type is url, this is the url of the image.

Parameters: parameters.biz_params

Parameter

Sub-parameter

Type

Required

Description

commands[i]

name

string

Yes

Indicates a direct pass-through to the agent. For this use case, the value is fixed to "agent_command". Copy it directly.

exec_params

object

Yes

Specifies the agent. This value is fixed for this use case. For details, see the commands.exec_params parameter description.

user_defined_params

image_to_image

object

No

User-defined agent parameters. image_to_image indicates the parameters passed to the image generation agent. For this use case, populate this only when passing custom prompt variables to the image generation agent.

Parameters: parameters.biz_params.commands.exec_params

Parameter

Sub-parameter

Type

Required

Description

exec_params

app_id

string

Yes

Specifies the agent. For this use case, the value is fixed to "image_to_image". Copy it directly.

intent

string

Yes

Specifies the action of the agent. For this use case, the value is fixed to "open_image_to_image". Copy it directly.

slots

list[]

Yes

The slot information required by the agent.

Parameters: parameters.biz_params.commands.exec_params.slots

name

string

Yes

The name of the slot.

norm_value

string

Yes

The value of the slot.

For the image generation agent, you must populate the function and size slots as shown in the following example.

{
      "slots": [
          {
            "name": "function",
            "norm_value" :"imageAssistant"
          },
          {
            "name":"size",
            "norm_value":"768*768"
          }
      ]
}

The function parameter specifies the image generation feature to use. Supported values:

  • textToImage: text-to-image
  • imageAssistant: Image Assistant
  • sketchToImage: Sketch-to-image

The size parameter specifies the resolution of the generated image. The supported resolution range depends on the selected feature and model. See the following table for supported resolutions.

Image generation mode

Model configuration

Supported resolutions

text-to-image

Lightweight Edition

The image width and height range is [200, 2048], and the aspect ratio range is [1:4, 4:1]. For example, 256x2048 is not allowed because the aspect ratio is 1:8.

Balanced Edition

Advanced Edition

Image resolution: total pixels between [1280*1280, 1440*1440]

Image aspect ratio: [1:4, 4:1]

Image Assistant/Sketch-to-image

Lightweight Edition

Image resolution: the width and height are both within [512, 4096] pixels.

Balanced Edition

Image resolution: the width and height are both within [512*512, 2048*2048] pixels.

Advanced Edition

Image resolution: total pixels within [768*768, 2048*2048] pixels.

Parameters: parameters.biz_params.user_defined_params.image_to_image.user_prompt_params

ImportantPopulate this parameter only when passing custom variables for prompts to the image generation agent.

In the prompt area of the Sketch-to-image configuration panel, click the {x} Custom Variable button in the upper-right corner to add custom variables. ${style} is a default, built-in variable.

Parameter

Sub-parameter

Type

Required

Description

user_prompt_params

positive

object

No

Custom variables for the prompt.

negative

object

Yes

Custom variables for the negative prompt.

"user_defined_params": {
    "image_to_image": {
        "user_prompt_params": {
            "positive": {
                "hair": "长头发",
                "name": "张三"
            },
            "negative": {
                "style": "低分辨率"
            }
        }
    }
}

Text-to-image request example

{
    "model": "multimodal-dialog",
    "input": {
        "directive": "Request",
        "app_id": "xxxxxx",
        "text": ""
    },
    "parameters": {
        "client_info": {
            "user_id": "test-251222",
            "device": {
                "uuid": "test-251222-123"
            }
        },
      "biz_params": {
        "commands": [
          {
            "name": "agent_command",
            "exec_params": {
              "app_id": "image_to_image",
              "intent": "open_image_to_image",
		"slots": [
		  {
		    "name": "function",
		    "norm_value" :"textToImage"
			},
		  {
		    "name":"size",
		    "norm_value":"768*768"
		        }
		]
            }
          }
        ],
	  "user_defined_params": {
		"image_to_image": {
                  "user_prompt_params":{
                    "positive":{
                       "hair": "长头发",
                       "name": "张三"
                      },
                    "negative":{
                       "style": "低分辨率"
                      }
                    }
		}
	  }
      }
    }
}

Text-to-image cURL request example

curl --location "https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation" \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer your_api_key' \
  --header 'X-DashScope-SSE: enable' \
  --data '{
    "model": "multimodal-dialog",
    "input": {
        "directive": "Request",
        "app_id": "xxxxxx",
        "text": ""
    },
    "parameters": {
        "client_info": {
            "user_id": "test-251222",
            "device": {
                "uuid": "test-251222-123"
            }
        },
        "biz_params": {
            "commands": [
                {
                    "name": "agent_command",
                    "exec_params": {
                        "app_id": "image_to_image",
                        "intent": "open_image_to_image",
                        "slots": [
                            {
                                "name": "function",
                                "norm_value": "textToImage"
                            },
                            {
                                "name": "size",
                                "norm_value": "768*768"
                            }
                        ]
                    }
                }
            ],
            "user_defined_params": {
                "image_to_image": {
                    "user_prompt_params": {
                        "positive": {
                            "hair": "长头发",
                            "name": "张三"
                        },
                        "negative": {
                            "style": "低分辨率"
                        }
                    }
                }
            }
        }
    }
}'

Important

  1. Replace your_api_key (API_KEY), app_id, and img_url in the request.
  2. Set the text field to an empty string (""). Otherwise, the request enters intent recognition.
  3. Pass user_defined_params only when you need to pass custom Agent prompt variables in image generation modes.

Image assistant request example

{
    "model": "multimodal-dialog",
    "input": {
        "directive": "Request",
        "app_id": "xxxxxx",
        "text": ""
    },
    "parameters": {
        "images": [
            {
                "type": "url",
                "value": "img_url"
            }
        ],
        "client_info": {
            "user_id": "test-251222",
            "device": {
                "uuid": "test-251222-123"
            }
        },
        "biz_params": {
            "commands": [
                {
                    "name": "agent_command",
                    "exec_params": {
                        "app_id": "image_to_image",
                        "intent": "open_image_to_image",
                        "slots": [
                            {
                                "name": "function",
                                "norm_value": "imageAssistant"
                            },
                            {
                                "name": "size",
                                "norm_value": "768*768"
                            }
                        ]
                    }
                }
            ],
            "user_defined_params": {
                "image_to_image": {
                    "user_prompt_params": {
                        "positive": {
                            "hair": "长头发",
                            "name": "张三"
                        },
                        "negative": {
                            "style": "低分辨率"
                        }
                    }
                }
            }
        }
    }
}

Image assistant cURL request example

curl --location "https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation" \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer your_api_key' \
  --header 'X-DashScope-SSE: enable' \
  --data '{
    "model": "multimodal-dialog",
    "input": {
        "directive": "Request",
        "app_id": "xxxxxx",
        "text": ""
    },
    "parameters": {
        "images": [
            {
                "type": "url",
                "value": "img_url"
            }
        ],
        "client_info": {
            "user_id": "test-251222",
            "device": {
                "uuid": "test-251222-123"
            }
        },
        "biz_params": {
            "commands": [
                {
                    "name": "agent_command",
                    "exec_params": {
                        "app_id": "image_to_image",
                        "intent": "open_image_to_image",
                        "slots": [
                            {
                                "name": "function",
                                "norm_value": "imageAssistant"
                            },
                            {
                                "name": "size",
                                "norm_value": "768*768"
                            }
                        ]
                    }
                }
            ],
            "user_defined_params": {
                "image_to_image": {
                    "user_prompt_params": {
                        "positive": {
                            "hair": "长头发",
                            "name": "张三"
                        },
                        "negative": {
                            "style": "低分辨率"
                        }
                    }
                }
            }
        }
    }
}'

Important

  1. Replace your_api_key with your API key, app_id with your app ID, and img_url with the image URL.
  2. Set the text field to an empty string (""). Otherwise, the request is routed for intent recognition.
  3. Use the user_defined_params parameter only when passing custom variables to the agent prompt for image generation.

Sketch-to-image request

Use the Image Assistant request example, changing the value of the function slot in parameters.biz_params.commands.exec_params.slots from imageAssistant to sketchToImage.

Text response events

Top-level parameter

Second-level parameter

Type

Required

Description

output

event

string

Yes

Event name: RespondingContent

dialog_id

string

Yes

A unique identifier for the dialog.

round_id

string

Yes

A unique identifier for the current round of interaction.

llm_request_id

string

Yes

The request ID for the call to the large language model (LLM).

text

string

Yes

The URL of the generated image. The value is RUNNING while generation is in progress and becomes the image URL after generation completes.

spoken

string

Yes

Content for Text-to-Speech (TTS) synthesis. Ignore this field if TTS is not enabled. In this use case, the content is the same as the text field.

finished

bool

Yes

Indicates whether the output stream has finished.

finish_reason

string

No

The reason for finishing. Currently, only one value is supported:

  • stop indicates normal completion

extra_info

object

No

Other extended information. Currently supported:

  • agent_info: agent information, see

Parameters for output.extra_info.agent_info

Top-level parameter

Second-level parameter

Type

Required

Description

round

string

Yes

The dialog round number.

device

device_id

string

Yes

The device.uuid used in the request.

intent_infos

intent

string

Yes

The agent used. For this use case, the value is fixed to "open_image_to_image".

domain

string

Yes

The agent used. For this use case, the value is fixed to "image_to_image".

Response example

{
    "output": {
        "round_id": "xxx",
        "llm_request_id": "xxx",
        "extra_info": {
            "agent_info": {
                "round": 1,
                "device": {
                    "device_id": "test-251222-123"
                },
                "intent_infos": [
                    {
                        "intent": "open_image_to_image",
                        "domain": "image_to_image"
                    }
                ]
            },
            "query": ""
        },
        "dialog_id": "6c4340eb-xxx-4055-8d66-0794df7986e0",
        "spoken": "img_url",
        "finished": true,
        "text": "img_url",
        "event": "RespondingContent"
    },
    "request_id": "xxx"
}

Important

  1. During image generation, the service sends a heartbeat every 3–5 seconds. While in progress, output.text is RUNNING and output.finished is false.
  2. When image generation completes, output.finished becomes true, and output.text contains the image URL.
id:1 合成过程中
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbbfe84a8xxx","llm_request_id":"f22feed889994756b3787de8b08bxxx","extra_info":{"tool_infos":[{"output":"","tool_name":"image_to_image","arguments":{},"type":"agent","success":true}]},"dialog_id":"00cd961c-bf58-49f5-b3bb-7c5ca9xxx","spoken":"","finished":false,"text":"","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fcb7xxx"}
id:2 合成过程中
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbbfe84xxx","llm_request_id":"f22feed889994756b3787de8b0xxx","extra_info":{"agent_info":{"round":1,"device":{"device_id":"xxx-test-251222-123"},"intent_infos":[{"intent":"open_image_to_image","domain":"image_to_image"}]},"query":""},"dialog_id":"00cd961c-bf58-49f5-b3bb-7c5xxx","spoken":"RUNNING","finished":false,"text":"RUNNING","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fcb7xxx"}
id:3 合成过程中
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbbfe84xxx","llm_request_id":"f22feed889994756b3787de8bxxx","extra_info":{"agent_info":{"round":1,"device":{"device_id":"xxx-test-251222-123"},"intent_infos":[{"intent":"open_image_to_image","domain":"image_to_image"}]},"query":""},"dialog_id":"00cd961c-bf58-49f5-b3bb-7c5cxxx","spoken":"RUNNING","finished":false,"text":"RUNNING","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fcb71xxx"}
id:4 合成过程中
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbb1xxx","llm_request_id":"f22feed889994756b3787xxx","extra_info":{"agent_info":{"round":1,"device":{"device_id":"xxx-test-251222-123"},"intent_infos":[{"intent":"open_image_to_image","domain":"image_to_image"}]},"query":""},"dialog_id":"00cd961c-bf58-49f5-b3bb-xxx","spoken":"RUNNING","finished":false,"text":"RUNNING","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fclxxx"}
id:5 合成结束
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbbfxxx","llm_request_id":"f22feed889994756b3787de8xxx","extra_info":{"tool_infos":[{"output":"","tool_name":"image_to_image","arguments":{},"type":"agent","success":true}],"agent_info":{"round":1,"device":{"device_id":"xxx-test-251222-123"},"intent_infos":[{"intent":"open_image_to_image","domain":"image_to_image"}]},"query":""},"finish_reason":"stop","dialog_id":"00cd961c-bf58-49f5-b3bb-7cxxx","spoken":"https://dashscope-result-sh.oss-cn-shanghai.aliyuncs.com/1d/03/20260107/c6e3cb4b/eb8e554d-66ff-44ca-ba0d-f05481405d2c-1-1.png?Expires=1767854752&OSSAccessKeyId=YOUR_ACCESS_KEY_ID&Signature=YOUR_SIGNATURE","finished":true,"text":"https://dashscope-result-sh.oss-cn-shanghai.aliyuncs.com/1d/03/20260107/c6e3cb4b/eb8e554d-66ff-44ca-ba0d-f05481405d2c-1-1.png?Expires=1767854752&OSSAccessKeyId=YOUR_ACCESS_KEY_ID&Signature=YOUR_SIGNATURE","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fcbxxx"}