Access MLLMs

Access self-developed MLLMs that comply with OpenAPI specifications

You can integrate MLLMs that comply with OpenAI specifications into your workflow. Only streaming mode is supported.

1. To integrate an MLLM into a workflow, set the Select Model parameter to Access Self-developed Model (Based on OpenAPI Specifications) and configure the following parameters in the configuration panel of the MLLM node.

   Parameter	Type	Required	Description	Example
   ModelId	String	Yes	The model name. Corresponds to the model field in the OpenAI specification.	abc
   API-KEY	String	Yes	The authentication information. Corresponds to the api_key field in the OpenAPI specification.	AUJH-pfnTNMPBm6iWXcJAcWsrscb5KYaLitQhHBLKrI
   Model URL (HTTPS)	String	Yes	The service request URL. Corresponds to the base_url field in the OpenAPI specification.	http://www.abc.com
   Maximum Number of Images per Call	Integer	Yes	Some MLLMs limit the number of image frames per request. Set this parameter to match the model limit. Video frames are automatically sampled based on this value.	15

2. At runtime, the workflow assembles the data into an OpenAI-compliant POST request and sends it to the HTTPS endpoint you configured. The following table describes the input parameters.

   Parameter	Type	Description	Example
   messages	Array	The conversation history. Up to 20 records can be retained. Records at the top of the array represent earlier messages. Note Only JPEG Base64-encoded data from extracted frames can be passed. Image data from previous conversations is not included as context.	[ { "role": "user", "content": "What is the weather like today?" }, { "role": "assistant", "content": "It is sunny today." }, { "role": "user", "content": "What will the weather be like tomorrow?" }, { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": "data:image/jpeg;base64,xxxx" } }, { "type": "text", "text": "What is this?" } ] } ]
   model	String	The model name.	abc
   stream	Boolean	Specifies whether to use streaming mode. Only streaming mode is supported.	True
   extendData	Object	Additional information.	{'instanceId':'68e00b6640e*****3e943332fee7','channelId':'123','userData':'{"aaaa":"bbbb"}'}
   instanceId	String	The instance ID.	68e00b6640e*****3e943332fee7
   channelId	String	The channel ID.	123
   userData	String	The UserData value passed when the instance was started.	{"aaaa":"bbbb"}