Integrate a Model Studio workflow

Connect directly to the workflow

When you connect to the service, you can configure parameters to establish a direct connection to the workflow. This forwards requests directly to the Model Studio workflow.

For more information, see Directly call Model Studio and third-party Agents.

Pass-through parameters

To pass arguments to a custom variable in the workflow, use the Start or UpdateInfo commands to pass through and update workflow parameters.

parameters.bizParams.user_defined_params

"user_defined_params":{
    "xxxxx":{
        "name": "Little Bear Cafeteria"
    }
}

Example using the Android SDK

private void updateParams(){
    try {
        JSONObject agentIdParams = new JSONObject();
        agentIdParams.put("name","Little Bear Cafeteria");

        JSONObject userDefinedParams = new JSONObject();
        userDefinedParams.put("2009506107285893120", agentIdParams); // The key is the app_id.


        MultiModalRequestParam updateParams = MultiModalRequestParam
                .builder()
                .bizParams(MultiModalRequestParam.BizParams.builder()
                        .userDefinedParams(userDefinedParams)
                        .build())
                .build();

        Log.i(TAG, "bizParams: " + updateParams.getParametersAsJson());
        multiModalDialog.updateInfo(updateParams.getParametersAsJson());
    } catch (JSONException e) {
        throw new RuntimeException(e);
    }
}

Configure incremental output

To return workflow results incrementally, configure the following settings.

Enable streaming output for the workflow

Configure multimodal incremental output

In the Start message, configure the service to return intermediate results from the dialog system and enable incremental delivery.

parameters.downstream

"parameters":{
  "downstream":{
    "intermediate_text": "transcript,dialog",
    "incremental_response": true
  }
}

Example using the Android SDK

private MultiModalRequestParam buildRequestParams() {
    Map<String,Object> map = new HashMap<>();
    map.put("incremental_response", true);

    return MultiModalRequestParam.builder()
            .clientInfo(MultiModalRequestParam.ClientInfo.builder()
                    .device(MultiModalRequestParam.ClientInfo.Device.builder()
                            .uuid("uuid_12345").build()) // Set this to your device UUID.
                    .userId("your_user_id")  // The userId must be unique for each user. We recommend using the device UUID. This ID is used to track conversation history.
                    .build())
            .upStream(MultiModalRequestParam.UpStream.builder()
                    .type("AudioAndVideo")
                    .build())
            .downStream(MultiModalRequestParam.DownStream.builder()
                    .voice("longanhuan") // The TTS voice model must match the model configured on the console. For example, longxiaochun_v2 corresponds to cosyvoice_v2.
                    .sampleRate(48000)
                    .intermediateText("transcript,dialog")
                    .passThroughParams(map)
                    .build())
            .build();
}

Response timeout

Error message: "SOCKET_TIMEOUT_EXCEPTION: Read timed out"

Check the processing time of your workflow. An error occurs if the workflow does not return a result within the time limit specified for the multimodal application. The current time limit is 10 seconds. Optimize your workflow's processing path. If timeouts persist, contact technical support for assistance.

Avoid JSON output

The multimodal application does not perform any extra processing on the workflow's output. Therefore, ensure the workflow returns output as plain text. If the output is in JSON format, the symbols in the JSON string are synthesized into speech and read aloud.

Avoid Markdown output

Output in Markdown format can interfere with audio synthesis, so you should avoid this format whenever possible. While you can add restrictive prompts to the multimodal application's prompt, you must also apply them within the workflow's prompt. Otherwise, the workflow might still generate Markdown-formatted content.