AI transformer plug-in

How it works

When a request arrives at the cloud-native gateway, the ai-transformer plug-in intercepts the traffic and routes it through the LLM for transformation:

Client ──> Gateway ──> LLM (request transform) ──> Upstream service
                                                         │
Client <── Gateway <── LLM (response transform) <────────┘

1. Request phase (optional): The gateway sends the incoming request headers and body to the configured LLM, using your request prompt as the system instruction. The LLM returns the transformed request, and the gateway forwards it to the upstream service.

2. Response phase (optional): After the upstream service responds, the gateway sends the response headers and body to the LLM, using your response prompt as the system instruction. The LLM returns the transformed response, and the gateway sends it back to the client.

You can enable one or both phases depending on your use case. Each enabled phase adds an LLM round-trip to the request pipeline.

Enabling both phases doubles the added latency because the gateway makes two separate LLM calls per request.

Use cases

Prerequisites

Before you begin, make sure that you have:

• An MSE cloud-native gateway instance

• An Alibaba Cloud DashScope API key (get one from the DashScope console)

Execution attributes

Configuration parameters

Convert XML responses to JSON

This example transforms XML responses from an httpbin backend into JSON. Request-phase transformation is disabled.

request:
    enable: false
    prompt: "If the request path starts with /httpbin, remove the /httpbin prefix. If the request path does not start with /httpbin, do not modify the request path."
response:
    enable: true
    prompt: "Modify the HTTP response based on the following requirements: 1. Change content-type to application/json. 2. Convert the body from XML to JSON. 3. Remove content-length."
provider:
    serviceName: qwen
    domain: dashscope.aliyuncs.com
    apiKey: xxxxxxxxxxxxx

<?xml version='1.0' encoding='us-ascii'?>

<!--  A SAMPLE set of slides  -->

<slideshow
    title="Sample Slide Show"
    date="Date of publication"
    author="Yours Truly"
    >

    <!-- TITLE SLIDE -->
    <slide type="all">
      <title>Wake up to WonderWidgets!</title>
    </slide>

    <!-- OVERVIEW -->
    <slide type="all">
        <title>Overview</title>
        <item>Why <em>WonderWidgets</em> are great</item>
        <item/>
        <item>Who <em>buys</em> WonderWidgets</item>
    </slide>

</slideshow>

{
  "slideshow": {
    "title": "Sample Slide Show",
    "date": "Date of publication",
    "author": "Yours Truly",
    "slides": [
      {
        "type": "all",
        "title": "Wake up to WonderWidgets!"
      },
      {
        "type": "all",
        "title": "Overview",
        "items": [
          "Why <em>WonderWidgets</em> are great",
          "",
          "Who <em>buys</em> WonderWidgets"
        ]
      }
    ]
  }
}

Write effective prompts

Transformation accuracy depends on prompt clarity. Follow these guidelines to get consistent results:

Vague prompts produce unpredictable results. Be as explicit as possible about the expected input format, transformation logic, and output format.

Considerations for production

• Latency overhead. Each enabled phase adds an LLM round-trip. Enabling both request and response transformation doubles the added latency. Evaluate whether the transformation benefit justifies the latency cost for latency-sensitive workloads.

• LLM output variability. LLMs may produce inconsistent, malformed, or unexpected output. Test prompts thoroughly before deploying to production. Monitor transformation results to detect regressions.

• Provider support. Only Qwen (via Alibaba Cloud DashScope at dashscope.aliyuncs.com) is supported. No other LLM providers or custom endpoints are available.

• Request size. Large request or response bodies increase LLM processing time and may exceed model token limits. Consider body size when writing prompts for transformation.