Model APIs

1. Log on to the AI Gateway console and choose Instance. In the top menu bar, select a region, then click the target instance ID.

2. In the navigation pane on the left, choose Model API, then click Create Model API.

3. Select a use case and click Create.

   The use case determines the available Protocol options and default routes. Supported use cases:

   • Text Generation: Supports OpenAI-compatible and Anthropic protocols.

   • Image Generation

   • Video Generation

   • Speech Synthesis

   • Embedding

   • Rerank

   • Others

4. Configure basic information.

   In the dialog, complete the Select a scenario. step, then configure the Create Model API form:

   • Protocol: Each protocol provides a set of built-in default routes for the selected use case, which allows you to quickly generate compatible interfaces for services such as OpenAI, DashScope, and vLLM.

     Note

     Protocol conversion may change the Token statistics structure. For example, the input token statistics for the Alibaba Cloud Model Studio (DashScope) protocol include cache tokens, while the input token statistics for the Anthropic protocol do not include cache tokens. Pay attention to the differences in statistical metrics across protocols when viewing observability data.

   • API Name: A unique name within your account. Up to 64 characters; supports letters, digits, underscores (_), and hyphens (-).

   • Domain Name: Select one or more domain names for API access. Each domain name + BasePath combination must be unique.

     If you do not have a domain name, click the Add Domain Name button to create one.

   • base path: The base request path of the API. Defaults to /. Optionally enable Remove during backend forwarding.

     Note

     If you enable Remove during backend forwarding, the system strips the base path from the request before forwarding to the backend. For example:

     • The base path is set to /api.

     • The original request path is /api/users.

     • The path forwarded to the backend service becomes /users.

   • AI Request Monitoring: Enables metrics, logs, and traces. Logging and tracing require SLS. Select Record request content and Log response to record model requests and responses.

     Important

     When enabled, all AI request content including the request body is recorded to the access log. Ensure SLS is properly configured and data security safeguards are in place.

   • Model Service: Supports Single-model Service, Multi-model Service (by model name), Multi-model Service (by proportion), Multiple Services (by observability metrics), and Multiple Services (intelligent routing).

     • Single-model Service: Select one AI service and set the Model Name. The model name can be passed through or rewritten.

     • Multi-model Service (by model name): Routes requests to different services by matching the model name in the request body with a rule. The rule supports wildcards such as ? and *. For example, qwen-* can match qwen-max and qwen-long.

     • Multi-model Service (by proportion): Select multiple AI services and set their weights. The model name can be passed through or rewritten.

     • Multi-Service (by Metrics): Automatically routes requests to the optimal service based on observability metrics such as response time and success rate, without manual weight configuration.

     • Multi-Model Service (Smart Routing): The system automatically selects the most suitable model based on model characteristics. Intelligent Routing.

       Note

       To use the Multiple Services (by observability metrics) and Multiple Services (intelligent routing) features, you must upgrade AI Gateway to version 2.1.15 or later.

   • fallback: You can Enable this feature to configure a sequence of fallback policies. The same service can be used in multiple policies.

   • first packet timeout: Maximum wait time (ms) for the first data packet in a streaming response. Set to 0 to disable.

   • resource: Select the default resource group, an existing one, or create a new one for grouping, authorizing, and monitoring resources.

     To create a new Resource Group, click Create Resource Group.

5. Review your configuration and click OK.

Text generation

Image generation

Video generation

Speech synthesis

Embedding

Text reranking (rerank)

Others