Model API supports custom routes for fine-grained traffic control based on request characteristics.
Create a route
Log on to the AI Gateway console and choose Instance. In the top menu bar, select a region, then click the target instance ID.
In the navigation pane on the left, choose Model API, then click the target API name to go to the API Details page.
Click the Routes tab, and then click Create Route. In the Create Route panel, configure the following parameters:
Route Name: A custom name for the route.
Path: The route path for the Model API. Supported paths include:
/v1/chat/completions,/v1/completions,/v1/embeddings, and/v1/fine-tuning.More Match Rules: Configure the Method, Header, and Query Parameters to differentiate multiple routes that share the same path.
Service Model: Can be Single-model Service or Multiple services (by proportion).
Single-model Service: Deploys and runs a single model in one service instance. This is suitable for applications that focus on a single task or model.
Multiple services (by proportion): Runs multiple models in the same service instance, with resources and requests allocated based on specified ratios. This option is ideal for applications that handle multiple tasks or models simultaneously.
Services: The backend service that this route forwards requests to.
Fallback: When enabled, you can add a fallback service. Services are attempted in descending order of priority.
NoteYou can use a single service as the fallback for multiple routes.
Confirm the configuration and click Add.
Edit a route
Log on to the AI Gateway console and choose Instance. In the top menu bar, select a region, then click the target instance ID.
In the navigation pane on the left, choose Model API, then click the target API name to go to the API Details page.
Click the Routes tab, and from the route list, select the route that you want to edit.
Click Edit Route and modify the parameters. For parameter details, see Create a route.
Confirm your changes and click OK.
Publish a route
Log on to the AI Gateway console and choose Instance. In the top menu bar, select a region, then click the target instance ID.
In the navigation pane on the left, choose Model API, then click the target API name to go to the API Details page.
Click the Routes tab, and from the route list, select the route that you want to edit.
Click Publish Route, and then click OK in the confirmation dialog box.
Unpublish a route
Log on to the AI Gateway console and choose Instance. In the top menu bar, select a region, then click the target instance ID.
In the navigation pane on the left, choose Model API, then click the target API name to go to the API Details page.
Click the Routes tab, and from the route list, select the route that you want to edit.
Click , and then click Shutdown in the confirmation dialog box.
Delete a route
You can only delete unpublished routes. To delete a published route, you must first unpublish it.
Log on to the AI Gateway console and choose Instance. In the top menu bar, select a region, then click the target instance ID.
In the navigation pane on the left, choose Model API, then click the target API name to go to the API Details page.
Click the Routes tab, and from the route list, select the route that you want to edit.
Click , and then click Delete in the confirmation dialog box.