Unify access to multiple LLM services

1. Create an ALB Extensible Edition instance

1. Log on to the ALB console, select the China (Ulanqab) region, and click Create ALB.

2. On the buy page, complete the following configurations and click Create Now.

   • Region: China (Ulanqab) is selected by default.

   • Network Type: Select Internet.

   • VPC and Zone: Select the target VPC, select Ulanqab Zone A and Ulanqab Zone B, select available vSwitches, and select Automatically assign EIP.

   • IP Version: Select IPv4.

   • Edition: Select Extensible.

3. On the Confirm Order page, check the instance configuration and click Activate Now.

2. Create AI Service server groups

Create two AI Service server groups to connect to Alibaba Cloud Model Studio and DeepSeek respectively.

3. Create a listener

1. In the ALB console, click the target instance ID to go to the Instance Details page. Click the Listener tab and click Create Listener.

2. In the Configure Listener step, set Listener Protocol to HTTPS and Listener Port to 443. Then, click Next.

3. In the Configure SSL Certificate step, select the server certificate that matches your custom domain name and click Next.

4. In the Select Server Group step, select the AI Service type and the Alibaba Cloud Model Studio server group sgp-ai-qwen. Then, click Next.

   The server group you select here is used for the default rule of the listener. This rule processes requests that do not match any other forwarding rules. Adjust the settings based on your needs.

5. In the Configuration Review step, check the configuration and click Submit.

4. Configure forwarding rules

Requests are matched against forwarding rules in ascending order of priority. When a request matches a rule, the corresponding action is executed, and no subsequent rules are matched. Create forwarding rules with the AI Model condition to route requests to the appropriate backend AI service based on the model name in the request body.

Model Name is an optional parameter. The AI Model forwarding condition matches all requests with paths /v1/completions, /v1/chat/completions, or /v1/embeddings, and request content that conforms to the OpenAI-compatible protocol. If you specify a Model Name, the model field in the request body must also match the specified model name.

5. Configure DNS resolution

Point your custom domain name to the DNS name of the ALB instance using a canonical name (CNAME) record. Clients access ALB through your custom domain name.

This tutorial uses Alibaba Cloud DNS as an example. For domain names not registered with Alibaba Cloud, you must first add the domain name to the Alibaba Cloud DNS console.

1. In the ALB console, copy the Domain Name of the target instance.

2. Log on to the Alibaba Cloud DNS console. In the Actions column of the target domain name, click Settings. On the Settings page, click Add Record.

3. Add a CNAME record with the following information and click OK.

   • Record Type: Select CNAME.

   • Hostname: Enter a domain prefix such as ai. If your root domain is example.com, the domain for accessing ALB is ai.example.com.

   • Query Source and TTL: Keep the default values.

   • Record Value: Enter the DNS name of the ALB instance.

4. In the Change Resource Record Confirmation dialog box, check the DNS information and click OK.

6. Verify the configuration

• Use curl commands to verify that requests are correctly routed to the corresponding LLM services. Replace ai.example.com with your actual configured domain name. Make sure DNS resolution has taken effect.

• Client requests must conform to the OpenAI-compatible protocol. The request path must be /v1/completions, /v1/chat/completions, or /v1/embeddings, and the request body must contain a model field. Forwarding rules route requests based on the value of the model field.

curl -v \
    -H "Content-Type: application/json" \
    -d '{
        "model": "qwen-turbo",
        "messages": [
            {
                "role": "user", 
                "content": "Who are you"
            }
        ]
    }' \
    https://ai.example.com/v1/chat/completions

{
    "choices": [
        {
            "message": {
                "role": "assistant",
                "content": "I am Qwen, a large-scale language model developed by the Tongyi Lab of Alibaba Group. My Chinese name is Tongyi Qianwen, and my English name is Qwen. I can perform various tasks such as answering questions, creating text, logical reasoning, and programming. You can call me Qwen or Tongyi Qianwen. How can I help you?"
            },
            "finish_reason": "stop",
            "index": 0
        }
    ],
    "object": "chat.completion",
    "model": "qwen-turbo"
}

curl -v \
    -H "Content-Type: application/json" \
    -d '{
        "model": "deepseek-chat",
        "messages": [
            {
                "role": "user", 
                "content": "Who are you"
            }
        ]
    }' \
    https://ai.example.com/v1/chat/completions

{
    "object": "chat.completion",
    "model": "deepseek-chat",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "Hello! I am DeepSeek, an AI assistant created by DeepSeek company!\n\nI am a plain text model. Although I do not support multi-modal recognition, I have a file upload feature that can help you process images, txt, pdf, ppt, word, excel and other files, and read text information from them for analysis and processing.\n\nSome of my features:\n- Completely free to use, with no paid plans\n- Context length of up to 128K, allowing for very long conversations\n- Supports web search (needs to be enabled manually)\n- Can be used by downloading the app from the official app store\n- Knowledge cutoff is July 2024\n\nI am happy to help you answer questions, assist with thinking, process documents, or engage in various conversations! Is there anything I can help you with? ✨"
            },
            "logprobs": null,
            "finish_reason": "stop"
        }
    ]
}

Billing

• ALB Extensible Edition: Currently in public preview and free to use.

• Public network access fees: Internet NAT Gateway charges instance fees and Capacity Unit (CU) fees. The EIPs bound to the NAT gateway and the ALB Extensible Edition instance have their own billing rules, and fees are charged by EIP.

• Domain name and DNS resolution fees: In addition to domain registration fees charged by your domain provider, configuring DNS resolution on Alibaba Cloud requires paying public authoritative DNS resolution fees.

• Certificate fees: Purchasing certificates from Alibaba Cloud or uploading certificates to Alibaba Cloud requires paying server certificate fees.

• Model fees: Calling the LLM APIs of Alibaba Cloud Model Studio and DeepSeek incurs charges.

Regions where ALB Extensible Edition is available