Access Model Studio APIs over a private network by using an endpoint

How it works

After you create an interface endpoint in a VPC, PrivateLink establishes a private endpoint connection between your VPC and Model Studio. This connection is unidirectional: it allows resources in your VPC to access Model Studio, but Model Studio cannot use this connection to access resources in your VPC.

When compute resources in your VPC access the endpoint, traffic is routed to the Model Studio service through PrivateLink and bypasses the public internet.

To access the service privately from a VPC in a different region, see Access Model Studio APIs privately across regions.

Model Studio is available in the following regions:

• Public cloud: China (Beijing) and Singapore.

  Private network access is not currently supported in the US (Virginia) region.

Access Model Studio APIs with an endpoint

# Replace the original domain name with the endpoint service domain name from the previous step.
curl -X POST http://ep-***.dashscope.cn-beijing.privatelink.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "qwen-flash",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user", 
            "content": "Who are you?"
        }
    ]
}'

import os
from openai import OpenAI
client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Replace the original domain name with the endpoint service domain name from the previous step.
    base_url="http://ep-***.dashscope.cn-beijing.privatelink.aliyuncs.com/compatible-mode/v1",
)
completion = client.chat.completions.create(
    model="qwen-flash",
    messages=[
        {'role': 'system', 'content': 'You are a helpful assistant.'},
        {'role': 'user', 'content': 'Who are you?'}],
)
print(completion.model_dump_json())

import os
from http import HTTPStatus
# We recommend that you use DashScope SDK v1.14.0 or later.
import dashscope
from dashscope import Generation
# Replace the original domain name with the endpoint service domain name from the previous step.
dashscope.base_http_api_url = "http://ep-***.dashscope.cn-beijing.privatelink.aliyuncs.com/api/v1"
dashscope.api_key = os.getenv("DASHSCOPE_API_KEY")
messages = [{
    'role': 'user', 'content': 'Who are you?'
}]
response = Generation.call(
    model="qwen-flash",
    messages=messages,
    result_format='message'
)
if response.status_code == HTTPStatus.OK:
    print(response)
else:
    print('Request id: %s, Status code: %s, error code: %s, error message: %s' % (
        response.request_id, response.status_code,
        response.code, response.message
    ))

// We recommend that you use DashScope SDK v2.12.0 or later.
import java.util.Arrays;
import com.alibaba.dashscope.aigc.generation.Generation;
import com.alibaba.dashscope.aigc.generation.GenerationParam;
import com.alibaba.dashscope.aigc.generation.GenerationResult;
import com.alibaba.dashscope.common.Message;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.InputRequiredException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.protocol.Protocol;
import com.alibaba.dashscope.utils.JsonUtils;
public class Main {
    public static GenerationResult callWithMessage() throws ApiException, NoApiKeyException, InputRequiredException {
        // Replace the original domain name with the endpoint service domain name from the previous step.
        Generation gen = new Generation(Protocol.HTTP.getValue(), "http://ep-***.dashscope.cn-beijing.privatelink.aliyuncs.com/api/v1");
        Message systemMsg = Message.builder()
                .role(Role.SYSTEM.getValue())
                .content("You are a helpful assistant.")
                .build();
        Message userMsg = Message.builder()
                .role(Role.USER.getValue())
                .content("Who are you?")
                .build();
        GenerationParam param = GenerationParam.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .model("qwen-flash")
                .messages(Arrays.asList(systemMsg, userMsg))
                .resultFormat(GenerationParam.ResultFormat.MESSAGE)
                .build();
        return gen.call(param);
    }
    public static void main(String[] args) {
        try {
            GenerationResult result = callWithMessage();
            System.out.println(JsonUtils.toJson(result));
        } catch (ApiException | NoApiKeyException | InputRequiredException e) {
            // Print the error message.
            System.err.println("An error occurred while calling the generation service: " + e.getMessage());
        }
    }
}

Cross-region private access

Model Studio is deployed in China (Beijing) and Singapore. To access Model Studio APIs over a private network from a VPC in another region, choose one of the following methods based on your connection requirements:

• For access between regions that are either both in mainland China or both outside mainland China (for example, a VPC in China (Hangzhou) accessing Model Studio in China (Beijing)): use Method 1: Enable a cross-region endpoint.

• For cross-border access between the Chinese mainland and other regions (for example, a VPC in China (Beijing) accessing Model Studio in Singapore): use Method 2: Connect VPCs across regions by using CEN.

Method 1: Cross-region endpoint (recommended)

For more information about the procedure, see Access a cross-region Alibaba Cloud service by using an interface endpoint. The key configuration settings for Model Studio are:

• Region: Select the region of your source VPC.

• Type: Select Alibaba Cloud Service.

• Service Region: Select the Enable cross-region endpoint check box and select China (Beijing) or Singapore.

• Endpoint Service: Select com.aliyuncs.dashscope from the list of services.

• Cross-region configuration: Cross-region traffic fees are billed through Cloud Data Transfer (CDT). The bandwidth limit varies based on the connection range. The default bandwidth is 1,000 Mbit/s for connections between regions in the Chinese mainland and 100 Mbit/s for connections between regions outside the Chinese mainland.

The other parameters are the same as those for same-region configurations. After you complete the configuration, add an inbound rule to the security group that is associated with the endpoint to allow traffic from resources in the source VPC on ports 80 and 443.

Method 1 does not support connections between the Chinese mainland and other regions. For cross-border access, use Method 2.

After the configuration is complete, when you access the default service domain name of the endpoint from the source VPC, PrivateLink directly routes traffic to the region where the Model Studio service is deployed. This enables private cross-region access.

Method 2: Cross-region VPC connection with CEN (for cross-border scenarios)

This method requires that the endpoint be in the same region as the Model Studio service. You can use Cloud Enterprise Network (CEN) to connect VPCs across regions.

1. Complete the configurations in Access Model Studio APIs by using an endpoint.

2. Use Cloud Enterprise Network (CEN) to configure a cross-region VPC connection. Note the following:

   • Select VPCs that have different CIDR blocks to avoid connection failures due to network conflicts.

   • To establish a cross-region VPC connection between the Chinese mainland and other regions by using CEN, you must submit an application as prompted in the console. The approval process typically takes one to two business days. For more information, see Cross-border connection FAQ.

3. In the security group that is associated with the endpoint, add an inbound rule to allow traffic from resources in the source VPC on ports 80 and 443.

After the configuration is complete, when you access the default service domain name of the endpoint from the source VPC, a Transit Router (TR) routes traffic to the endpoint in the region where the Model Studio service is deployed. This enables private cross-region access to Model Studio APIs.

By default, the default domain name of an endpoint is accessible from an interconnected VPC in another region. However, a custom domain name is valid only within the endpoint's VPC. To access Model Studio APIs from the source VPC over a private network by using a custom domain name, see Quick start with Private DNS. Create a private domain name that matches the custom domain name, then create a CNAME record that resolves it to the endpoint default domain name:

1. Add a private authoritative domain name that is the same as the custom domain name, such as vpc-cn-beijing.dashscope.aliyuncs.com. Set its effective scope to the source VPC.

2. Add a DNS record: set Record Type to CNAME, set Host Record to @, and set Record Value to the default domain name of the target endpoint, such as ep-***.dashscope.cn-beijing.privatelink.aliyuncs.com.

   Note: When you configure Private DNS, do not use underscores (_) in the host record or full domain name. Otherwise, API calls may fail. Domain names should contain only letters, numbers, and hyphens (-), for example, test-for-dns.dashscope.aliyuncs.com, not test_for_dns.dashscope.aliyuncs.com.

After the configuration is complete, you can access Model Studio APIs from the source VPC by using the custom domain name. If you use a private domain name that is different from the custom domain name, see Configure Private DNS.

Billing

Using PrivateLink and Private DNS incurs additional costs. Cross-border scenarios also incur Cloud Enterprise Network (CEN) cross-region fees. For cost estimates, see the following billing documents:

• Billing of PrivateLink

• Billing of CEN (for cross-border scenarios only)

• Billing of Private DNS

FAQ

1. Why can't my ECS instance access Model Studio APIs over a private network?

   Check the following items:

   1. Confirm that the resources are in the same VPC.

      If the ECS instance and the endpoint are in different VPCs, you cannot access Model Studio APIs over the private network. In this case, you must first configure VPC connections.

   2. Check the security group that is associated with the endpoint. Confirm that an inbound rule is added to allow traffic from the CIDR block of the source ECS instance on port 80 (HTTP) or 443 (HTTPS).

   3. Check that you are using the correct endpoint service domain name.

      When you access the Model Studio platform over a private network by using the default domain name, only HTTP is supported.

2. Can an endpoint be accessed from the public internet?

   No. PrivateLink only establishes private connections within the Alibaba Cloud internal network. Endpoints cannot be accessed from the public internet, and you cannot associate an Elastic IP Address (EIP) with an endpoint ENI.

3. Why do I get an error when I call a model when using a custom domain name with Private DNS?

   This issue usually occurs when the host record (or full domain name) that you configure in Private DNS contains invalid characters, such as an underscore (_). Domain names should contain only letters, digits, and hyphens (-).

   Configure the DNS record by following these steps:

   1. Authoritative Domain: In Private DNS, add a DNS record for the dashscope.aliyuncs.com authoritative domain.

   2. Host Record: Set Record Type to CNAME and enter your custom domain prefix, such as test-for-dns-right. Note: The host record cannot contain underscores (_).

      Correct example	Incorrect example
      On the Edit Record page, set Record Type to CNAME. For Host Record, use a hyphen (-) to separate words, for example, test-for-dns-right. The suffix is .dashscope.aliyuncs.com. For Record Value, enter the corresponding endpoint address.	On the Edit Record page, the Host Record field is test_for_dns_wrong. This uses an underscore (_) as a separator, which is invalid. DNS host records do not support underscores. You must use valid characters, such as hyphens (-).

   3. Record Value: Enter the default domain name of the Model Studio endpoint. Example: ep-***.dashscope.cn-beijing.privatelink.aliyuncs.com.

   After the configuration is complete, you can call the model at https://test-for-dns-right.dashscope.aliyuncs.com/api/v1. The endpoint for the OpenAI-compatible mode is https://test-for-dns-right.dashscope.aliyuncs.com/compatible-mode/v1/chat/completions.

   Using a domain name that contains an underscore, such as https://test_for_dns_wrong.dashscope.aliyuncs.com/api/v1, will cause the API call to fail.