Model Studio PrivateLink private network endpoint VPC-Alibaba Cloud Model Studio(Model Studio)-阿里云帮助中心

To call Model Studio APIs from a VPC without routing traffic over the public internet, add a PrivateLink connection in the Model Studio console and associate it with your workspace.

How it works

The Model Studio gateway supports the following two access methods:

Public network access: Call Model Studio APIs over the public MaaS domain name {WorkspaceId}.{RegionId}.maas.aliyuncs.com. Traffic is routed over the public internet.
PrivateLink private connection: Add a PrivateLink connection for your VPC in the Model Studio console, and then call Model Studio APIs over the private domain name. Traffic stays within the Alibaba Cloud network and does not traverse the public internet.

After you add a PrivateLink connection in the Model Studio console, PrivateLink establishes a private connection (endpoint connection) between your VPC and Model Studio. The connection is unidirectional: resources in your VPC can access Model Studio, but Model Studio cannot access your VPC through this connection.

When resources in your VPC access the private domain name, PrivateLink routes traffic to Model Studio without traversing the public internet.

Model Studio private network access is supported in the following regions (the region drop-down list in the console is authoritative):

China (Beijing)
China (Shanghai)
China (Hangzhou)
China (Shenzhen)
China (Ulanqab)
China (Zhangjiakou)

Access Model Studio APIs over a PrivateLink connection

Step 1: Add a PrivateLink connection in the Model Studio console

Log on to the Model Studio console. Click Settings in the upper-right corner of the page and select Network configuration.
The first time you use this feature, complete the following preparations in the Add PrivateLink panel:
- Authorize service-linked role for PrivateLink: Click Authorize Now, and then click Confirm Authorization to create the service-linked role AliyunServiceRoleForBackrouterAccessNet.
- Activate PrivateLink: Click Enable Now to activate the PrivateLink service.
In the VPC private network access section, click Add and configure the following parameters. Leave the other parameters at their default values.
- Region: Select the region of your Model Studio service, for example, China (Beijing).
- VPC: Select the VPC from which you want to access Model Studio. The PrivateLink connection is created in this VPC, and only resources such as ECS instances and containers in this VPC can reach Model Studio through the private domain name. If no VPC is available, click Create VPC to create one in the VPC console. To use a VPC shared by another account, select the shared VPC. For more information, see Cross-account access.
- Zones and Switches: Select a zone that is supported by Model Studio and contains a vSwitch, and then select the vSwitch. The PrivateLink connection occupies one internal IP address of the selected vSwitch in each zone as the internal DNS resolution address of the private domain name. You can add up to two zone and vSwitch combinations. Add two for high availability so that traffic can fail over if one zone becomes unavailable. If no vSwitch is available, click Create Switch to create one in the VPC console.
- Security Group: Select the security group to associate with the PrivateLink connection. The security group controls which resources can access the connection. Make sure that the security group allows inbound traffic on port 443 (HTTPS). If no security group is available, click Create Security Group to create one in the ECS console.
Click Confirm and wait until the Status of the connection changes to Created. The connection takes effect in about 1 to 3 minutes. After it takes effect, any cloud server in the VPC can use the private domain name to access Model Studio.

Step 2: Associate the PrivateLink connection with a workspace

A workspace that needs to access Model Studio over the private network must be associated with a PrivateLink connection.

In the left-side navigation pane of the Model Studio console, choose Workspaces, and then create a workspace or edit the target workspace.
In Advanced Configuration, set PrivateLink to the target PrivateLink connection. Each option shows the endpoint ID with its region and VPC. If the target connection is not listed, click Create PrivateLink in the drop-down list to create one. After the association, the workspace can access Model Studio through the private connection of the corresponding VPC.
To remove the association, edit the workspace, clear PrivateLink, and save the changes. Alternatively, on the Network configuration page, expand the row of the target PrivateLink connection and click Unbind in the Associated workspace column to go to the corresponding workspace. A PrivateLink connection that is associated with workspaces cannot be deleted until it is unbound.

Step 3: Get the private domain name

After you complete Step 2, a private domain name is generated for each associated workspace. On the Network configuration page, expand the row of the target PrivateLink connection in the VPC private network access list and copy the private domain name in the Endpoint column. The Associated workspace column shows the ID of the workspace to which the domain name belongs. The private domain name is in the format of {WorkspaceId}-{VpcId}.{region-id}.maas.aliyuncs.com, where WorkspaceId is the workspace ID and VpcId is the ID of the VPC to which the PrivateLink connection belongs.

You can also find the private domain name in the API Host column of the target workspace on the Workspaces page. After the association, this column lists both the public and the private domain name.

The private domain name can be resolved and accessed only within the selected VPC. Use HTTPS (port 443) to call Model Studio APIs. If no workspace is associated, the Endpoint column in the expanded row is empty.

Step 4: Verify the connection

Replace the domain name in the base_url of your API request with the private domain name from the previous step, and then make the call from within the VPC.

For example, to call the Qwen text model in OpenAI-compatible mode:

Before replacement: https://{WorkspaceId}.{RegionId}.maas.aliyuncs.com/compatible-mode/v1/chat/completions
After replacement: https://{WorkspaceId}-{VpcId}.{RegionId}.maas.aliyuncs.com/compatible-mode/v1/chat/completions

Example call:

HTTP

# Replace the original domain name with the private domain name from the previous step.
curl -X POST https://{WorkspaceId}-{VpcId}.{RegionId}.maas.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "qwen-flash",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "Who are you?"
        }
    ]
}'

OpenAI Python SDK

import os
from openai import OpenAI
client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Replace the original domain name with the private domain name from the previous step.
    base_url="https://{WorkspaceId}-{VpcId}.{RegionId}.maas.aliyuncs.com/compatible-mode/v1",
)
completion = client.chat.completions.create(
    model="qwen-flash",
    messages=[
        {'role': 'system', 'content': 'You are a helpful assistant.'},
        {'role': 'user', 'content': 'Who are you?'}],
)
print(completion.model_dump_json())

DashScope Python SDK

import os
# DashScope SDK version 2.0.1 or later is recommended.
import dashscope
from dashscope.v2 import DashScopeError
# Configure the private domain name: explicitly set base_url, replacing the original domain name with the private domain name from the previous step.
client = dashscope.Client(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    base_url="https://{WorkspaceId}-{VpcId}.{RegionId}.maas.aliyuncs.com/api/v1",
)
messages = [{
    'role': 'user', 'content': 'Who are you?'
}]
try:
    response = client.chat.completions.create(
        model="qwen-flash",
        messages=messages,
    )
    print(response)
except DashScopeError as e:
    print('Request id: %s, HTTP status code: %s, error code: %s, error message: %s' % (
        e.request_id, e.status_code, e.code, e.message
    ))
    print('For more information, see the documentation at: https://help.aliyun.com/en/model-studio/developer-reference/error-code')

DashScope Java SDK

// DashScope SDK version 2.12.0 or later is recommended.
import java.util.Arrays;
import com.alibaba.dashscope.aigc.generation.Generation;
import com.alibaba.dashscope.aigc.generation.GenerationParam;
import com.alibaba.dashscope.aigc.generation.GenerationResult;
import com.alibaba.dashscope.common.Message;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.InputRequiredException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.protocol.Protocol;
import com.alibaba.dashscope.utils.JsonUtils;
public class Main {
    public static GenerationResult callWithMessage() throws ApiException, NoApiKeyException, InputRequiredException {
        // Replace the original domain name with the private domain name from the previous step.
        Generation gen = new Generation(Protocol.HTTP.getValue(), "https://{WorkspaceId}-{VpcId}.{RegionId}.maas.aliyuncs.com/api/v1");
        Message systemMsg = Message.builder()
                .role(Role.SYSTEM.getValue())
                .content("You are a helpful assistant.")
                .build();
        Message userMsg = Message.builder()
                .role(Role.USER.getValue())
                .content("Who are you?")
                .build();
        GenerationParam param = GenerationParam.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .model("qwen-flash")
                .messages(Arrays.asList(systemMsg, userMsg))
                .resultFormat(GenerationParam.ResultFormat.MESSAGE)
                .build();
        return gen.call(param);
    }
    public static void main(String[] args) {
        try {
            GenerationResult result = callWithMessage();
            System.out.println(JsonUtils.toJson(result));
        } catch (ApiException | NoApiKeyException | InputRequiredException e) {
            // Print the error message.
            System.err.println("An error occurred while calling the generation service: " + e.getMessage());
        }
    }
}

Before you make the call, make sure that you have obtained and configured an API key. To pass the API key directly, replace $DASHSCOPE_API_KEY with your API key.

Cross-account access

By default, a PrivateLink connection can be created only in a VPC that belongs to the current account, and only resources in that VPC can access Model Studio through the private domain name. If the VPC that contains your calling resources belongs to another Alibaba Cloud account, use either of the following methods:

Method 1: VPC sharing: Share the vSwitch in the caller's VPC with the Model Studio account, and create the PrivateLink connection directly in the caller's VPC.
Method 2: VPC interconnection: Create the PrivateLink connection in a VPC that belongs to the Model Studio account. The caller's VPC is connected to that VPC through a VPC peering connection or Cloud Enterprise Network (CEN) and then accesses the PrivateLink connection.

Neither method requires you to share the PrivateLink connection itself between accounts. If the caller is not the Model Studio account, the Model Studio account provides the private domain name and the workspace API key to the caller.

VPC sharing involves two accounts:

VPC owner account: Owns the VPC and vSwitch, and shares the vSwitch.
Model Studio account: Accepts the share, adds the PrivateLink connection in the Model Studio console, and associates the workspace.

The PrivateLink connection is created in the shared VPC, and resources in that VPC that belong to any account can use the connection. Alternatively, the VPC owner account can create a PrivateLink connection in the VPC itself for other accounts in the shared VPC to use. In either case, the PrivateLink fees are billed to the account that creates the PrivateLink connection. The following steps use the Model Studio account creating the PrivateLink connection as an example.

Step 1: The VPC owner account shares the vSwitch

In the Resource Sharing console, click Create Sharing Unit and complete Configure Basic Information and Add Resources (add the vSwitch to share), Associate Permissions (select AliyunRSDefaultPermissionVSwitch), Associate Resource Users (enter the UID of the Model Studio account), and Confirm and Submit. VPC sharing shares a vSwitch, not the entire VPC. For more information, see Share a VPC.

Sign in to the Resource Sharing console with the Model Studio account and accept the share on the Shared with Me page. If the sharing unit is already associated, skip this step.

Step 3: The Model Studio account adds a PrivateLink connection

Sign in to the Model Studio console and add a PrivateLink connection as described in Step 1 of this topic. Note the following settings:

Region: Select the region where the shared VPC resides. The region must support Model Studio private access.
VPC: Select the shared VPC.
Zones and Switches: Select the shared vSwitch in the shared VPC.
Security Group: Select an available security group in the shared VPC, and make sure that its inbound rules allow the calling resources to access port 443.

Other parameters are the same as those for the same-account flow.

Step 4: Complete the remaining configuration and make calls

After the PrivateLink connection is created, associate the workspace and obtain the private domain name as described in Step 2, Step 3, and Step 4 of this topic. The caller in the shared VPC uses that private domain name and the workspace API key to make calls.

Method 2: VPC interconnection

VPC interconnection involves two accounts:

Model Studio account: Creates a PrivateLink connection in its own VPC (the Model Studio VPC) and associates the workspace.
Caller account: Owns the VPC that contains the calling resources (the caller VPC).

After the VPCs are interconnected, the network between them is bidirectional. Either account can create a PrivateLink connection in its own VPC for the other account to call, which means either account can act as the Model Studio account.

Step 1: Connect the networks

Connect the Model Studio VPC and the caller VPC by using either of the following methods. The CIDR blocks of the two VPCs must not overlap. For a comparison of the two methods in terms of selection and billing, see Overview of VPC interconnection.

VPC peering connection
1. Either account creates a peering connection in the VPC console, sets Accepter Account Type to Cross-account, and enters the UID of the peer account and the ID of the peer VPC.
2. The peer account accepts the connection on the VPC peering connection page of the VPC console.
3. Each account adds a route to the route table of its own VPC. The destination CIDR block is the CIDR block of the peer VPC, and the next hop is the peering connection.
For more information, see VPC peering connections.
Cloud Enterprise Network
1. Either account creates a CEN instance and creates a transit router in the region where the VPCs reside.
2. The peer account grants permissions on its VPC to the CEN instance.
3. Create a VPC connection for each of the two VPCs on the transit router to attach both VPCs to the same transit router.
For more information, see Use Enterprise Edition transit routers to connect VPCs across regions and accounts.

Step 2: Allow access in the PrivateLink security group

After the networks are connected, the caller must still be allowed by the security group of the PrivateLink connection to access the connection. The Model Studio account adds the following inbound rule to the security group associated with the PrivateLink connection:

Parameter	Value
Action	Allow
Protocol Type	Custom TCP
Port Range	443
Authorization Object	The CIDR block of the caller VPC, for example, `192.168.0.0/16`. You can view it on the VPC details page in the VPC console

If this rule is not added, calls time out even if the networks are connected.

Step 3: Make calls

The Model Studio account provides the private domain name and the workspace API key to the caller account. Resources in the caller VPC make calls as described in Step 4 of this topic. If a connection timeout error such as Failed to connect to <private domain name> port 443 is returned, check the network configuration in Step 1 and the security group rule in Step 2.

Billing

Using PrivateLink incurs additional fees. For more information, see PrivateLink billing.

PrivateLink fees and model invocation fees are billed to the account that creates the PrivateLink connection, regardless of which account owns the VPC. For VPC peering connection and CEN billing, see VPC peering connections and CEN billing rules.

FAQ

Why can't my ECS instance access Model Studio APIs over the PrivateLink connection?
Troubleshoot as follows:
1. Check whether the ECS instance is in the same VPC as the PrivateLink connection.
  
  If the ECS instance is in a different VPC, it cannot access Model Studio over the private network. You must first configure VPC interconnection, for example, a VPC peering connection or CEN, and allow access in the PrivateLink security group. For more information, see Method 2: VPC interconnection.
2. Check the security group associated with the PrivateLink connection and make sure that an inbound rule allows traffic from the CIDR block of the ECS instance on port 443 (HTTPS).
3. Check the status of the PrivateLink connection.
  
  The private domain name can be used only after the connection status is Created and the connection has taken effect (about 1 to 3 minutes after creation).
4. Check whether the workspace is associated.
  
  A private domain name is generated only after a workspace is associated with the PrivateLink connection. If the Endpoint column in the expanded row is empty, associate the workspace as described in Step 2.
Can an endpoint be accessed from the public internet?
No. PrivateLink establishes private connections only within the Alibaba Cloud network. Endpoints cannot be accessed from the public internet, and endpoint ENIs cannot be associated with elastic IP addresses (EIPs).
Do I need to associate a workspace with a legacy PrivateLink connection?
No. A legacy PrivateLink connection, which was created as an interface endpoint in the PrivateLink (VPC) console, is associated with all workspaces by default. You can use it to access Model Studio APIs over the private network without performing Step 2.

When you expand a legacy connection on the Network configuration page, the Endpoint column shows vpc-{region-id}.dashscope.aliyuncs.com (for example, vpc-cn-beijing.dashscope.aliyuncs.com), and the Associated workspace column indicates that the connection is associated with all workspaces. This differs from the current flow, which lists {WorkspaceId}-{VpcId}.{region-id}.maas.aliyuncs.com for each associated workspace.

You can no longer create a PrivateLink connection in this way. To add one, follow Step 1 to create it in the Model Studio console, and then follow Step 2 to associate the workspaces that need private network access.