Fully-managed dedicated gateway

Key features

Access control: Control public network and VPC access using an allowlist.
Custom domain access: Configure custom domains and certificates to expose services.
Cross-account VPC access: Enable servers in another account's same-region VPC to access EAS services through a VPC endpoint.
Authoritative DNS resolution: Resolve gateway domains authoritatively when accessing EAS from other clouds or on-premises data centers. Requires network connectivity to Alibaba Cloud.

Billing

Gateway fee: Supports pay-as-you-go and subscription billing. Billing for EAS.
Additional fees:
- VPC access: Incurs additional PrivateLink fees (instance and data processing). Billing of PrivateLink.
- Public network access: Public traffic is billed through Cloud Data Transfer (CDT). CDT console.

1. Create a dedicated gateway

Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).
On the Inference Gateway tab, click Create Dedicated Gateway, and then select Fully managed dedicated gateway.
Configure the gateway parameters. Select a Gateway specification based on the Appendix: Dedicated Gateway Capacity Guide.
Click Buy now and complete the payment.

The gateway is ready when its Status changes to Running.

Note

You can update the gateway specification and node count after creation. Changes take effect in 3-5 minutes.
Set a gateway as the default to auto-select it for future deployments.

2. Bind a service to the gateway

This example deploys a new service. To change the gateway of an existing service, update the service configuration.

Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).
On the Inference Service tab, click Deploy Service. In the Custom Model Deployment section, click Custom Deployment.
In the Network Information section, select Dedicated Gateway and choose the dedicated gateway that you created from the drop-down list.

3. Enable network access

3.1 Configure public access

On the Inference Gateway tab, click the name of the target fully-managed dedicated gateway to go to its details page.
In the Gateway Access Control section, on the Internet tab, turn on the Access Portal switch. When the status is Activated, public access to the gateway is enabled.
The gateway blocks all public access by default. Click Add to Whitelist and enter the public IP CIDR blocks (for example, 192.0.2.0/24) you want to allow.
- Separate multiple entries with a comma (,) or a line break.
- Add 0.0.0.0/0 to allow all public IPs. Maximum 15 CIDR blocks.

Verify the public network connectivity of the gateway.

On the Internet tab, find the Endpoint. On the Gateway Access Control page, click the Public Network tab, view the Domain Name (in a format such as gw-c***s.aliyuncs.com), and confirm that the Access Entry status is Enabled.

From a device in an allowed IP range, ping the domain. A successful reply confirms connectivity.

C:\Users\xxx>ping gw-cxxx.cn-beijing.pai-eas.aliyuncs.com

Pinging nlb-w0ncxxx.cn-beijing.nlb.aliyuncs.com [xxx] with 32 bytes of data:
Reply from xxx: bytes=32 time=24ms TTL=89
Reply from xxx: bytes=32 time=29ms TTL=89
Reply from xxx: bytes=32 time=24ms TTL=89
Reply from xxx: bytes=32 time=24ms TTL=89

Ping statistics for xxx:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 24ms, Maximum = 29ms, Average = 25ms

To disable public network access, on the Internet tab, turn off the Access Portal switch.

The ping command may still resolve the IP, but telnet connections to the service port will fail, confirming public access is disabled.

C:\Users\xxx> ping gw-ccqv77ddlxxx-xxx.cn-beijing.pai-eas.aliyuncs.com

Pinging gw-ccq xxx cn-beijing.pai-eas.aliyuncs.com [47.xxx.xxx.xxx] with 32 bytes of data:
Reply from xxx: bytes=32 time=26ms TTL=89
Reply from xxx: bytes=32 time=26ms TTL=89
Reply from xxx: bytes=32 time=27ms TTL=89
Reply from xxx: bytes=32 time=26ms TTL=89

Ping statistics for 47.xxx.xxx.xxx:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 26ms, Maximum = 27ms, Average = 26ms

(base) niki@xxx ~ % telnet gw-ccqxxx.cn-beijing.pai-eas.aliyuncs.com 80
Trying 47.xx.xx.xxx...
telnet: connect to address 47.xx.xx.xxx: Bad file descriptor
telnet: Unable to connect to remote host

3.2 Configure VPC access

On the Inference Gateway tab, click the name of the target fully-managed dedicated gateway to go to its details page.
In the Gateway Access Control section, on the VPC tab, click Add VPC. Select the VPC and vSwitch you want to connect.
- Cross-account VPC access: Add a VPC from a different account in the same region. Servers in that VPC can then access EAS services through the gateway's VPC endpoint. The target account must have PAI-EAS, PrivateLink, and PrivateZone enabled.
  
  Note
  This feature requires allowlist approval. Submit a ticket to enable it.
  
  In the Add VPC dialog box, enter the Account ID (target account UID for cross-account access), VPC (ID), and vSwitch ID. The vSwitch must be in a gateway-supported availability zone (for example, cn-hangzhou-j, cn-hangzhou-k, or cn-hangzhou-i). Enable Authoritative DNS resolution if needed, then click OK.
- Authoritative DNS resolution for gateway domains: Use this when accessing EAS from other clouds or on-premises data centers over an existing Alibaba Cloud network connection. Only one VPC configuration can have authoritative DNS resolution enabled at a time.
By default, a new VPC entry allows all VPC IPs (0.0.0.0/0). Click Modify Whitelist to restrict access.

Verify the private network connectivity of the gateway.

On the VPC tab, find the Endpoint. On the Access Control page, select the VPC tab, and obtain the gateway domain name from the Domain Name Address column (for example, gw-xxx.eas.aliyuncs.com).

From a terminal within the VPC, ping the domain. A successful reply confirms connectivity.

Note

After allowlist configuration, terminals in any availability zone within the VPC can access the gateway, not just zones with gateway vSwitches.

[root@iZ2xxx ~]# ping gw-567lydxxx-vpc.cn-beijing.pai-eas.aliyuncs.com
PING ep-2zeixxx.epsrv-2zemwo87lxxx.cn-beijing.privatelink.aliyuncs.com (192.xxx.xxx.xxx) 56(84) bytes of data.
64 bytes from 192.xxx.xxx.11: icmp_seq=1 ttl=102 time=1.11 ms
64 bytes from 192.xxx.xxx.11: icmp_seq=2 ttl=102 time=1.05 ms
64 bytes from 192.xxx.xxx.11: icmp_seq=3 ttl=102 time=0.572 ms
64 bytes from 192.xxx.xxx.11: icmp_seq=4 ttl=102 time=0.515 ms
64 bytes from 192.xxx.xxx.11: icmp_seq=5 ttl=102 time=0.519 ms
64 bytes from 192.xxx.xxx.11: icmp_seq=6 ttl=102 time=0.514 ms
64 bytes from 192.xxx.xxx.11: icmp_seq=7 ttl=102 time=0.508 ms
^C
--- ep-2zeixxx.epsrv-2zemwo87lxxx.cn-beijing.privatelink.aliyuncs.com ping statistics ---
7 packets transmitted, 7 received, 0% packet loss, time 6111ms
rtt min/avg/max/mdev = 0.508/0.682/1.107/0.252 ms

To disable private network access, find the vSwitch in the list and click Delete in the Configure vSwitch column.

From a VPC terminal, access the domain. The following output confirms private access is disabled.

[root@iZ2zef8xxx ~]# ping gw-xxx-vpc.cn-beijing.pai-eas.aliyuncs.com
PING ep-2zei6xxxlcb.epsrv-2zeute2zixxx.cn-beijing.privatelink.aliyuncs.com (1xxx) 56(84) bytes of data.

[root@iZ2zcr0StvqwCommduccSU ~]# telnet gw-ccqvxxx-vpc.cn-beijing.pai-eas.aliyuncs.com 80
Trying 1xxx...

4. Test service invocation

On the Inference Service tab, find the target service and click Invocation Method in the Service Type column.
On the Dedicated Gateway tab, get the Internet Endpoint, VPC Endpoint, and Token.
Use a curl command to send a request and verify that the response is correct.
- Public invocation: Run the command from any machine with internet access.
- Private invocation: Run the command from a terminal within the associated VPC.
```
curl <endpoint_url> -H'Authorization:<token>'
```
The test endpoint expects a GET request with no parameters and should return True.

5. Configure a custom domain

1. (Optional) Prepare an SSL certificate

To use HTTPS, prepare an SSL certificate for your custom domain in Certificate Management Service.

Log on to the Certificate Management Service console and select SSL Certificate Management.
Choose Purchase Certificate or upload an existing certificate. Purchase SSL certificates and Upload SSL certificates.

2. Configure the custom domain

Public custom domain

On the gateway details page, switch to the Domain Name tab and click Create Domain Name.

In the Create domain panel, set Access method to Public, enter your domain, and select an SSL certificate. If needed, click Purchase/upload certificate to manage certificates.
Note
- Public custom domain settings may take up to 5 minutes to apply to deployed services.
- Verify that the service's public endpoint URL uses the custom domain.
Configure public DNS resolution.
1. On the Gateway tab of the gateway page, find the gateway's public domain address.
  
  On the Gateway Access Control page, click the Public network tab. Note the Domain address (for example, gw-c***s.aliyuncs.com) and confirm Access entry is Enabled.
2. Add a CNAME record for your public custom domain that points to the gateway's public domain address.
Set up authoritative public DNS using Domain Management and Add a DNS Record.

Private custom domain

On the gateway details page, switch to the Domain Name tab and click Create Domain Name.

Set Access method to VPC, enter your domain (for example, www.test.com), optionally select an SSL certificate, and click OK.
Changes take up to 5 minutes to apply. Verify that the VPC endpoint URL uses your private domain.

Monitoring and alerts

Enable logs, monitoring, and alerts to detect gateway anomalies early.

Procedure

Enable logs, monitoring, and alerts. On the gateway details page, go to the Logs, Monitoring, and Alert tabs and follow the on-screen instructions to enable each feature.
Create an alert policy. After enabling alerts, click Create EAS dedicated gateway alert policy to configure alert rules. Manage alert rules.

Important
The {{$labels.envoy_clusterid}} variable displays the gateway's internal ID, not its custom name. To improve alert readability, add a recognizable gateway name in the Alert content template. Example: PAI-EAS dedicated gateway {{$labels.envoy_clusterid}} node ({{$labels.pod_name}}) CPU utilization has exceeded {{ $labels.metrics_params_value }}%. Current value: {{ printf "%.2f" $value }}%.

Alert metrics

Parameter	Description	Recommendation
EAS dedicated gateway CPU utilization	The CPU utilization percentage of the gateway instance (Pod).	Recommendation: `Sustained for 5 minutes > 85%`. Scenario: Persistently high utilization suggests resource bottleneck. Consider scaling out.
EAS dedicated gateway memory utilization	The memory utilization percentage of the gateway instance (Pod).	Recommendation: `Sustained for 5 minutes > 85%`. Scenario: Persistently high utilization may cause OOMKilled events.
EAS dedicated gateway overall success rate	`(Number of requests with 2xx HTTP status codes / Total number of requests) * 100%`	Recommendation: `Sustained for 1 minute < 99.9%`. Scenario: Core availability metric. A drop indicates backend or configuration issues.
EAS dedicated gateway certificate expiration	Monitors the remaining validity (in days) of the gateway HTTPS certificate.	Recommendation: `Remaining days < 15`. Scenario: Prevents HTTPS disruption from an expired certificate.
EAS dedicated gateway 4xx/5xx request ratio	`(Number of 4xx or 5xx requests / Total number of requests) * 100%`. We recommend configuring separate alerts for 4xx and 5xx errors.	5xx Recommendation: `Sustained for 1 minute > 1%`. 4xx Recommendation: `Sustained for 5 minutes > 5%`. Scenario: 5xx spikes indicate server-side issues. 4xx spikes suggest client invocation or authentication errors.
EAS dedicated gateway average request RT	Average response time (ms) of all requests in the statistical period.	Recommendation: Set based on your service baseline, for example, `Sustained for 3 minutes > 200ms`. Scenario: RT increases directly affect user experience.
EAS dedicated gateway request volume (day-over-day)	Percentage change in request volume compared to the same period yesterday.	Recommendation: `Sustained for 10 minutes < -50%` or `> 200%`. Scenario: Detects sudden traffic drops (potential outage) or surges (potential attack or traffic spikes).
EAS dedicated gateway request throttling	Requests rejected by the gateway throttling policy in the statistical period.	Recommendation: `Count in 1 minute > 10`. Scenario: Frequent throttling means traffic exceeds capacity. Adjust throttling or scale out.

Usage notes

Auto scaling does not support scaling from zero with a dedicated gateway. Set the minimum replica count to at least 1.

FAQ

Q: Why do I get the error `Vswitch vsw-2zeqwh8hv0gb96zcd in zone cn-beijing-g is not supported, supported zones: [cn-beijing-i cn-beijing-l cn-beijing-k]` when adding a VPC?

The selected vSwitch is in an unsupported availability zone. Choose a vSwitch in a supported zone.