Quick start: Deploy AI inference with PPU in ACS

Create an ACS cluster

This section shows how to quickly create an ACS cluster using only essential parameters.

For detailed information about all cluster configuration parameters, see Create an ACS cluster.

1. Log on to the ACS console. In the left navigation pane, click Clusters.

2. On the Clusters page, click Create Kubernetes Cluster in the upper-left corner.

3. On the Create Cluster page, configure the following parameters and use the default settings for the rest.

   Parameter	Description	Example
   Cluster name	Enter a name for the cluster.	ACS-PPU-Inference
   Region	Select the region where the cluster will be located.	China (Wulanchabu)

4. Click Confirm Configuration. After all dependency checks pass, click Create Cluster.

   The cluster creation process takes about 5 to 10 minutes.

Create a storage volume

This section shows how to create a NAS file system to use as a persistent storage volume for model files.

For more information about using NAS or OSS storage volumes to persist model files, see Quickly build a data storage volume for large language models by using ACS.

1. On the Cluster List page, click the cluster name ACS-PPU-Inference to go to the Basic Information page, and then obtain the cluster's VPC and vSwitch IDs.

   1. In the Network section, under VPC, obtain the VPC ID of the cluster.

      

   2. In the Control Plane vSwitch section on the right, click Edit. Using China (Wulanchabu) Zone C as an example, obtain the vSwitch ID.

      

2. In the left-side navigation pane, choose Workloads > Custom Resources. Select the CustomResourceDefinition tab, click Create from YAML, paste the following YAML, and then click Create.

   Replace vpcId and vSwitchId in the YAML with the VPC and vSwitch IDs from the previous step.

   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
     name: nas-llm-sc
   mountOptions:
   - nolock,tcp,noresvport
   - vers=3
   parameters:
     volumeAs: filesystem
     fileSystemType: standard
     storageType: Performance
     regionId: cn-wulanchabu
     zoneId: cn-wulanchabu-c                  # This example uses China (Wulanchabu) Zone C. Ensure it matches the availability zone of your vSwitchId.
     vpcId: "vpc-0jljd55jeie816cxxxxxx"       # Replace this with the VPC ID of your cluster.
     vSwitchId: "vsw-0jl3qcn7m2gs449lxxxxx"   # Replace this with the vSwitch ID of your cluster.
     accessGroupName: DEFAULT_VPC_GROUP_NAME
     deleteVolume: "false"
   provisioner: nasplugin.csi.alibabacloud.com
   ---
   kind: PersistentVolumeClaim
   apiVersion: v1
   metadata:
     name: nas-llm-pvc
   spec:
     accessModes:
       - ReadWriteMany
     storageClassName: nas-llm-sc
     resources:
       requests:
         storage: 30Gi

3. After a success message appears, click nas-llm-pvc to view its status. A status of Bound indicates the NAS storage volume has been successfully created and bound.

   

Deploy Zhenwu 810E GPU resources

This section shows how to use the inference-xpu-pytorch inference image to create a workload that uses Zhenwu 810E GPU compute resources, mounts the NAS storage volume, and exposes it publicly with a LoadBalancer Service.

1. In the left-side navigation pane, choose Workloads > Deployments, click Create from YAML, paste the following YAML, and then click Create.

   Key parameters

   YAML configuration	Description
   alibabacloud.com/compute-class: gpu alibabacloud.com/gpu-model-series: PPU810E	The alibabacloud.com/compute-class label specifies that the instance compute type is gpu. The alibabacloud.com/gpu-model-series label specifies that the instance card type is PPU810E. For more information about GPU usage, see ACS Pod Instance Overview.
   requests: cpu: 8 memory: 16G alibabacloud.com/ppu: 1	The requested resource specifications. For more specifications, see PPU pod specifications.
   image: acs-registry-vpc.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-xpu-pytorch:25.06-v1.5.2-vllm0.8.5-torch2.6-cu126-20250610	Use the VPC endpoint to accelerate AI container image pulls and reduce pull times. Here, cn-wulanchabu is the region ID for China (Wulanchabu).
   volumes: - name: data persistentVolumeClaim: claimName: nas-llm-pvc # The PVC for the NAS storage volume created in the previous section. volumeMounts: - mountPath: /mnt # NAS mount path name: data	The nas-llm-pvc is the PersistentVolumeClaim for the NAS storage that you created. It is mounted to the /mnt directory in the pod.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     labels:
       app: llm-test
     name: llm-test
     namespace: default
   spec:
     progressDeadlineSeconds: 600
     replicas: 1
     revisionHistoryLimit: 10
     selector:
       matchLabels:
         app: llm-test
     template:
       metadata:
         labels:
           alibabacloud.com/compute-class: gpu
           alibabacloud.com/gpu-model-series: PPU810E
           alibabacloud.com/compute-qos: default
           app: llm-test
       spec:
         containers:
           - command:
             - sh
             - -c
             - sleep infinity
             image: acs-registry-vpc.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-xpu-pytorch:25.06-v1.5.2-vllm0.8.5-torch2.6-cu126-20250610
             imagePullPolicy: IfNotPresent
             name: llm-test
             resources:
               limits:
                 cpu: 8
                 memory: 16G
                 alibabacloud.com/ppu: 1
                 ephemeral-storage: 30Gi
               requests:
                 cpu: 8
                 memory: 16G
                 alibabacloud.com/ppu: 1
                 ephemeral-storage: 30Gi
             terminationMessagePath: /dev/termination-log
             terminationMessagePolicy: File
             volumeMounts:
               - mountPath: /mnt # NAS mount path
                 name: data
               - mountPath: /ppu-data
                 name: ephemeral
         dnsPolicy: ClusterFirst
         restartPolicy: Always
         schedulerName: default-scheduler
         securityContext: {}
         terminationGracePeriodSeconds: 30
         volumes:
           - name: data
             persistentVolumeClaim:
               claimName: nas-llm-pvc # nas-llm-pvc is the PersistentVolumeClaim for the NAS storage you created.
           - name: ephemeral
             emptyDir:
               sizeLimit: 30G
   ---
   apiVersion: v1
   kind: Service
   metadata:
     annotations:
       service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "internet"
       service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: ipv4
     labels:
       app: llm-test
     name: svc-llm
     namespace: default
   spec:
     externalTrafficPolicy: Local
     ports:
     - name: serving
       port: 8000
       protocol: TCP
       targetPort: 8000
     selector:
       app: llm-test
     type: LoadBalancer

2. After a success message appears, click llm-test to view its status.

   The pod takes about 5 minutes to reach the Running state, as it first needs to pull the container image.

Deploy the large language model

This section uses the Qwen2.5-1.5B-Instruct large language model as an example to show how to download and use model files in the container.

To speed up the download, you can increase your network bandwidth. For more information, see How do I speed up model downloads?.

1. Find the pod named llm-test-xxxxxxxxx-xxxxx and click Terminal in the Actions column to open a terminal session for the llm-test container.

2. Run the following commands to install ModelScope and download the model files to the specified directory.

   The model download takes about 5 minutes.

   pip install modelscope
   modelscope download --model Qwen/Qwen2.5-1.5B-Instruct --local_dir /mnt/Qwen2.5-1.5B-Instruct

   Expected output:

   ...
   Installing collected packages: modelscope
   Successfully installed modelscope-1.27.1
   ...
   Successfully Downloaded from model Qwen/Qwen2.5-1.5B-Instruct.                                                                                                                  | 1.00M/2.88G [00:00<16:35, 3.10MB/s]

Start the inference service

This section describes how to start the large language model inference service from the command line inside the container.

1. After the model is downloaded, run the model startup command.

   python3 -m vllm.entrypoints.openai.api_server --model /mnt/Qwen2.5-1.5B-Instruct/ --trust-remote-code --gpu-memory-utilization 0.95

   Expected output:

   ...
   INFO:     Started server process [150]
   INFO:     Waiting for application startup.
   INFO:     Application startup complete.

   Note

   Because you started the inference service manually in the terminal, keep the terminal session open. To configure the inference service to start automatically, proceed to the next step.

2. (Optional) In the ACS console, choose Workloads > Deployments. Find the llm-test workload and click Edit in the Actions column. In the Lifecycle section, replace the Command for Startup with the following content. Then, click Update on the right side of the page and confirm.

   ["sh","-c","python3 -m vllm.entrypoints.openai.api_server --model /mnt/Qwen2.5-1.5B-Instruct/ --trust-remote-code --gpu-memory-utilization 0.95"]

   

3. In the ACS console, choose Network > Services from the left-side navigation pane. Find the svc-llm service that you created. The External IP is the public IP address for your inference service.

4. Run the following curl command from your local machine to verify the inference service's chat completion capability.

   Replace <IP> with the external IP address of the service that you obtained in the previous step.

   curl http://<IP>:8000/v1/chat/completions \
       -H "Content-Type: application/json" \
       -d '{
       "model": "/mnt/Qwen2.5-1.5B-Instruct/",  
       "messages": [
       {"role": "system", "content": "You are a friendly AI assistant."},
       {"role": "user", "content": "Tell me about some tourist attractions in Hangzhou."}
       ]}'
   

   Expected output:

   

You have successfully deployed an AI inference service on ACS using Zhenwu 810E.

FAQ

Related topics

• Quickly build a data storage volume for large language models by using ACS

• Quickly deploy a Qwen3 inference service on Zhenwu 810E by using ACS

• Quickly deploy a QwQ-32B inference service by using GU8TF and Zhenwu 810E in ACS

• Quickly deploy a DeepSeek V3/R1 inference service by using Zhenwu 810E in ACS

PPU pod specifications

Pay-as-you-go, reserved capacity, and BestEffort instances share the same specification table.