Deploy and call a model service on EAS

I. Prerequisites

Activate PAI and create a workspace with your Alibaba Cloud main account. In the PAI console, select a region and complete authorization and product activation.

II. Billing

This topic uses pay-as-you-go public resources to create the model service. For more information about the billing rules, see EAS billing.

III. Preparations

To deploy a model service, you need to prepare model files and code files, such as a web interface. If the available Alibaba Cloud images do not meet your requirements, you must create a custom image.

# Download the model
from modelscope import snapshot_download
model_dir = snapshot_download('Qwen/Qwen3-0.6B')

from flask import Flask

app = Flask(__name__)

@app.route('/hello/model')
def hello_world():
    # You can call the model here to get results.
    return 'Hello World'

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8000)

IV. Service deployment

1. Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).

2. Click Deploy Service. In the Custom Model Deployment section, click Custom Deployment.

3. Configure the following key parameters. Keep defaults for other parameters.

   • Deployment Method: Select Image-based Deployment.

   • Image Configuration: In the Alibaba Cloud Image list, select vllm:0.11.2-mows0.5.1.

   • Mount storage: This example stores the model file in OSS at path oss://examplebucket/models/Qwen/Qwen3-0___6B. Select OSS and configure as follows.

     • Uri: OSS path where the model is located. Set to oss://examplebucket/models/.

     • Mount Path: Destination path in the service instance where the file is mounted, such as /mnt/data/.

   • Command: The official image has a default startup command. Modify as needed. For this example, change to vllm serve /mnt/data/Qwen/Qwen3-0___6B.

   • Resource Type: Select Public Resources. For Resource Specification, select ecs.gn7i-c16g1.4xlarge. To use other resource types, see Resource configurations.

4. Click Deploy. Service deployment takes about 5 minutes. When Service Status changes to Running, the service is successfully deployed.

V. Online debugging

Use online debugging to verify the service. Configure the request method, path, and body for your model.

For this example:

1. On the Inference Service tab, click the destination service to go to the service overview page. Switch to the Online Debugging tab.

2. In the Request Parameter Online Tuning section of the debugging page, set request parameters and click Send Request. Request parameters:

   • Chat interface: Append /v1/chat/completions to the existing URL.

   • Headers: Add a request header. Set key to Content-Type and value to application/json.

     

   • Body:

     {
       "model": "/mnt/data/Qwen/Qwen3-0___6B",
       "messages": [
         {
           "role": "user",
           "content": "Hello!"
         }
       ],
       "max_tokens": 1024
     }

3. Response:

   

VI. Service invocation

curl http://16********.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/****/v1/chat/
completions \
-H "Content-Type: application/json" \
-H "Authorization: *********5ZTM1ZDczg5OT**********" \
-X POST \
-d '{
  "model": "/mnt/data/Qwen/Qwen3-0___6B",
  "messages": [
    {
      "role": "user",
      "content": "Hello!"
    }
  ],
  "max_tokens": 1024
}' 

import requests

# Replace with the actual endpoint.
url = 'http://16********.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/***/v1/chat/completions'
# For the header, set Authorization value to the actual token.
headers = {
    "Content-Type": "application/json",
    "Authorization": "*********5ZTM1ZDczg5OT**********",
}
# Construct the service request based on the data format required by the model.
data = {
  "model": "/mnt/data/Qwen/Qwen3-0___6B",
  "messages": [
    {
      "role": "user",
      "content": "Hello!"
    }
  ],
  "max_tokens": 1024
}
# Send the request.
resp = requests.post(url, json=data, headers=headers)
print(resp)
print(resp.content)

VII. Stop or delete service

This service uses pay-as-you-go public resources. Stop or delete the service to avoid charges when no longer needed.

VIII. Related documentation

• Improve LLM service efficiency: Deploy the LLM intelligent router.

• EAS feature overview: EAS overview.