Model cache acceleration

Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).

Click Deploy Service. In the Custom Model Deployment section, click Custom Deployment.

The key parameters are as follows. For details about other parameters, see custom deployment.

1. In the Environment Information section, configure Mount storage to mount your model files to a container directory. For example, to mount files from OSS:

   • Uri: The OSS path where the model files are located, such as oss://path/to/models/Qwen3-8B/.

   • Mount Path: The path inside the container where the files are mounted, such as /mnt/models/Qwen3-8B/.

2. In the Features section, turn on the Distributed cache acceleration switch and configure the following parameters:

   Parameter	Description
   Maximum Memory Usage	The maximum memory that the cache can use, in GB. When this limit is exceeded, the system evicts data based on an LRU policy. Example: 20 GB.
   Source Path	The source directory to accelerate. Enter the container path where OSS or NAS storage is mounted.
   Accelerated Path	The local cache path. Your application reads models from this directory. This path must be different from the source path. Example: /mnt/models/Qwen3-8B-fast/.
   Model Cache Prefetch Service	(Optional) Select a deployed cache warm-up service. This option is recommended for new deployments or scenarios that require fast cold starts, such as those involving large model files or frequent scale-outs. To use this option, you must first deploy a cache warm-up service.

3. In the Environment Information section, modify the Command to Run. Change the model file path in the command from the source path to the accelerated path. For example, when deploying an LLM service:

   vllm serve /mnt/models/Qwen3-8B-fast/

After configuring the parameters, click Deploy.

Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).

On the Elastic Algorithm Service (EAS) page, click Deploy Service. In the Custom Model Deployment section, click JSON Deployment.

Enter the JSON configuration. Example:

{
    "cloud": {
        "computing": {
            "instances": [
                {
                    "type": "ecs.gn6e-c12g1.3xlarge"
                }
            ]
        },
        "networking": {
            "security_group_id": "your-security-group-id",
            "vpc_id": "your-vpc-id",
            "vswitch_id": "your-vswitch-id"
        }
    },
    "containers": [
        {
            "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.11.2-py312-mows0.5.1",
            "port": 8000,
            "script": "vllm serve /mnt/models/Qwen3-8B-fast/"
        }
    ],
    "metadata": {
        "cpu": 12,
        "disk": "30Gi",
        "gpu": 1,
        "instance": 1,
        "memory": 92000,
        "name": "vllm_test",
        "workspace_id": "your-workspace-id"
    },
    "storage": [
        {
            "mount_path": "/mnt/models/Qwen3-8B/",
            "oss": {
                "path": "oss://path/to/models/Qwen3-8B/",
                "readOnly": false
            }
        },
        {
            "cache": {
                "capacity": "10G",
                "path": "/mnt/models/Qwen3-8B/",
                "cacheroot_service": "your-cacheroot-service"
            },
            "mount_path": "/mnt/models/Qwen3-8B-fast/"
        }
    ]
}

The following table describes parameters related to model cache acceleration. For information about other parameters, see JSON deployment.

Click Deploy.