inference-nv-pytorch 26.03 release notes-Container Compute Service(ACS)-阿里云帮助中心

What's new

Main features

Provides images for CUDA 12.8 and CUDA 13.0:
- CUDA 12.8 — supports amd64 only
- CUDA 13.0 — supports amd64 and aarch64
Upgrades vLLM to v0.17.1 with Qwen3.5 model support.

Bug fixes

None.

Image details

Image name	inference-nv-pytorch
Tag	26.03-vllm0.17.1-pytorch2.10-cu128-20260317-serverless	26.03-vllm0.17.1-pytorch2.10-cu130-20260317-serverless
Supported architecture	amd64	amd64	aarch64
Use case	large model inference	large model inference	large model inference
Framework	PyTorch	PyTorch	PyTorch
Requirements	NVIDIA Driver release ≥ 570	NVIDIA Driver release ≥ 580	NVIDIA Driver release ≥ 580
System components	Ubuntu 24.04 Python 3.12 Torch 2.10.0 CUDA 12.8 NCCL 2.29.7 diffusers 0.37.0 flash_attn 2.8.4 flash_attn_3 3.0.0 flashinfer-python 0.6.4 imageio-ffmpeg 0.6.0 ray 2.54.0 transformers 4.57.6 triton 3.6.0 torchaudio 2.10.0 torchvision 0.25.0 vllm 0.17.1 xfuser 0.4.5 xgrammar 0.1.29	Ubuntu 24.04 Python 3.12 Torch 2.10.0+cu130 CUDA 13.0.2 NCCL 2.29.7 diffusers 0.37.0 flash_attn 2.8.4 flash_attn_3 3.0.0 flashinfer-python 0.6.4 imageio-ffmpeg 0.6.0 ray 2.54.0 transformers 4.57.6 triton 3.6.0 torchaudio 2.10.0+cu130 torchvision 0.25.0+cu130 vllm 0.17.1 xfuser 0.4.5 xgrammar 0.1.29 ljperf 0.1.0+d0e4a408	Ubuntu 24.04 Python 3.12 Torch 2.10.0+cu130 CUDA 13.0.2 NCCL 2.29.7 flash_attn 2.8.4 flashinfer-python 0.6.4 transformers 4.57.6 ray 2.54.0 vllm 0.17.1 triton 3.6.0 torchaudio 2.10.0+cu130 torchvision 0.25.0+cu130 xgrammar 0.1.29 ljperf 0.1.0+477686c5

Assets

Public images

CUDA 12.8

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:26.03-vllm0.17.1-pytorch2.10-cu128-20260317-serverless

CUDA 13.0

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:26.03-vllm0.17.1-pytorch2.10-cu130-20260317-serverless

VPC images

To speed up image pulls within a VPC, replace the registry hostname with a region-specific VPC endpoint.

Change the image path from:

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/{image:tag}

To:

acs-registry-vpc.{region-id}.cr.aliyuncs.com/egslingjun/{image:tag}

Placeholder	Description	Example
`{region-id}`	Region ID of your ACS cluster	`cn-beijing`, `cn-wulanchabu`
`{image:tag}`	Image name and tag	`inference-nv-pytorch:26.03-vllm0.17.1-pytorch2.10-cu128-20260317-serverless`

Note

Compatible with standard ACS products and multi-tenant Lingjun environments. Not supported in single-tenant Lingjun environments.

Driver requirements

CUDA 12.8: NVIDIA Driver release ≥ 570
CUDA 13.0: NVIDIA Driver release ≥ 580

Quick start

This example uses Docker only to pull the inference-nv-pytorch image and test the inference service with the Qwen2.5-7B-Instruct model.

Note

To use this image in ACS, select it from the Artifact Center when you create a workload in the console, or specify the image reference in a YAML file. For more information, see the following topics about building model inference services with ACS GPU compute:

Pull the image.

docker pull egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

Download the model from ModelScope.

pip install modelscope
cd /mnt
modelscope download --model Qwen/Qwen2.5-7B-Instruct --local_dir ./Qwen2.5-7B-Instruct

Enter the container.

docker run -it --rm --gpus all --network=host --privileged --init --ipc=host \
--ulimit memlock=-1 --ulimit stack=67108864  \
-v /mnt/:/mnt/ \
egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

Verify vLLM chat completions.

Start the server.

python3 -m vllm.entrypoints.openai.api_server \
--model /mnt/Qwen2.5-7B-Instruct \
--trust-remote-code --disable-custom-all-reduce \
--tensor-parallel-size 1

Run a client-side test.

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "/mnt/Qwen2.5-7B-Instruct",  
    "messages": [
    {"role": "system", "content": "You are a friendly AI assistant."},
    {"role": "user", "content": "Introduce deep learning."}
    ]}'

Output:

{"id":"chat-d3c28759793d4376a65bfc4e40b59a71","object":"chat.completion","created":1735278194,"model":"/mnt/deep_learning_test/testsuite/dataset/llms_inference_qwen7b-v2.5_accelerate/checkpoint/7B-V2.5/","choices":[{"index":0,"message":{"role":"assistant","content":"Deep learning is a subset of machine learning inspired by the biological nervous system, particularly the interaction between neurons in the brain. Deep learning utilizes deep neural networks to process and analyze large amounts of data to identify effective predictive models. This technology has achieved significant success in various tasks, including image recognition, speech recognition, and natural language processing.\n\nIn deep learning, a neural network consists of multiple layers, including an input layer, several hidden layers, and an output layer. Each layer contains multiple nodes (or neurons) connected to nodes in other layers through weighted connections. During training, the neural network adjusts the weights of these connections based on the input data to minimize the error between the predicted output and the actual output. This process is typically achieved using optimization algorithms such as gradient descent.\n\nThe training of deep learning models requires substantial computational resources and data. In recent years, with advances in computing hardware (such as GPUs and TPUs) and the rapid growth of datasets, deep learning technology has been widely applied and developed. In addition to the applications mentioned above, deep learning is also extensively used in medical diagnosis, autonomous driving, gaming, and finance.","tool_calls":[]},"logprobs":null,"finish_reason":"stop","stop_reason":null}],"usage":{"prompt_tokens":237,"completion_tokens":213}}

The vLLM documentation covers additional features and configuration.

Known issues

The 26.03 image does not support the deepgpu-comfyui plugin.