DocsICP FilingConsole
Documentation
Home Page Container Compute Service User Guide Images ACS AI container images ACS AI image release notes training-nv-pytorch 26.07

training-nv-pytorch 26.07

Updated at:
Copy as MD
Product
My Favorites

training-nv-pytorch 26.07 is a container image for model training and inference. This topic describes the release highlights, image specifications, and how to get started with this image.

Release highlights

Main features

  • Upgraded PyTorch and related components to 2.12.1, and Triton to 3.7.1.

  • Training components: upgraded Transformer Engine to 2.16.1, Megatron-Core to 0.18.2, and Transformers to 5.12.1. Community features and corresponding fixes are merged.

  • Inference components: upgraded vLLM to 0.24.0 and FlashInfer to 0.6.12. Community features and corresponding fixes are merged.

Bug fixes

None

Key features and enhancements

PyTorch compilation optimization

torch.compile(), introduced in PyTorch 2.0, often delivers strong gains for small-scale, single-GPU workloads. But LLM training depends on GPU memory optimization and distributed frameworks such as FSDP or DeepSpeed, so torch.compile() may offer limited benefits or even degrade performance.

  • Control communication granularity in the DeepSpeed framework. This helps the compiler capture a more complete compute graph and apply broader compilation optimizations.

  • Use an optimized PyTorch build:

    • The PyTorch compiler frontend is improved to ensure that compilation succeeds even if a graph break occurs in the compute graph.

    • Pattern matching and dynamic shape support are strengthened to improve post-compilation performance.

Control communication granularity in the DeepSpeed framework. This helps the compiler capture a more complete compute graph and apply broader compilation optimizations.

Use an optimized PyTorch build:

  • The PyTorch compiler frontend is improved to ensure that compilation succeeds even if a graph break occurs in the compute graph.

  • Pattern matching and dynamic shape support are strengthened to improve post-compilation performance.

ACCL

ACCL is Alibaba Cloud’s high-performance communication library built for Lingjun. ACCL-N is the GPU-focused version. ACCL-N is a high-performance communication library customized from NVIDIA NCCL. It is fully compatible with NCCL, fixes issues in the upstream NCCL release, and includes performance and stability improvements.

Image specifications

The following table lists the specifications of the training-nv-pytorch 26.07 image.

Item

Value

Image name

training-nv-pytorch

Tag

26.07-serverless

Scenario

Training and inference

Framework

PyTorch

NVIDIA driver requirement

580 or later

Supported architectures

amd64, aarch64

Core components

amd64

  • Ubuntu: 24.04

  • Python: 3.12.7+gc

  • CUDA: 13.0.2

  • perf: 5.4.30

  • gdb: 15.1

  • torch: 2.12.1+ali.12.nv25.10

  • triton: 3.7.1

  • transformer_engine: 2.16.1+c9877beb

  • deepspeed: 0.18.8+ali

  • flash_attn: 2.8.3

  • flash_attn_3: 3.0.0

  • transformers: 5.12.1

  • grouped_gemm: 1.1.4

  • accelerate: 1.11.0+ali

  • diffusers: 0.34.0

  • mmengine: 0.10.3

  • mmcv: 2.1.0

  • mmdet: 3.3.0

  • opencv-python-headless: 4.11.0.86

  • ultralytics: 8.3.96

  • timm: 1.0.28

  • vllm: 0.24.0

  • flashinfer-python: 0.6.12

  • peft: 0.19.1

  • megatron-core: 0.18.2

aarch64

  • Ubuntu: 24.04

  • Python: 3.12.7+gc

  • CUDA: 13.0.2

  • gdb: 15.1

  • torch: 2.12.1+ali.12.nv25.10

  • triton: 3.7.1

  • transformer_engine: 2.16.1+c9877beb

  • deepspeed: 0.18.8+ali

  • flash_attn: 2.8.3

  • transformers: 5.12.1

  • grouped_gemm: 1.1.4

  • accelerate: 1.11.0+ali

  • diffusers: 0.34.0

  • mmengine: 0.10.3

  • mmcv: 2.1.0

  • mmdet: 3.3.0

  • opencv-python-headless: 4.11.0.86

  • ultralytics: 8.3.96

  • timm: 1.0.28

  • vllm: 0.24.0

  • flashinfer-python: 0.6.12

  • peft: 0.19.1

  • megatron-core: 0.18.2

Driver requirements

Images

Public images

CUDA 13.0.2 (driver 580 or later, amd64 and aarch64)

  • egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/training-nv-pytorch:26.07-serverless

VPC images

To quickly pull ACS AI container images in a VPC, replace the Asset URI of the specified AI container image egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/{image:tag} with acs-registry-vpc.{region-id}.cr.aliyuncs.com/egslingjun/{image:tag}.

  • {region-id}: the region ID of the ACS product. For example, cn-beijing and cn-wulanchabu.

  • {image:tag}: the name and tag of the AI container image. For example, inference-nv-pytorch:25.10-vllm0.11.0-pytorch2.8-cu128-20251028-serverless and training-nv-pytorch:25.10-serverless.

Images with the tag yy.mm-serverless (for example, 26.07-serverless) are applicable to Alibaba Cloud Container Compute Service (ACS) and Lingjun multi-tenant deployments. For Lingjun single-tenant deployments, use images tagged yy.mm (for example, 26.07).

Quick start

To use the training-nv-pytorch image in ACS (recommended), select the image from the Artifact Center page when you create a workload in the console, or specify the image in a YAML file.

The following example demonstrates how to pull the training-nv-pytorch image by using Docker, start a container, and run training examples.

Before you begin, make sure that NVIDIA driver 580 or later is installed. For more information, see Driver requirements.

1. Pull the image

Replace [tag] with the image tag that applies to your product: 26.07-serverless for ACS and the Lingjun multi-tenant product, or 26.07 for the Lingjun single-tenant product.

docker pull egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/training-nv-pytorch:[tag]

2. Start the container

The image includes built-in model training tools. The following example starts a container and runs training tasks.

LLM workloads

# Start the container and enter
docker run -it --gpus=all --ipc=host --net=host egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/training-nv-pytorch:[tag]

# Dense: Qwen3.5-4B LoRA smoke test
NPROC_PER_NODE=2 CUDA_VISIBLE_DEVICES=0,1 \
swift sft --model Qwen/Qwen3.5-4B --tuner_type lora \
    --dataset 'AI-ModelScope/alpaca-gpt4-data-zh#500' \
    --split_dataset_ratio 0.01 --num_train_epochs 1 \
    --target_modules all-linear --attn_impl flash_attn \
    --deepspeed zero2 --max_length 2048 --output_dir output

# MoE example: Qwen/Qwen3.5-35B-A3B
NPROC_PER_NODE=8 CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
megatron sft --model Qwen/Qwen3.5-35B-A3B \
    --dataset 'AI-ModelScope/alpaca-gpt4-data-zh#500' \
    --split_dataset_ratio 0.01 --num_train_epochs 1 --finetune true \
    --expert_model_parallel_size 8 --moe_grouped_gemm true \
    --micro_batch_size 1 --global_batch_size 8 \
    --recompute_granularity full --recompute_method uniform --recompute_num_layers 1 \
    --attention_backend flash --max_length 2048 \
    --output_dir megatron_output/Qwen3.5-35B-A3B

3. Usage recommendations

  • The image contains modified versions of libraries such as PyTorch and DeepSpeed. Do not reinstall these libraries.

  • In the DeepSpeed configuration, leave zero_optimization.stage3_prefetch_bucket_size empty or set it to auto.

Known issues

None

Previous：inference-nv-pytorch 25.02Next：training-nv-pytorch 26.06