AI acceleration overview-Platform For AI(PAI)-阿里云帮助中心

PAI provides acceleration tools for AI training and inference, covering dataset caching, distributed training, and model optimization.

Acceleration tools

PAI offers the following acceleration tools for different stages of the AI workflow:

Tool	Stage	Description
DataSetAcc	Data loading	Accelerates dataset sample I/O in storage-compute separation scenarios. Supports multiple storage media and file types. Non-intrusive: no code changes required.
TorchAcc	Training	PyTorch training acceleration framework. Optimizes distributed training with data parallelism, computation-communication overlap, AMP training, and GPU memory optimization.
EPL	Training	Large-scale distributed training framework for TensorFlow. Supports data parallelism, operator splitting, pipeline parallelism, and automatic parallelism strategies.
Pai-Megatron-Patch	Training	Accelerates Transformer model training in PyTorch by integrating multiple optimization techniques with Transformer model libraries.
PAI-Blade	Inference	General-purpose inference optimization for TensorFlow and PyTorch. Supports GPUs, CPUs, and edge devices. Applies graph optimization, AI compiler optimization, mixed precision, and automatic compression through a single API call.
BladeLLM	Inference	High-performance inference engine for large language models (LLMs). Optimizes deployment with model quantization, efficient batching, and GPU memory management.
Local cache acceleration	Data loading	Caches datasets on Lingjun compute nodes to eliminate repeated remote storage reads during multi-epoch training. Supports OSS and CPFS storage.

Select the tool that matches your optimization goal.

DataSetAcc

Accelerates dataset I/O for AI workloads in the cloud. See Dataset Accelerator overview.
TorchAcc

PyTorch training acceleration framework that converts dynamic graphs to static graphs for optimized distributed training. See Distributed training acceleration (TorchAcc).
EPL

Distributed training framework for TensorFlow with automatic parallelism strategies. See EPL for distributed training acceleration.
Pai-Megatron-Patch

Training optimization tool for PyTorch Transformer models. Improves training speed through acceleration switches. See Transformer training acceleration (Pai-Megatron-Patch).
PAI-Blade

General-purpose inference optimization tool that integrates graph optimization, mixed precision, and compression techniques. See Inference acceleration (Blade) overview.
BladeLLM

High-performance inference engine for LLM deployment with quantization and efficient batching. See LLM inference engine (BladeLLM).
Local cache acceleration

Caches datasets on Lingjun compute nodes for faster multi-epoch training. See Local cache acceleration.

For hands-on tutorials, see AI acceleration use cases.