GPU compute-optimized: gn, ebm, and scc

gn9gc, GPU-accelerated compute-optimized instance family

• Overview: gn9gc is Alibaba Cloud's 9th-generation cost-effective GPU cloud server instance family. It uses the latest-generation CIPU 2.0 to deliver cloud service capabilities, features high clock speed processors, and is configured with appropriate memory capacity. This instance family provides cost-effective instances for large language model (LLM) generation scenarios and video/image generation scenarios. The GPU can also directly provide graphics processing capabilities to support various rendering workloads.

• Use cases:

  • LLM inference: The new-generation GPU delivers compute power beyond the 8th generation with significantly improved memory bandwidth. Newly supported FP4 compute comprehensively improves inference performance and cost-effectiveness. Multi-GPU parallel inference efficiency is greatly enhanced.

• Compute:

  • Uses the latest CIPU 2.0 cloud processor.

    • The 2nd-generation CIPU provides higher cloud processing power with enhanced eRDMA, VPC, and EBS component capabilities. Supports containers (including but not limited to Docker, Clear Container, and Pouch).

  • Uses the new Blackwell architecture professional graphics card:

    • Supports OpenGL professional-grade graphics processing.

    • Supports RTX, TensorRT, and other common acceleration features, with newly upgraded FP4 support and PCIe Gen5 interconnect.

  • Key GPU specifications:

    GPU architecture	GPU memory	Computing performance	Video encoding/decoding	Inter-GPU interconnect	Acceleration APIs
    NVIDIA Blackwell	Capacity: 72 GB Bandwidth: 1,344 GB/s	TF32: 126 TFLOPS FP32: 52 TFLOPS FP16/BF16: 266 TFLOPS FP8/INT8: 530 TFLOPS FP4: 970 TFLOPS RT Core: 196 TFLOPS	3 x Video Encoder 3 x Video Decoder	PCIe interface: PCIe Gen5 x16 Bandwidth: 128 GB/s, P2P supported	DX12, OpenGL 4.6, Vulkan 1.3, CUDA 12.8, OpenCL 3.0, DirectCompute

• Storage:

  • I/O optimized.

  • Supports the NVMe protocol. For more information, see NVMe protocol.

  • Supported cloud disk types: elastic ephemeral disks, ESSDs, ESSD AutoPL disks, and regional ESSDs. For more information, see Block storage overview.

• Network:

  • Supports IPv4 and IPv6. For more information about IPv6, see IPv6.

  • Ultra-high network performance with up to 30 million PPS (8-GPU instances).

  • Supports ERI (Elastic RDMA Interface) for RDMA direct acceleration over VPC networks, with bandwidth up to 360 Gbit/s. Suitable for autonomous driving, embodied intelligence, computer vision, and traditional model training workloads.

  • Note

    For more information about ERI, see Enable eRDMA on an enterprise-level instance or Enable eRDMA on a GPU-accelerated instance.

The following table describes the instance types in the gn9gc instance family.

gn8v and gn8v-tee, GPU-accelerated compute-optimized instance family

These instance families are available in select regions, including those outside the Chinese mainland. To use them, contact your Alibaba Cloud sales representative.

• Introduction:

  • gn8v: An 8th-generation GPU-accelerated compute-optimized instance family from Alibaba Cloud for AI model training and inference on ultra-large language models (LLMs). This family provides instance types with one, two, four, or eight GPUs for various application requirements.

  • gn8v-tee: To enhance security for large model training and inference, Alibaba Cloud offers gn8v-tee, an 8th-generation instance family based on gn8v with a confidential computing feature. These instances encrypt data during GPU computation to protect your data.

• Use cases:

  • Cost-effective for multi-GPU parallel inference on LLMs with more than 70 billion parameters.

  • Each GPU provides 39.5 TFLOPS of FP32 compute power and delivers outstanding performance for traditional AI model training and autonomous driving training workloads.

  • The eight GPUs support NVLink interconnectivity and are suitable for training small- to medium-sized models.

• Features:

  • High-speed, large-capacity GPU memory: Each GPU is equipped with 96 GB of HBM3 GPU memory and provides up to 4 TB/s of memory bandwidth, significantly accelerating model training and inference.

  • High inter-GPU bandwidth: Multiple GPUs are interconnected with NVLink at 900 GB/s. This enables much higher efficiency for multi-GPU training and inference compared to previous-generation GPU instances.

  • LLM quantization: Supports FP8 compute power, which optimizes performance for large-scale parameter training and inference. This significantly improves training and inference speeds and reduces GPU memory usage.

  • (For gn8v-tee instances only) High security: Supports both CPU confidential computing with Intel® Trust Domain Extensions (TDX) and GPU confidential computing with NVIDIA Confidential Computing (CC). This provides end-to-end confidential computing for the entire model inference pipeline, protecting your inference data and enterprise models during model training and inference.

• Compute:

  • Powered by the latest CIPU 1.0.

    • Decouples compute from storage, letting you flexibly select the storage resources you need.

    • Provides bare metal capabilities, which support peer-to-peer (P2P) communication between GPU instances, unlike traditional virtualized instances.

  • Powered by 4th-generation Intel® Xeon® Scalable processors with a base frequency of up to 2.8 GHz and an all-core turbo frequency of up to 3.1 GHz.

• Storage:

  • I/O-optimized instance.

  • These instances support the NVMe protocol. For more information, see Overview of the NVMe protocol.

  • Supported cloud disk types: elastic ephemeral disk, ESSD, ESSD AutoPL disks, and Regional ESSD. For more information about cloud disks, see block storage overview.

• Network:

  • Supports IPv4 and IPv6. For more information about IPv6 communication, see IPv6 communication.

  • These instances support jumbo frames. For more information, see Jumbo frames.

  • Delivers ultra-high network performance with a packet forwarding rate of up to 30 million pps (on 8-GPU instances).

  • Supports elastic RDMA interface (ERI).

  • Note

    For information about how to use ERI, see Enable on enterprise-level instance or Enable eRDMA on GPU instances.

• Security: Supports the trusted computing feature (vTPM). This feature is available on gn8v instances but not on gn8v-tee instances. For more information, see Overview of trusted computing capabilities.

The following table describes the instance types in the gn8v family.

The following table describes the instance types in the gn8v-tee family.

gn8is, GPU-accelerated compute-optimized instance family

This instance family is available in select regions, including those outside the Chinese mainland. To use this instance family, contact your Alibaba Cloud sales representative.

• Introduction: gn8is is Alibaba Cloud's eighth-generation GPU-accelerated compute-optimized instance family, designed for the growing demands of AI-generated content (AIGC). Powered by the latest NVIDIA L20 GPUs, this family offers instance types with one, two, four, or eight GPUs, and various CPU-to-GPU ratios to meet diverse application needs.

• Features:

  • Graphics processing: Powered by 4th-generation Intel® Xeon® Scalable high-frequency processors, these instances provide robust CPU compute power for 3D modeling scenarios, ensuring smoother graphics rendering and design workflows.

  • Inference tasks: Equipped with the new NVIDIA L20 GPU, each with 48 GB of GPU memory, these instances accelerate inference tasks. They support the FP8 floating-point format and can be paired with Container Service for Kubernetes (ACK) to flexibly run inference for various AIGC models. They are especially suitable for inference tasks on large language models (LLMs) with fewer than 70 billion parameters.

• Use cases:

  • Use GRID drivers with images from Alibaba Cloud Marketplace to enable OpenGL and Direct3D capabilities. This provides workstation-grade graphics processing for workloads such as animation, film and television special effects, and rendering.

  • Use the container management capabilities of Container Service for Kubernetes (ACK) for more efficient and cost-effective AIGC image generation and LLM inference.

  • Other general-purpose AI applications, such as image recognition and speech recognition.

• Compute:

  • Powered by the new NVIDIA L20 enterprise-grade GPUs.

    • Supports common acceleration features such as TensorRT and the FP8 floating-point format to improve model inference performance.

    • Up to 48 GB of GPU memory per GPU. With multiple GPUs, instances in this family support single-instance inference for models with 70 billion or more parameters.

    • Enhanced graphics processing capabilities. After you install a GRID driver using Cloud Assistant or an image from Alibaba Cloud Marketplace, the graphics processing performance is twice that of 7th-generation platforms.

  • Key parameters of the NVIDIA L20 GPU:

    GPU architecture	GPU memory	Compute performance	Video encoding/decoding	Inter-GPU connectivity
    NVIDIA Ada Lovelace	Capacity: 48 GB Bandwidth: 864 GB/s	FP64: N/A FP32: 59.3 TFLOPS FP16/BF16: 119 TFLOPS FP8/INT8: 237 TFLOPS	3 × Video Encoders (+AV1) 3 × Video Decoders 4 × JPEG Decoders	PCIe interface: PCIe Gen4 x16 Bandwidth: 64 GB/s

  • Processor: Powered by the latest high-frequency Intel^® Xeon^® processors with an all-core turbo frequency of up to 3.9 GHz to handle complex 3D modeling demands.

• Storage:

  • All instances in this family are I/O-optimized instances.

  • These instances support the NVMe protocol. For more information, see Overview of the NVMe protocol.

  • Supported cloud disk types: elastic ephemeral disks, ESSDs, ESSD AutoPL disks, and Regional ESSDs. For more information about cloud disks, see Block storage overview.

• Network:

  • Supports IPv4 and IPv6. For more information about IPv6 communication, see IPv6 communication.

  • Supports Elastic RDMA Interface (ERI).

    Note

    For details on using ERI, see Enable eRDMA for enterprise-level instances or Enable eRDMA for GPU-accelerated instances.

• Security: These instances support the vTPM feature. For more information, see Overview of trusted computing.

gn7e, GPU-accelerated compute-optimized instance family

gn7i, GPU-accelerated compute-optimized instance family

gn7s, GPU-accelerated compute-optimized instance family

To use the gn7s instance family, submit a ticket.

gn7, GPU-accelerated compute-optimized instance family

gn7r, GPU-accelerated compute-optimized instance family

gn6i, GPU-accelerated compute-optimized instance family

gn6e, GPU-accelerated compute-optimized instance family

gn6v, GPU-accelerated compute-optimized instance family

GPU elastic bare metal server instance family ebmgn9g

• Instance family introduction: The ebmgn9g instance family is Alibaba Cloud's ninth-generation line of cost-effective, full-featured GPU bare metal instances. Powered by the latest CIPU 2.0, these instances combine high-frequency CPUs, large memory capacity, and new Blackwell architecture professional GPUs. They deliver high-performance and cost-effective GPU cloud services for a wide range of accelerated workloads, such as training for autonomous driving and embodied intelligence, large model inference, film and animation rendering, and services for the metaverse and cloud gaming.

• Use cases and features:

  • Autonomous driving/embodied intelligence:
    With 256 vCPUs running at an all-core frequency of up to 4.2 GHz and 2,304 GiB of memory, these instances handle the demanding data processing requirements for autonomous driving and embodied intelligence training.

  • Search and recommendation:
    The Blackwell GPU provides 126 TFLOPS of TF32 computing power. Each GPU is paired with 32 vCPUs and 153 GB/s of memory bandwidth, making it an optimal configuration for search and advertising services.

  • Large model inference:
    The new-generation GPU offers a significant performance leap over its predecessor, with video memory bandwidth increased to 1,344 GB/s. Support for FP4 computing power significantly improves inference performance and cost-effectiveness. The eight GPUs are interconnected via PCIe Gen5 with a bandwidth of 128 GB/s. This connection dramatically improves multi-card parallel inference efficiency.

  • Cloud gaming/rendering/metaverse:
    With a CPU frequency of up to 5 GHz, these instances are an excellent choice for 3D modeling. The GPU provides certified, workstation-grade graphics drivers with full OpenGL acceleration, making these instances the optimal choice for high-end film and animation development and CAD design.

• Powered by the latest CIPU 2.0:

  The second-generation CIPU offers increased processing power, enhancing the performance of eRDMA, VPC, and cloud disk components. Bare metal instances are ideal for workloads that require direct access to physical resources or have requirements such as hardware-based licensing. They also support containers, including but not limited to Docker, Clear Containers, and PouchContainer.

• Compute:

  • Powered by new Blackwell architecture professional GPUs:

    • Supports professional-grade OpenGL graphics processing.

    • Supports common acceleration features such as RTX and TensorRT, with newly added support for FP4 and PCIe Gen5 interconnect.

    • Utilizes a PCIe switch for interconnection, which improves NCCL performance by 36% compared to a direct-to-CPU connection. This can boost performance by up to 9% for large model inference with multi-card sharding.

  • Key GPU specifications:

    GPU architecture	Video memory	Compute performance	Encode/decode engines	Inter-GPU interconnect	Acceleration APIs
    Blackwell	Capacity: 48 GB Bandwidth: 1,344 GB/s	TF32: 126 TFLOPS FP32: 52 TFLOPS FP16/BF16: 266 TFLOPS FP8/INT8: 533 TFLOPS FP4: 970 TFLOPS RT core: 196 TFLOPS	3 × Video Encoder 3 × Video Decoder	PCIe Gen5 x16: 128 GB/s Supports P2P	Supports DX12, OpenGL 4.6, Vulkan 1.3, CUDA 12.8, OpenCL 3.0, and DirectCompute

  • Processor: AMD Turin-C processors with a frequency range of 3.3 GHz to 5 GHz and an all-core turbo of up to 4.2 GHz.

• Storage:

  • I/O-optimized instance.

  • Supports the NVMe protocol. For more information, see NVMe protocol overview.

  • Supported cloud disk types: elastic temporary disk, ESSD, ESSD AutoPL, and ESSD with zone-redundancy. For more information about block storage, see Block storage overview.

• Network:

  • Supports IPv4 and IPv6. For information about IPv6 communication, see IPv6 communication.

  • Delivers network performance of up to 30 million Network PPS.

  • Supports eRDMA, which enables RDMA pass-through acceleration over VPC networks. eRDMA increases bandwidth to 360 Gbit/s and is ideal for training workloads in autonomous driving, embodied intelligence, computer vision (CV), and traditional models.

    Note

    For instructions on how to use eRDMA, see Enable eRDMA for an enterprise-level instance or Enable eRDMA for a GPU instance.

The following table lists the specifications for the ebmgn9g instance family.

GPU elastic bare metal server instance family ebmgn9ge

• Introduction: The ebmgn9ge instance family is Alibaba Cloud's 9th generation of full-featured, cost-effective GPU bare metal instances. Powered by the latest CIPU 2.0, these instances combine high-frequency CPUs, large memory capacity, and new Blackwell architecture professional GPUs. They deliver high-performance and cost-effective GPU cloud services for a wide range of accelerated workloads, including autonomous driving and embodied intelligence training, large model inference, film and animation rendering, and metaverse and cloud gaming services.

• Use cases and features:

  • Autonomous driving and embodied intelligence:
    With 256 vCPUs running at an all-core frequency of over 4.2 GHz and 2.3 TB of memory, these instances meet the demanding data processing needs of autonomous driving and embodied intelligence training.

  • Search and recommendation:
    The Blackwell GPU provides 126 TFLOPS of TF32 computing power. Each GPU is paired with 32 vCPUs and 153 GB/s of memory bandwidth, providing an optimal configuration for search and advertising services.

  • Large model inference:

    The ebmgn9ge instance family is specifically designed for large language models, offering 72 GB of GPU memory per GPU. The GPU memory bandwidth reaches 1,344 GB/s, delivering high-performance inference for LLM scenarios. Combined with the new FP4 compute architecture and 128 GB/s of PCIe Gen5 bandwidth, an instance can support parallel inference for models larger than 671 billion parameters across 8 GPUs.

  • Cloud gaming, rendering, and metaverse:
    With a CPU frequency of up to 5 GHz, these instances are well-suited for 3D modeling. The GPU natively supports graphics capabilities and provides certified workstation-grade graphics drivers with full OpenGL acceleration, making them ideal for high-end film and animation development and CAD design.

• Powered by the latest CIPU 2.0:

  The 2nd generation CIPU delivers greater cloud processing power, enhancing performance for eRDMA, VPC, and EBS components. Bare metal instances allow workloads to directly access physical resources or meet requirements such as hardware-based licensing. They also support containers, such as Docker, Clear Containers, and Pouch.

• Compute:

  • Powered by new Blackwell architecture professional GPUs:

    • Supports professional-grade OpenGL graphics processing.

    • Supports common acceleration features like RTX and TensorRT, and adds support for FP4 compute and the PCIe Gen5 interconnect.

    • Utilizes a PCIe switch for interconnection, which improves NCCL performance by 36% compared to a direct-to-CPU connection. This can boost performance by up to 9% for large model inference with multi-GPU sharding.

  • GPU key specifications:

    GPU architecture	GPU memory	Compute performance	Video encode/decode engines	Inter-GPU interconnect	Accelerated APIs
    Blackwell	Capacity: 72 GB Bandwidth: 1,344 GB/s	TF32: 126 TFLOPS FP32: 52 TFLOPS FP16/BF16: 266 TFLOPS FP8/INT8: 533 TFLOPS FP4: 970 TFLOPS RT core: 196 TFLOPS	3 × Video Encoder 3 × Video Decoder	PCIe Gen5 x16: 128 GB/s Supports P2P	Supports DX12, OpenGL 4.6, Vulkan 1.3, CUDA 12.8, OpenCL 3.0, and DirectCompute

  • Processor: AMD Turin-C processors with a frequency range of 3.3 GHz to 5 GHz and an all-core turbo of up to 4.2 GHz.

• Storage:

  • I/O-optimized instance.

  • Supports the NVMe protocol. For more information, see NVMe Protocol Overview.

  • Supported cloud disk types: elastic temporary disk, ESSD, ESSD AutoPL disks, and ESSD Zone-Redundant Disk. For more information about block storage, see Block Storage Overview.

• Network:

  • Supports IPv4 and IPv6. For information about IPv6 communication, see IPv6 Communication.

  • Delivers network performance of up to 30 million network packets per second (PPS).

  • Supports ERI (Elastic RDMA Interface), which enables RDMA pass-through acceleration over VPC networks. ERI increases bandwidth to 360 Gbit/s and is ideal for training workloads in autonomous driving, embodied intelligence, computer vision (CV), and traditional models.

    Note

    For instructions on how to use ERI, see Enable on enterprise-level instance or Enable eRDMA on GPU instances.

The following table lists the instance types and specifications for the ebmgn9ge instance family.

GPU-accelerated elastic bare metal instance family ebmgn9gc

• Introduction: The ebmgn9gc instance family is Alibaba Cloud's 9th generation of full-featured, cost-effective GPU bare metal instances. Powered by the latest CIPU 2.0, these instances combine high-frequency CPUs, large-capacity memory, and new Blackwell architecture professional GPUs to deliver cost-effective GPU cloud services for a wide range of accelerated workloads, including autonomous driving and embodied intelligence training, large model inference, film and animation rendering, and metaverse and cloud gaming services.

• Use cases and features:

  • Autonomous driving and embodied intelligence:
    Offers 256 vCPUs with a frequency range of 3.3 GHz to 5.0 GHz and an all-core turbo of up to 4.2 GHz. Paired with 1.5 TB of memory, these instances meet the data processing demands of autonomous driving and embodied intelligence training.

  • Search and recommendation:
    The equipped Blackwell GPUs provide 126 TFLOPS of high-performance TF32 compute. Each GPU is paired with 32 vCPUs and 153 GB/s of memory bandwidth, providing an optimal configuration for search and advertising services.

  • Large model inference:

    The ebmgn9gc instances are designed for large language models. With 72 GB of GPU memory and 1,344 GB/s of memory bandwidth per GPU, they deliver high-performance inference for LLM scenarios. Combined with the new FP4 compute architecture and 128 GB/s of PCIe Gen5 bandwidth, an 8-card instance can support parallel inference for models with over 671 billion parameters.

  • Cloud gaming, rendering, and metaverse:
    With a CPU frequency of up to 5.0 GHz, these instances are well-suited for 3D modeling. The GPUs natively support graphics capabilities and provide certified workstation-grade graphics drivers with full OpenGL acceleration, making them a good choice for high-end film and animation development and CAD design.

• Powered by the latest CIPU 2.0:

  The 2nd generation CIPU offers higher cloud processing power and enhanced computing capabilities for eRDMA, VPC, and block storage. Bare metal instances allow workloads to directly access physical resources or meet requirements such as hardware-based licensing. They also support containers, such as Docker, Clear Container, and Pouch.

• Compute:

  • Powered by new Blackwell architecture professional GPUs:

    • Supports professional-grade OpenGL graphics processing features.

    • Supports common acceleration features such as RTX and TensorRT, and adds support for FP4 and PCIe Gen5 interconnect.

    • Utilizes a PCIe switch for interconnection, which improves NCCL performance by 36% compared to a direct-to-CPU connection. This can boost performance by up to 9% for large model inference with multi-card sharding.

  • GPU key parameters:

    GPU architecture	GPU memory	Compute performance	Video encoding/decoding	GPU-to-GPU interconnect	Acceleration APIs
    Blackwell	Capacity: 72 GB Bandwidth: 1,344 GB/s	TF32: 126 TFLOPS FP32: 52 TFLOPS FP16/BF16: 266 TFLOPS FP8/INT8: 533 TFLOPS FP4: 970 TFLOPS RT Core: 196 TFLOPS	3 × Video Encoder 3 × Video Decoder	PCIe Gen5 x16: 128 GB/s Supports P2P	Supports DX12, OpenGL 4.6, Vulkan 1.3, CUDA 12.8, OpenCL 3.0, and DirectCompute

  • Processor: AMD Turin-C processor with a frequency range of 3.3 GHz to 5.0 GHz and an all-core turbo of up to 4.2 GHz.

• Storage:

  • I/O optimized instance.

  • Supports the NVMe protocol. For more information, see NVMe protocol overview.

  • Supported cloud disk types: Elastic Temporary Disk, ESSD, ESSD AutoPL, and ESSD Zone-Redundant Disk. For more information about block storage, see Block Storage Overview.

• Network:

  • Supports IPv4 and IPv6. For information about IPv6 communication, see IPv6 communication.

  • Delivers network performance of up to 30 million packets per second (PPS).

  • Supports elastic RDMA (eRDMA), which enables RDMA pass-through acceleration over VPC networks. eRDMA increases bandwidth to 360 Gbit/s and is ideal for training workloads in autonomous driving, embodied intelligence, computer vision (CV), and traditional models.

    Note

    For instructions on how to use eRDMA, see Enable on enterprise-level instance or Enable eRDMA on GPU instances.

The following table lists the instance types and specifications for the ebmgn9gc instance family.

GPU compute-optimized elastic bare metal server instance family ebmgn8v

This instance family is currently available only in select regions. To request access, contact your Alibaba Cloud sales representative.

• Instance family overview: ebmgn8v is Alibaba Cloud's 8th-generation accelerated computing instance family (Elastic Bare Metal Server instance family), designed for AI model training and ultra-large parameter models. Each instance is a bare metal host equipped with eight GPU cards.

• Use cases:

  • Cost-effective for multi-GPU parallel inference on large language models (LLMs) with more than 70 billion parameters.

  • Each GPU delivers 39.5 TFLOPS of FP32 compute, making it ideal for traditional AI model training and autonomous driving workloads.

  • NVLINK interconnection among the eight GPUs supports small- and medium-sized model training scenarios.

• Key features and positioning:

  • High-speed, large-capacity GPU memory: Each GPU has 96 GB of HBM3 memory, delivering a memory bandwidth of 4 TB/s. This significantly accelerates model training and inference.

  • High inter-GPU bandwidth: The GPUs are interconnected via 900 GB/s NVLINK, providing significantly higher efficiency for multi-GPU training and inference compared to previous GPU generations.

  • Large model quantization technology: Supports FP8 compute, which optimizes performance for training and inference with large-scale parameters. This significantly increases computational speed and reduces GPU memory usage.

• Compute:

  • Powered by the latest CIPU (Cloud Processing Unit) 1.0:

    • Decouples compute and storage, enabling flexible selection of storage resources. Compared to 7th-generation GPU instances, the inter-host bandwidth is increased to 160 Gbit/s for faster data transfer and processing.

    • Unlike traditional virtualized instances, the CIPU provides bare metal capabilities that support P2P communication between GPU instances.

  • Powered by 4th Generation Intel® Xeon® Scalable processors, providing 192 vCPUs with an all-core turbo frequency of up to 3.1 GHz.

• Storage:

  • These are I/O optimized instances.

  • These instances support the NVMe protocol. For more information, see Overview of the NVMe protocol.

  • Supported cloud disk types: elastic temporary disks, ESSD cloud disks, ESSD AutoPL cloud disks, and zone-redundant ESSD cloud disks. For more information about cloud disks, see Block Storage Overview.

• Network:

  • These instances support IPv4 and IPv6. For information about IPv6 communication, see IPv6 communication.

  • Delivers ultra-high network performance of up to 30 million PPS.

  • Supports Elastic RDMA Interface (ERI), which enables RDMA pass-through acceleration over VPC networks. ERI increases bandwidth to 160 Gbit/s and is ideal for computer vision (CV) and traditional model training workloads.

    Note

    For instructions on how to use ERI, see Enable on enterprise-level instance or Enable eRDMA on GPU instances.

The following table describes the instance types and specifications of the ebmgn8v instance family.

GPU-accelerated ebmgn8ia instance family

This instance family is currently available only in select regions. To request access, contact your Alibaba Cloud sales representative.

• Introduction: The ebmgn8ia, an elastic bare metal instance family, is Alibaba Cloud's eighth-generation accelerated computing instance family. It is designed for search and recommendation, simulation, and other GPU-intensive workloads that require a high vCPU-to-GPU ratio. Powered by the latest NVIDIA L20 GPUs, each instance is a bare metal server equipped with two high-frequency processors and four GPUs.

• Features and use cases:

  • High frequency: These instances are equipped with two AMD EPYC™ Genoa 9T34 processors. Each processor has 64 physical cores, providing a total of 256 vCPUs per instance, with a frequency range of 3.4 GHz to 3.75 GHz. This significantly boosts single-core CPU performance, making it ideal for CAD modeling and accelerating preprocessing for CAE simulations.

  • Sparse resource ratio: Each GPU is paired with 64 vCPUs and 384 GiB of memory, delivering an average memory bandwidth of 230 GB/s per GPU. This configuration is suitable for GPU computing scenarios that require high I/O throughput, such as advertising, search, recommendation, traditional CAE simulations, and certain CPU-based rendering tasks in film production.

• Powered by the latest CIPU 1.0:

  • Decouples compute and storage, allowing you to flexibly select storage resources. Compared to the previous generation, the inter-machine bandwidth of this instance type is increased to 160 Gbit/s for faster data transfer and processing.

  • The CIPU provides bare metal capabilities, enabling PCIe P2P communication between GPU instances, an improvement over traditional virtualized instances.

• Compute:

  • Powered by new enterprise-class NVIDIA L20 GPUs:

    • Supports common acceleration features such as vGPU, RTX, and TensorRT.

    • Supports FP8 precision to improve computational efficiency.

  • NVIDIA L20 key specifications:

    GPU architecture	GPU memory	Compute performance	Video encode/decode engines	GPU interconnect
    NVIDIA Ada Lovelace	Capacity: 48 GB Bandwidth: 864 GB/s	FP64: N/A FP32: 59.3 TFLOPS FP16/BF16: 119 TFLOPS FP8/INT8: 237 TFLOPS	3 × Video Encoder (+AV1) 3 × Video Decoder 4 × JPEG Decoder	PCIe interface: PCIe Gen4 x16 Bandwidth: 64 GB/s

  • Processor: AMD EPYC™ Genoa 9T34 processors with a frequency range of 3.4 GHz to 3.75 GHz.

• Storage:

  • These are I/O optimized instances.

  • These instances support the NVMe protocol. For more information, see Overview of the NVMe protocol.

  • Supported cloud disks: elastic temporary disk, ESSD cloud disk, ESSD AutoPL cloud disk, and ESSD Zone-redundant cloud disk. For more information, see Block storage overview.

• Network:

  • These instances support IPv4 and IPv6. For information about IPv6 communication, see IPv6 communication.

  • Delivers ultra-high network performance of up to 30,000,000 PPS.

  • Supports Elastic RDMA Interface (ERI), which enables RDMA passthrough over VPC networks. ERI increases bandwidth to 160 Gbit/s and is ideal for training computer vision (CV) and traditional models.

    Note

    For instructions on using ERI, see Enable eRDMA on an enterprise-level instance or Enable eRDMA on a GPU-accelerated instance.

GPU compute-optimized elastic bare metal family ebmgn8is

This instance family is currently available only in select regions outside the Chinese mainland. For inquiries, contact your Alibaba Cloud sales representative.

• Instance family overview: The ebmgn8is is the 8th-generation accelerated computing instance family from Alibaba Cloud. As an elastic bare metal instance family, it is built for the growing demand in AI-generated workloads. Powered by the latest NVIDIA L20 GPUs, each instance is a bare metal host with eight GPU cards.

• Features and positioning:

  • Graphics processing: These instances are powered by high-frequency 4th Generation Intel® Xeon® Scalable processors. They provide ample CPU computing power for 3D modeling, ensuring smoother graphics rendering and design.

  • Inference tasks: Powered by the new NVIDIA L20 GPUs, where each card provides 48 GB of memory to accelerate inference tasks. These instances support the FP8 floating-point format. Combined with Alibaba Cloud Container Service for Kubernetes (ACK), they flexibly support inference on various AIGC models and are especially suitable for inference tasks on large language models (LLMs) with up to 70 billion parameters.

  • Training tasks: These instances offer cost-effective computing power, delivering double the FP32 compute performance compared to 7th-generation inference instances. They are particularly suitable for training computer vision (CV) models that use FP32, and for training other small to medium-sized models.

• Use cases:

  • Use GRID images with GRID graphics drivers from the Cloud Marketplace to enable OpenGL and Direct3D graphics capabilities. This provides workstation-grade graphics processing for workloads such as animation, film and television special effects, and rendering.

  • Combined with the containerized management capabilities of ACK, you can more efficiently and cost-effectively support AIGC graphics generation and large language model (LLM) inference (up to 130 billion parameters).

  • Other general AI scenarios, such as image recognition and speech recognition.

• Powered by the latest CIPU 1.0:

  • Features decoupled compute and storage, which allows you to flexibly select the required storage resources. Compared with the previous generation, the inter-instance bandwidth of this instance family is increased to 160 Gbit/s to enable faster data transfer and processing.

  • CIPU provides bare metal capabilities that enable PCIe P2P communication between GPUs, which is not supported by traditional virtualized instances.

• Compute:

  • Powered by new enterprise-class NVIDIA L20 GPUs:

    • Supports common acceleration features such as vGPU, RTX, and TensorRT.

    • Uses a PCIe switch for interconnection. Compared with a direct-to-CPU connection, this design improves NCCL performance by 36% and can increase inference performance by up to 9% when you run sharded inference for large models across multiple cards.

  • Key NVIDIA L20 parameters:

    GPU architecture	GPU memory	Compute performance	Video encoding/decoding	Inter-GPU interconnect
    NVIDIA Ada Lovelace	Capacity: 48 GB Bandwidth: 864 GB/s	FP64: N/A FP32: 59.3 TFLOPS FP16/BF16: 119 TFLOPS FP8/INT8: 237 TFLOPS	3 × Video Encoder (+AV1) 3 × Video Decoder 4 × JPEG Decoder	PCIe interface: PCIe Gen4 x16 Bandwidth: 64 GB/s

  • Processor: Intel^® Xeon^® Scalable processors (SPR) with a 3.4 GHz base frequency and an all-core turbo frequency of up to 3.9 GHz.

• Storage:

  • These are I/O optimized instances.

  • These instances support the NVMe protocol. For more information, see Overview of the NVMe protocol.

  • Supported cloud disk types: elastic ephemeral disks, ESSDs, ESSD AutoPLs, and zone-redundant ESSDs. For more information about cloud disks, see Block storage overview.

• Network:

  • These instances support IPv4 and IPv6. For information about IPv6 communication, see IPv6 communication.

  • Delivers ultra-high network performance with a packet forwarding rate of 30 million PPS.

  • Supports ERI (Elastic RDMA Interface), which enables RDMA passthrough acceleration in a VPC network. ERI increases bandwidth to 160 Gbit/s and is ideal for CV and traditional model training workloads.

    Note

    To use ERI, see Enable on enterprise-level instance or Enable eRDMA for a GPU instance.

Ebmgn7e instance family

After starting an ebmgn7e instance, you must check if the Multi-Instance GPU (MIG) feature is enabled or disabled because its default state is not guaranteed. For more information about MIG, see NVIDIA Multi-Instance GPU User Guide.

The following table describes whether ebmgn7e instances support the MIG feature.

GPU-accelerated ebmgn7i instance family

GPU-accelerated ebmgn7 instance family

The Multi-Instance GPU (MIG) feature may be either enabled or disabled by default when an ebmgn7 instance starts. You must check its status and enable or disable it as needed. For more information about MIG, see the NVIDIA Multi-Instance GPU User Guide.

The following table describes MIG support for ebmgn7 instances.

Ebmgn6e GPU-accelerated instance family

Ebmgn6v instance family

GPU compute instance family ebmgn6i

sccgn7ex, GPU-accelerated compute-optimized SCC instance family