This topic describes offline asynchronous task scenarios for GPU-accelerated instances. It explains how to use the asynchronous invocation and asynchronous task features of Function Compute for tasks such as offline AI inference and AI training. It also covers how to use custom images for GPU-accelerated offline applications.
Scenarios
In offline asynchronous application scenarios, workloads have one or more of the following features.
Long running time
The processing time for a task typically ranges from minutes to hours. Response time is not a critical factor.
Immediate return after submission
After a task is invoked, a response is returned immediately. This prevents the main business logic from being blocked by long-running processes.
Real-time task status awareness
Offline GPU tasks require the ability to monitor their running status and be canceled mid-execution.
Parallel processing
Offline GPU tasks often process large amounts of data and require significant GPU resources. Parallel processing can speed up execution.
Data source integration
Offline GPU tasks have diverse data source requirements. During processing, they frequently interact with various storage products, such as Object Storage Service (OSS), and messaging products, such as MNS. For more information, see Introduction to triggers.
Function Compute provides the following advantages for offline asynchronous application workloads.
Simplified business architecture
Logic that is time-consuming, resource-intensive, or error-prone can be separated from the main flow and processed asynchronously. This improves system response speed, resource utilization, and availability.
Shortest path for an asynchronous task
You can use the asynchronous GPU processing capabilities of Function Compute to build a cost-effective asynchronous processing platform for AI services.
Sufficient GPU resource supply
Function Compute provides an ample supply of GPU resources. When your application encounters large-scale offline tasks, Function Compute can elastically provision a massive amount of GPU computing power in seconds. This helps prevent business losses that can be caused by an insufficient GPU supply or delays in scaling. This feature is ideal for offline services with fluctuating or unpredictable traffic patterns.
Data source integration
Function Compute supports various data source triggers, such as OSS and MNS. This helps simplify the logic for interacting with data sources.
How it works
After you deploy a GPU function, you can submit offline GPU tasks using the asynchronous invocation mode or the asynchronous task mode. Function Compute uses elastic instances by default (in contrast to the snapshot feature) to provide the infrastructure capabilities required for offline asynchronous scenarios.
When the Function Compute platform receives multiple offline GPU tasks submitted asynchronously, it automatically launches multiple elastic GPU instances for parallel processing. The ample supply of GPU computing power ensures that many offline GPU tasks can run in parallel, which minimizes queuing time. If the total number of offline GPU tasks exceeds the GPU resource quota for a single Alibaba Cloud account in a region, the excess tasks are queued. You can easily monitor the number of queued, processing, and completed offline GPU tasks. You can also cancel any offline GPU tasks that are no longer needed. When an offline GPU task is complete, a callback can be triggered to interact with different downstream services based on whether the task succeeded or failed.
Container support
In Function Compute GPU scenarios, only deployments that use custom images are supported. For more information about how to use custom images, see Introduction to custom images.
Functions that use custom images must include a web server. This server is required to handle different code execution paths and to allow the function to be triggered by events or HTTP requests. This approach is suitable for scenarios that involve multi-path request execution, such as AI training and inference.
Deployment methods
You can deploy your models in Function Compute by using one of the following methods:
Use the Function Compute console. For more information, see Create a function in the console.
Call SDKs. For more information, see List of operations by function.
Use Serverless Devs. For more information, see Common Serverless Devs commands.
For more deployment examples, see start-fc-gpu.
Asynchronous mode
Because offline application scenarios involve long running times, you must trigger them using asynchronous invocations. After the function is triggered, a response is returned immediately. A standard asynchronous invocation does not return an execution status. Therefore, you must configure the asynchronous invocation as an asynchronous task. Asynchronous tasks allow you to query the execution status of each request at any time and cancel running requests if necessary. For more information about asynchronous invocations and asynchronous tasks, see Asynchronous invocation and Asynchronous tasks.
Concurrent invocations
The maximum number of concurrent invocations for your GPU function in a region, such as the China (Hangzhou) region, depends on the instance concurrency of the GPU function and the quota for physical GPU cards.
GPU function instance concurrency
By default, the concurrency of a GPU-accelerated instance is 1. This means a single GPU function instance can process only one request or offline GPU task at a time. You can adjust the concurrency configuration of a GPU function instance in the console or using the Serverless Devs tool. For more information, see Set instance concurrency. Choose different concurrency configurations for different application scenarios. For compute-intensive offline GPU tasks, keep the instance concurrency at the default value of 1.
Usage limit of physical GPU cards
By default, the maximum number of physical GPU cards for a single Alibaba Cloud account in a region is 30. The actual value is subject to the information in Quota Center. If you require more physical cards, you can submit a request in Quota Center.
The relationship between GPU instance specifications and instance concurrency is as follows:
An Ada.1 GPU has 48 GB of memory, and a Tesla series GPU has 16 GB of memory. Function Compute allocates the full memory of a GPU card to a single GPU container. Because the default GPU card quota is a maximum of 30 per region, a maximum of 30 GPU containers can run simultaneously in that region.
If the instance concurrency of a GPU function is 1, the function can process up to 30 inference requests concurrently in a region.
If the instance concurrency of a GPU function is 5, the function can process up to 150 inference requests concurrently in a region.
Running time
GPU-accelerated functions in Function Compute support a maximum running time of 86,400 seconds (24 hours). When you use the asynchronous task mode, you can run tasks and cancel them while they are in progress. This is suitable for long-running scenarios such as AI inference, AI training, audio and video processing, and 3D reconstruction.