Instance scaling limits and elastic policies

Scenarios

You can use this policy when your function has predictable traffic patterns or periodic peaks. If the invocation concurrency exceeds the capacity of the minimum instances, the system automatically scales out additional elastic instances.

Configuration example

You can configure two scheduled actions. The first action scales out the minimum number of instances before traffic increases. The second action scales in the minimum number of instances after traffic decreases. The following figure illustrates this process.

The following example shows how to use the PutProvisionConfig API to configure scheduled scaling parameters. This policy is for a function named function_1. The time zone is set to Asia/Shanghai (UTC+8). The policy is active from 2024-08-01 10:00:00 to 2024-08-30 10:00:00 (UTC+8). Every day at 20:00 (UTC+8), the minimum number of instances scales out to 50. At 22:00 (UTC+8), it scales in to 10.

"scheduledActions": [
    {
      "name": "scale_up_action",
      "startTime": "2024-08-01T10:00:00",
      "endTime": "2024-08-30T10:00:00",
      "target": 50,
      "scheduleExpression": "cron(0 0 20 * * *)",
      "timeZone": "Asia/Shanghai"
    },
    {
      "name": "scale_down_action",
      "startTime": "2024-08-01T10:00:00",
      "endTime": "2024-08-30T10:00:00",
      "target": 10,
      "scheduleExpression": "cron(0 0 22 * * *)",
      "timeZone": "Asia/Shanghai"
    }
  ]

Cron expression description

Scenarios

Function Compute periodically collects key metrics and automatically scales the minimum number of instances up or down based on the configured Range for the Minimum Number of Instances. This helps align the instance count with actual resource usage. This policy is ideal for functions with unpredictable traffic patterns because it helps maintain a stable level of resource utilization.

Key metric descriptions

Metric-based scaling policies automatically adjust the minimum number of instances by tracking the following key metrics. Choosing the right metric is crucial for achieving efficient elasticity.

• Instance concurrency utilization

  • Definition: The ratio of the total number of concurrent requests being processed by all provisioned instances (instances within the minimum instance range) to the maximum total number of concurrent requests that these instances can handle, measured over a collection period.

  • Formula: Total current concurrent requests / (Current minimum number of instances × Concurrency per instance)

  • Scenarios: Suitable for most general-purpose web services, API gateways, and other I/O-intensive or CPU-intensive businesses where the primary bottleneck is request processing capacity.

• Memory utilization

  • Definition: The memory usage of all provisioned instances during the collection period.

  • Formula: Average memory used / Function memory configuration.

  • Scenarios: Suitable for memory-intensive businesses, such as big data processing, image transformation, and deep learning model pre-processing, where the performance bottleneck is memory consumption rather than request concurrency.

• GPU resource utilization

  • Definition: For GPU-accelerated instances, you can track more granular GPU resource usage, including GPU utilization and GPU memory utilization.

    • GPU utilization: Reflects how busy the GPU computing cores are.

    • GPU memory utilization: Reflects the usage of GPU memory.

  • Scenarios: Specifically for functions that require GPU acceleration, such as AI inference and scientific computing. You can choose the appropriate metric for scaling based on whether the model depends more on compute resources or GPU memory.

Configuration example

This example uses the Instance Concurrency Utilization metric. When traffic increases and triggers the scale-out threshold, the number of instances scales out until it reaches the upper limit of the configured range. Requests that exceed this capacity are allocated to on-demand elastic instances. When traffic decreases and triggers the scale-in threshold, the number of instances begins to scale in. The following figure illustrates this process.

The following example shows how to use the PutProvisionConfig API to configure a metric-based scaling policy for a function named function_1. The policy is active from 2024-08-01 10:00:00 to 2024-08-30 10:00:00 in the Asia/Shanghai (UTC+8) time zone. The policy tracks the ProvisionedConcurrencyUtilization metric with a target Instance Concurrency Utilization of 60%. The system scales out to a maximum of 100 when utilization exceeds 60% and scales in to a minimum of 10 when utilization falls below 60%.

"targetTrackingPolicies": [
    {
      "name": "action_1",
      "startTime": "2024-08-01T10:00:00",
      "endTime": "2024-08-30T10:00:00",
      "metricType": "ProvisionedConcurrencyUtilization",
      "metricTarget": 0.6,
      "minCapacity": 10,
      "maxCapacity": 100,
      "timeZone": "Asia/Shanghai"
    }
  ]