Quota scheduling policies

更新时间:
复制 MD 格式

MaxCompute subscription computing resources are limited. When multiple jobs run concurrently, a scheduling policy allocates these resources. Configuring the appropriate policy ensures that your computing resources are allocated effectively.

Usage notes

  • In resource scheduling, job priority takes precedence over the quota scheduling policy. Resources are always allocated to jobs with higher priority first. For jobs with the same priority, resources are then allocated based on the configured scheduling policy.

  • You cannot configure a scheduling policy for a level-2 quota of the Interactive type.

  • As of May 24, 2023, the default scheduling policy for new level-2 quotas in all regions is FIFO. For level-2 quotas created before this date, the default policy is FAIR. For more information, see the announcement.

Overview

Feature

FAIR

FIFO

Scheduling behavior

  • For jobs with the same priority, resources are distributed evenly among all jobs running concurrently.

  • For jobs with different priorities, resources are first distributed evenly to higher-priority jobs. If any resources remain, they are then distributed evenly to lower-priority jobs.

  • For jobs with the same priority, resources are allocated to the first job submitted.

  • For jobs with different priorities, resources are always allocated to the higher-priority job, even if it was submitted after a lower-priority job.

Use cases

  • Interactive scenarios: Multiple data engineers submit jobs concurrently and need to track the progress of all jobs.

  • Multi-baseline dependencies: Multiple baselines share a quota's resources, and all baseline tasks must run concurrently to ensure timely job output.

Batch workloads: Multiple jobs form a single, larger task where overall throughput and final completion time are more important than the completion time of any individual job.

Advantages

Multiple jobs can run concurrently.

Shorter average runtime for individual jobs.

Risks

Longer average runtime for individual jobs.

A large job might consume most of the resources, causing other jobs in the same quota to queue.

Note
  • You can check the job status in Logview. If the sub-status is Waiting for cluster resource, it means the job is queued and awaiting compilation.

  • You can mitigate this risk by setting a per-job compute unit limit. For instructions, see Configure quotas.

策略特征

Configure a scheduling policy

  • Log on to the MaxCompute console and select a region in the upper-left corner.

  • In the left-side navigation pane, click Quotas.

  • On the Quotas page, find the level-1 quota you want to configure and click Quota Configuration in the Actions column.

    Note

    Only subscription quotas for standard computing resources have the Quota Configuration option.

  • On the Quota Configuration page, on the Basic Configurations tab, click Edit Basic Configurations. In the Scheduling Policy column, select the appropriate policy for the level-2 quota.

    Parameter

    Description

    Scheduling policy

    The scheduling policies for a MaxCompute level-2 quota include FAIR and FIFO.

  • Click OK. After the configuration is saved, click Apply Immediately in the Actions column for the change to take effect.

Related documentation

For more information about quota settings, see Quota management for computing resources.