Consumption monitoring alert

Updated at:

To prevent unexpected charges when using pay-as-you-go MaxCompute resources due to higher-than-expected consumption from computing tasks, you can set consumption monitoring thresholds for cost control. If resource usage or spending exceeds a preset threshold, the system automatically triggers an alert and notifies you by text message, email, DingTalk, or phone, allowing you to respond promptly and mitigate potential cost risks.

Features

MaxCompute pay-as-you-go resources scale elastically with no preset limits on resource usage. To avoid unexpected costs, MaxCompute provides the following consumption control features:

Feature

Control method

Description

Per-statement resource limit

Limit resource consumption

MaxCompute estimates the resource consumption of an SQL statement before it runs. If the estimated consumption exceeds a preset threshold, the statement is blocked, and the system returns a failure status and an error message. This feature helps prevent a single SQL statement from incurring high costs.

Real-time consumption monitoring alert

Cost alert

You can configure real-time consumption monitoring and alerts for unbilled standard SQL and MapReduce computing tasks in Cloud Monitor. The system aggregates cumulative daily or monthly consumption at the project level. For more information about billing, see Compute fees (pay-as-you-go). When cumulative consumption exceeds a preset threshold, the system notifies you by phone, text message, email, or DingTalk.

Note

Real-time consumption refers only to metered usage that has not yet been billed. It does not include any discounts or promotions.

Daily cumulative spend limit

Cost alert

Monitors the daily cumulative consumption of standard SQL in a project. If the sum of the day's accumulated consumption and the estimated consumption of a new SQL statement exceeds the preset threshold, the new statement is blocked. The system returns a failure status and an error message. This feature helps prevent daily standard SQL consumption from exceeding your budget.

Note

Daily cumulative SQL consumption refers only to metered usage that has not yet been billed. It does not include any discounts or promotions.

High-consumption alert

Cost alert

You can set up alerts for billed amounts in the Alibaba Cloud Expenses and Costs console. MaxCompute pay-as-you-go charges are billed daily, so a day's consumption appears on the following day's bill. This method monitors only historical spending. If a daily bill exceeds your threshold, the system sends a text message notification at approximately 09:00 the next day.

Per-statement resource limit

Configuration

Set a per-statement resource limit in either of the following ways:

  • Project-level setting

    The project owner or a user with the Super_Administrator role must run the following command to enable or disable the per-statement resource limit.

    -- Enable the per-statement resource limit.
    SETPROJECT odps.sql.metering.value.max=<m_value>;
    -- Disable the per-statement resource limit.
    SETPROJECT odps.sql.metering.value.max;

    The m_value parameter is the maximum consumption threshold for a single SQL statement. It is calculated as SQL read volume (GB) × SQL complexity and is not a monetary amount.

    Note

    A project's per-statement consumption is constrained by m_value. If a statement's estimated consumption exceeds this m_value, the system blocks its execution.

  • Session-level setting

    To enable a per-statement resource limit for a single execution, submit the following command along with your SQL statement. If you do not include this command, the limit is disabled. This setting is valid only for the current execution.

    SET odps.sql.metering.value.max=<m_value>;

Recommendations

Follow these best practices:

  • Set a project-level m_value based on historical SQL consumption and your expected spending.

    • You can download daily detailed metering data and calculate the resource consumption for SQL tasks (tasks with a data classification of ComputationSql) at the project level. Resource consumption is calculated as SQL read volume (GB) × SQL complexity. Determine a suitable value based on the highest normal daily consumption, factoring in business fluctuations.

    • Alternatively, you can convert compute consumption to a cost (SQL read volume (GB) × SQL complexity × 0.3). Based on your business needs, determine an acceptable cost, and then convert that cost back into a consumption value: m_value = cost amount (in CNY) / 0.3.

  • Set a project-level limit first, then use a session-level m_value limit for specific, exceptional SQL statements.

    Note
    • Before setting a session-level limit, confirm that the specific SQL statement truly requires it to avoid misuse.

    • If both project-level and session-level limits are set, the session-level limit takes precedence.

      For example, if a project's m_value is set to 100, but a specific SQL statement within that project has a session-level m_value of 200, the statement is blocked only if its estimated consumption exceeds 200, not 100. Other SQL statements without a session-level limit still adhere to the project's limit of 100 and are blocked if their estimated consumption exceeds 100.

  • Setting either a project-level or session-level limit affects task execution. Use these limits with a task failure alert feature, such as DataWorks intelligent monitoring, to detect task failures promptly and intervene manually. This prevents critical tasks from being blocked and impacting business operations.

Relying solely on the per-statement resource limit may not fully prevent unexpectedly high bills. Consider also using the real-time consumption monitoring alert to monitor the cumulative consumption of SQL and MapReduce tasks in your project.

Real-time consumption monitoring alert

  1. Log on to the Cloud Monitor console.

  2. Create an alert contact.

    1. In the left-side navigation pane, choose Alerts > Alert Contacts.

    2. On the Alert Contacts page, click Create Alert Contact.

    3. In the Set Alert Contact panel, enter the name, mobile number, and email address.

    4. After you configure the contact, click OK.

  3. Create an alert contact group.

    1. On the Alert Contacts page, click the Alert Contact Group tab.

    2. On the Alert Contact Group page, click Create Contact Group.

    3. In the Create Contact Group panel, enter the Group Name and Description, and select contacts from the Existing Contacts list to add them to the Selected Contacts list.

    4. After you configure the group, click OK.

  4. Configure a threshold alert rule.

    For detailed instructions, see Create an alert rule. Key parameters are described in the following table.

    Parameter

    Description

    Service

    Select MaxCompute-General.

    Resource Range

    Select All Resources.

    Rule Description

    Based on your business needs, select Cost > Daily/Monthly consumption of pay-as-you-go jobs.

    For more information about monitoring metrics, see Monitoring and alerts.

Daily cumulative spend limit

Configuration

Set a daily cumulative spend limit for a project by running the following command. Only the project owner or a user with the Super_Administrator role can run this command.

SETPROJECT odps.costcontrol.rule={"byDate":{"sql":<Limit>}};
  • byDate: Sets a daily limit. The cumulative consumption is calculated for the period from 00:00:00 to 23:59:59 (UTC+8). The counter resets on the next day.

  • Limit: The daily cumulative spending threshold for standard SQL in the project, specified in CNY. This consumption tracking does not include any discounts or promotions.

    Note

    Cost per SQL statement = Scanned data volume (GB) × Complexity × 0.3

Usage notes

After you set the limit, the following rules apply:

Rule type

System behavior

Example and recommendation

First-time setup

The system accumulates the total daily consumption of standard SQL in the project and estimates the consumption of newly submitted standard SQL statements.

Example: When you first set the project-level limit by running SETPROJECT odps.costcontrol.rule={"byDate":{"sql":100}};, the system starts accumulating the daily total and estimating the cost of new tasks.

Updating an existing value

The system compares the new consumption estimate against the accumulated value and the new limit to determine whether to block the task.

Suppose the original daily limit was 100 and the current accumulated consumption is 99. You change the limit to 150 and then submit a new standard SQL statement.

The system first estimates the cost of the new SQL statement (for example, 20). If the sum of the estimated cost and the accumulated consumption (20 + 99) is less than or equal to 150, the SQL statement runs. Otherwise, it is blocked.

Sum of daily accumulated and estimated consumption exceeds the Limit

The system blocks the newly submitted standard SQL task from executing and returns an error message.

  • Error example:

    After you set a consumption limit using the setproject odps.costcontrol.rule={"byDate":{"sql":100}}; command, if the daily cumulative consumption of standard SQL in the project exceeds the limit, the task fails with the following error:

    Exceed Cost Limit : 
    {"AlreadyCost":"100.1","InstanceId":"xxx","Limit":"100","Project":"xxx","TaskType":"SQL","ThisTaskWillCost":"0","TimeWindow":"BYDATE"}
  • Recommendation:

    Use this feature with a task failure alert function, such as DataWorks intelligent monitoring, to be promptly notified of task failures. This allows you to intervene manually and prevent critical tasks from being blocked and impacting your business.

High-consumption alert

Set a high-consumption alert for MaxCompute pay-as-you-go services in the Alibaba Cloud Expenses and Costs console.

Note

For information about how to set a high-consumption alert in the new Expenses and Costs console, see Cost alert.

  1. Log on to Expenses and Costs.

  2. On the Cost Monitoring > Cost Alerting page, next to Daily Bill Alert, click Details.

  3. On the Daily Bill Alert page, select an Alerting Commodity, enter an Threshold, and click Increase.