2023
This topic lists the product announcements for MaxCompute in 2023. Learn about new features, syntax updates, and permission changes to improve your development efficiency.
December 28, 2023: MaxCompute upgrades public cloud endpoints
Dear MaxCompute user,
Thank you for your support of MaxCompute. Effective January 4, 2024, MaxCompute will upgrade its endpoint service architecture and introduce new endpoint addresses in six regions: China (Beijing), China (Shanghai), China (Hangzhou), China (Shenzhen), China (Zhangjiakou), and China (Chengdu). Due to limitations in the legacy architecture, the original endpoints cannot be upgraded or scaled to meet growing business demands. The existing endpoints will remain available, and a deprecation date will be announced separately. We recommend switching to the new endpoints as soon as possible.
If you have any questions or need assistance, contact us through your dedicated DingTalk group or by joining the MaxCompute Developer Community DingTalk group (ID: 11782920).
Thank you for your cooperation.
December 27, 2023: Bandwidth for OSS external tables changes to tenant-specific
Dear MaxCompute user,
Thank you for your support of MaxCompute. We are upgrading the network architecture for data read and write operations on OSS external tables. The upgrade will take effect on January 29, 2024, in the China (Hangzhou) region, and on February 29, 2024, in the China (Shanghai) and China (Beijing) regions. After the upgrade, these operations will use the bandwidth of your tenant or the corresponding bucket. Previously, operations on OSS external tables in these regions used a shared public bandwidth pool provided by MaxCompute, which offered limited isolation and observability. This upgrade enhances data access stability, provides bandwidth isolation between tenants, and allows for flexible configuration of bandwidth resources for external tables.
If you have any questions or need assistance, contact us through your dedicated DingTalk group or by joining the MaxCompute Developer Community DingTalk group (ID: 11782920).
Thank you for your cooperation.
December 27, 2023: Project-level Information Schema not auto-installed for new projects
Dear MaxCompute user,
Thank you for your support of MaxCompute. On August 1, 2023, MaxCompute launched the tenant-level Information Schema, which allows you to query metadata across all your projects at once and provides access to more comprehensive metadata. Therefore, starting March 1, 2024, MaxCompute will no longer automatically install the project-level Information Schema package in new projects. If your business relies on querying metadata, we recommend using the tenant-level Information Schema to access more complete information. For more information, see Tenant-level Information Schema.
If you have any questions or need assistance, contact us through your dedicated DingTalk group or by joining the MaxCompute Developer Community DingTalk group (ID: 11782920).
Thank you for your cooperation.
December 15, 2023: Changes to MaxCompute project lifecycle management
Dear MaxCompute user,
Thank you for your support of MaxCompute. Starting December 25, 2023 (Monday), MaxCompute will roll out an update to the project lifecycle management process. This change will affect projects in the China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Ulanqab), China (Shenzhen), China (Chengdu), Alibaba Finance Cloud, and Alibaba Gov Cloud regions. This change is particularly important for users of subscription computing resources.
Before the update
If a project's default computing resource was a subscription, the project's lifecycle was tied to the subscription's lifecycle. For more information, see Use quotas for computing resources. When the subscription expired, the project was suspended and became unavailable. If not renewed within 15 days, the project was permanently deleted. Unsubscribing from the computing resources also permanently deleted the project.
After the update
If the Standard Edition pay-as-you-go service is activated in the same region as your subscription computing resources, the following policies apply. To activate the service, click Pay-as-you-go.
The project's lifecycle is no longer affected by the lifecycle of its subscription computing resources. An expiring or released subscription will only freeze or release the corresponding quota, without affecting the project's status. This ensures your data is not deleted and remains accessible through other means.
ImportantIf a subscription quota is frozen or released and you do not switch the project's default quota to an active one, you cannot run computing jobs in that project. For more information about how to set or specify a quota, see Use quotas for computing resources.
The project's lifecycle will align with the pay-as-you-go service. If your account has overdue payments, the project will be frozen. If the payment is not settled within 15 days, the project will be permanently deleted.
For postpaid bills related to storage and download usage in the region, the Product Details will change from MaxCompute (Subscription) to MaxCompute (Pay-As-You-Go). Correspondingly, the Billable Item in your Usage Records will also change to MaxCompute (Pay-As-You-Go).
When the Pay-as-you-go Standard Edition service is not enabled in the region of your subscription computing resources, the project lifecycle remains the same as it was before the upgrade. Therefore, we strongly recommend that you click Pay-as-you-go to enable the pay-as-you-go service as soon as possible if you have not yet enabled the Pay-as-you-go Standard Edition service. The service is free to enable. This prevents irreversible data loss that can occur if your data is disabled or deleted when your subscription computing resources expire or are released.
For more information about project lifecycles, see Overdue payments and service suspension and Resource release and service cancellation.
Thank you for your cooperation.
December 4, 2023: Discontinuation of MaxCompute Management
Dear MaxCompute user,
Thank you for your support of MaxCompute. Since the release of the new MaxCompute console in September 2022, we have continuously migrated and enhanced the capabilities of the original MaxCompute Management into the new console, offering an improved user experience and expanded management features. The new console now covers all core functionalities of MaxCompute Management. Consequently, we will stop maintaining MaxCompute Management and begin to phase it out starting January 10, 2024 (Wednesday).
Before the discontinuation, you can still view information in the old MaxCompute Management interface, but we ask that you no longer make any configuration changes there. The original features have been mapped to the new console as follows:
Resource overview: This feature, which shows the usage trend of subscription quotas, is now part of Resource Observation in the new console. Resource Observation covers all original capabilities and adds monitoring for other resources. For more information, see Resource Observation.
Project O&M: The project list view is now available under Projects in the new console. Projects covers all original capabilities. For more information, see Manage projects.
You can now manage level-2 quotas for subscription resources under Quotas in the new console. The Quotas feature includes all original capabilities and adds support for elastically reserved CUs and management of exclusive resource groups for data transmission, all with a significantly improved user experience. For more information, see Manage projects.
You can now view job snapshots and historical jobs in Job O&M in the new console. Job O&M covers all original capabilities. For more information, see Job O&M.
Optimization recommendations (supported in some regions): These capabilities are now fully integrated into DataWorks Data Governance. This includes recommendations for tasks with continuous empty output, continuous full table scans, and tasks suitable for dynamic filtering. For more information, see Output is empty and Brute-force scan.
During this transition, if you have any questions or need assistance, please contact us through your dedicated DingTalk group or by joining the MaxCompute Developer Community DingTalk group (ID: 11782920).
Thank you for your cooperation.
November 21, 2023: RAM permission control for Super_Administrator role
Dear MaxCompute user,
Starting November 21, 2023 (Tuesday), you can configure the Super Administrator role for a project in the MaxCompute console by navigating to Projects > Manage > Parameter Configuration. For more information about this role, see Built-in roles. This operation is controlled by RAM permissions. A RAM user with the UpdateUsersToSuperAdmin permission can assign members to a project's Super_Administrator role. This allows administrators to manage project settings that require Super_Administrator privileges, which addresses cases where a project owner cannot directly access the MaxCompute service to grant permissions for configuring parameters that require the super_administrator role. For more information, see RAM permissions.
Thank you for your cooperation.
September 27, 2023: Discontinuation of select subscription packages and developer editions
Dear MaxCompute user,
Thank you for your support of MaxCompute. To provide more flexible and suitable product offerings, we are adjusting our purchasing options. The following offerings will be discontinued at 00:00:00 on October 31, 2023 (Tuesday): the Storage-Intensive 160, Storage-Intensive 320, and Storage-Intensive 600 packages under the Subscription Package Edition; non-reserved computing resources; and the Pay-as-you-go Developer Edition.
If you are using a Subscription Package Edition, your associated projects will become unavailable after the service is discontinued. Before this date, you must change your Subscription Package Edition to the Subscription Standard Edition.
If you are using non-reserved computing resources, they will become unavailable after the service is discontinued. Before this date, you must reduce your non-reserved computing resources to 0. If you are using the Pay-as-you-go Developer Edition, your associated projects will become unavailable after the service is discontinued. Before this date, you must activate the pay-as-you-go Standard Edition and switch your project's computing quota to the Standard Edition default postpaid quota. To avoid business disruption and continue enjoying the full capabilities of MaxCompute, please complete these transitions as soon as possible. If you have any questions, contact us through your dedicated DingTalk group or join the MaxCompute Developer Community (ID: 11782920) for assistance. If you do not use any of these offerings, you can disregard this notice.
Thank you for your understanding and cooperation.
September 25, 2023: SQL subquery billing and max job runtime updates
Dear MaxCompute user,
Effective December 1, 2023 (Friday), MaxCompute will adjust the following SQL behaviors. This applies to users in regions outside the Chinese mainland, as well as users of Alibaba Finance Cloud and Alibaba Gov Cloud. Please review these changes in advance.
A billing issue for subqueries has been fixed. Previously, the subquery part of an SQL statement containing an IN clause was not metered under the pay-as-you-go model. Starting 10:00:00 on December 1, 2023, IN subqueries will be billed correctly.
The default maximum runtime for SQL jobs will be changed from 72 hours to 24 hours to prevent inactive jobs from occupying system resources for extended periods. This change will take effect at 10:00:00 on December 1, 2023. If your SQL job needs more than 24 hours to run, set the
set odps.sql.job.max.time.hours=72;flag in your job before this change takes effect. The maximum supported runtime is 72 hours.
Thank you for your cooperation.
September 21, 2023: MaxCompute job analysis feature
Dear MaxCompute user,
Starting September 21, 2023 (Thursday), MaxCompute is gradually rolling out the job analysis feature in public cloud regions in the Chinese mainland.
The job analysis feature allows you to view the resource consumption of a specific job and the resource allocation of its computing quota at a given time on the Job O&M page. For more information, see Job O&M. When a job runs longer than expected or its performance is slower than anticipated, you can use the resource consumption tab to check whether the issue is caused by insufficient computing resources or resource contention among multiple jobs. Based on your findings, you can optimize task execution by adjusting job priorities or managing computing resources. For more information, see Job priority or Configure a quota.
For recommended methods to view job resource consumption, see View the resource consumption of a job and the resource allocation of its computing quota at a specific time.
Thank you for your cooperation.
September 15, 2023: MaxCompute Notebook now in public preview
Dear MaxCompute user,
Starting September 15, 2023 (Friday), the MaxCompute Notebook feature will be available for public preview in the China (Hangzhou), China (Beijing), China (Shanghai), and China (Shenzhen) regions.
MaxCompute Notebook is a fully managed, interactive environment for data analysis and mining. It offers a web-based interactive development environment for data engineers, data analysts, and data scientists. You can use SQL, PyODPS, Python, and other languages to analyze and explore data, unlock its value, and develop integrated big data and AI applications. For more information about this feature, see Use MaxCompute Notebook.
To apply for the public preview, see Apply for a feature public preview.
Thank you for your cooperation.
September 1, 2023: Price reduction for data transmission exclusive resources (Subscription)
Dear MaxCompute user,
MaxCompute has re-architected its exclusive resource groups for data transmission, improving operational efficiency. This allows the same resources to support higher concurrency, reducing overall costs. As a result, effective 00:00 on September 1, 2023, we are reducing the price for the exclusive data transmission resource service to 75 CNY/month per concurrent unit.
Thank you for choosing MaxCompute.
August 31, 2023: Price reduction for elastically reserved CUs
Dear MaxCompute user,
Alibaba Cloud is adjusting the price of elastically reserved CUs for subscription MaxCompute plans. The changes are as follows:
The price in public cloud regions is adjusted from
0.625 CNY/CU/hourto0.315 CNY/CU/hour.The price in Alibaba Finance Cloud regions is adjusted from
1.1875 CNY/CU/hourto0.595 CNY/CU/hour.
The new prices will take effect on September 20, 2023 (Wednesday). After this date, your usage of elastically reserved CUs for subscription MaxCompute plans will be billed at the new rates.
Thank you for your cooperation.
August 18, 2023: MaxCompute console cost analysis feature
Dear MaxCompute user,
The cost analysis feature in the MaxCompute console has been progressively rolled out starting August 18, 2023. This feature helps you analyze your pay-as-you-go bills by drilling down from costs to usage details. It provides various reports that simplify expense analysis for administrators. Currently, cost analysis only supports data for pay-as-you-go computing and storage. The cost data is derived from usage records (raw costs calculated directly from billing formulas) and may have minor discrepancies with your final bill due to rounding or precision differences. For more information, see Cost analysis.
Thank you for your cooperation.
August 8, 2023: Tenant-level Information Schema now generally available
Dear MaxCompute user,
The tenant-level Information Schema provides project metadata and usage history from a tenant-wide perspective. Compared to the previous project-level Information Schema, you can now pull a specific type of metadata from all projects under the same metadata center in a single query, improving your query efficiency. When you use this feature, a project named SYSTEM_CATALOG is created under your Alibaba Cloud account, containing a built-in INFORMATION_SCHEMA. By accessing the read-only views provided by this built-in schema, you can query metadata and usage history for all projects associated with your account. For more information, see Tenant-level Information Schema.
Thank you for your cooperation.
August 1, 2023: MaxCompute Service Level Agreement (SLA) changes
Dear MaxCompute user,
To provide you with higher quality service, we are updating the MaxCompute Service Level Agreement (SLA), effective September 1, 2023 (Friday). The new SLA will calculate service availability and compensation standards separately for each service item. For details, see the MaxCompute Service Level Agreement.
To help you better understand how service availability and compensation are calculated under the new SLA, we have provided case studies for service unavailability scenarios. For details, see MaxCompute service unavailability compensation examples.
Thank you for your cooperation.
July 31, 2023: MaxCompute Pay-as-you-go Spot Edition launched
Dear MaxCompute user,
The MaxCompute Pay-as-you-go Spot Edition is officially launched on July 31, 2023 (Monday), in the China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), and China (Chengdu) regions.
The MaxCompute Pay-as-you-go Spot Edition is a type of pay-as-you-go computing resource. By activating it, you can use MaxCompute's spare computing capacity. Jobs that run on these resources (SpotJobs) have a lower unit price compared to standard pay-as-you-go jobs. This edition is ideal for workloads that are cost-sensitive but not time-critical, allowing you to save up to 66.66% on computing costs. To get started, log in to the Alibaba Cloud MaxCompute Pay-as-you-go Spot Edition activation page.
For more information about billing and usage, see Pay-as-you-go Spot Edition.
Thank you for your cooperation.
July 3, 2023: Transaction Table 2.0 now in public preview
Dear MaxCompute user,
The MaxCompute Transaction Table 2.0 feature is available for public preview starting July 3, 2023 (Monday).
MaxCompute is launching Transaction Table 2.0, a significant upgrade to transaction tables. It supports primary keys, high-concurrency upsert and delete operations at the minute level, a Time-Travel feature, and incremental queries, all while providing snapshot isolation for reads and writes. For a functional overview and usage instructions, see Overview of near-real-time data warehouses.
This new feature is available in all regions. To apply for the public preview, see Apply for a feature public preview.
Thank you for your cooperation.
July 3, 2023: New JSON data type now in public preview
Dear MaxCompute user,
The new JSON data type in MaxCompute is available for public preview starting July 3, 2023 (Monday).
MaxCompute now supports a native JSON data type. When you insert JSON data, MaxCompute automatically optimizes it into a columnar format, which improves compression ratios and enhances query and analysis performance. For a functional overview and usage instructions, see JSON data type.
This feature is available in all regions. If you plan to use this feature with other cloud services, first confirm that those services also support the JSON data type before you proceed.
Thank you for your cooperation.
June 8, 2023: Computing resource optimization recommendation feature
Dear MaxCompute user,
Starting 20:00:00 on June 8, 2023 (Thursday), MaxCompute will progressively roll out its computing resource optimization recommendation feature in public cloud regions in the Chinese mainland.
This feature analyzes the last 30 days of job request data for your subscription level-1 quota. It then simulates a daily job request distribution and, based on your desired job output expectations, evaluates your current resource usage and recommends an improved resource configuration. This helps improve resource utilization and optimize cost and efficiency.
For more information, see Optimize computing resource configurations.
Thank you for your cooperation.
May 24, 2023: New level-2 quotas now default to FIFO scheduling
Dear MaxCompute user,
Quota scheduling policies include FIFO (First-In, First-Out) and FAIR. Our observations show that most jobs are part of multi-task scenarios, where a FIFO policy is more likely to reduce the average runtime of individual jobs.
Therefore, starting from May 24, 2023, the default scheduling policy for newly created level-2 quotas in the China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), and China (Chengdu) regions is adjusted to FIFO (Existing quotas are not affected and remain FAIR), and you can modify the scheduling policy. The default scheduling policy for existing quotas is FAIR. The following section provides a detailed description of the scheduling policies for level-2 quotas:
Scheduling policy | Allocation (same priority) | Allocation (different priority) |
FIFO | Resources are allocated to the first submitted job. | Resources are allocated to the highest-priority job, even if it was submitted later than a lower-priority job. |
FAIR | Resources are distributed evenly among all jobs submitted at the same time. | Resources are first distributed evenly among higher-priority jobs. Any remaining resources are then distributed evenly among lower-priority jobs. |
You can choose the appropriate scheduling policy based on your business needs. Note that setting the scheduling policy is not currently supported for interactive level-2 quotas. For more information, see Scheduling policies for computing resource quotas.
Thank you for your cooperation.
May 10, 2023: MaxCompute service upgrade
Dear MaxCompute user,
To improve MaxCompute's network disaster recovery, we will perform an upgrade from 15:00:00 to 16:00:00 on May 24, 2023 (Wednesday). This upgrade will affect the China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), and China (Chengdu) regions.
During the upgrade, the MaxCompute access layer will undergo a network change, which involves adding A records to domain names. Under normal circumstances, this will not affect your service. However, if you have established a connection through CEN or a leased line and have configured MaxCompute's VIP address in your route table, you must contact Alibaba Cloud support immediately to add the new VIPs to your route table.
We apologize for any inconvenience this may cause. If you have any questions, you can contact us at any time by submitting a ticket or calling our service hotline.
Thank you for your cooperation.
April 30, 2023: Renewal ends for select MaxCompute offerings
Dear MaxCompute user,
Thank you for your support of MaxCompute. To provide more flexible product offerings that better suit your business needs, we are optimizing our existing specifications. The renewal option for the following offerings will be discontinued at 00:00:00 on April 30, 2023 (Sunday), and the services will be fully discontinued at 00:00:00 on October 31, 2023 (Tuesday): the Storage-Intensive 160, Storage-Intensive 320, and Storage-Intensive 600 packages under the Subscription Package Edition, and non-reserved computing resources.
We recommend that you change your Subscription Package Edition to the Subscription Standard Edition for computing resources and reduce your non-reserved computing resources to zero as soon as possible. For downgrade instructions, see Reference. You can also use the new elastically reserved CUs, which are based on standard computing resources and support hourly scaling.
For the affected offerings, the renewal function will remain available in the console until the renewal discontinuation date. However, we strongly advise against renewal and recommend completing the configuration change early to avoid business disruption. During this period, subscription resources cannot be upgraded, only downgraded. After renewal is discontinued, you will not be able to renew or change your configuration. You can continue to use your purchased resources until the service discontinuation date, after which the resources will become unavailable.
If you have any questions or need assistance, contact us through your dedicated DingTalk group or the "MaxCompute Offerings Sunset Support" group (ID: 10405002914). If you do not use any of these offerings, please disregard this notice.
Thank you for your cooperation.
April 11, 2023: Special offers for new MaxCompute users
Dear MaxCompute user,
On April 11, 2023 (Tuesday), MaxCompute launched a special offer for new users in the Alibaba Cloud Free Trial Center. This offer is based on MaxCompute's pay-as-you-go products and provides free quotas, including a 5,000 CU-hour computation resource pack and an 8 GB-month storage resource pack (equivalent to 100 GB of standard storage for a period).
This offer is only available to real-name verified new accounts that have never purchased any MaxCompute product. After purchasing through the Free Trial Center, you can get started with MaxCompute by following the guide at Quickly explore MaxCompute by using a built-in public dataset.
Thank you for your cooperation.
April 6, 2023: New Job O&M feature
Dear MaxCompute user,
Starting April 6, 2023 (Thursday), MaxCompute will progressively roll out its new Job O&M feature. The new Job O&M feature provides a comprehensive list of all MaxCompute jobs for operational management. If you are a data developer, this feature helps you view and terminate your jobs. If you are an administrator, it allows you to monitor resource load in conjunction with quota groups and terminate jobs.
The new Job O&M feature provides both current job information and job snapshots. By setting a Time Range, you can:
Filter for jobs that completed within the specified time range.
Filter for jobs that were running at the end time or within the preceding three minutes. This provides a job snapshot, and the point in time of the snapshot is known as the job snapshot observation time.
For more information about the new Job O&M feature, see Job O&M.
Thank you for your cooperation.
March 11, 2023: Behavioral adjustments in MaxCompute PyODPS
Dear MaxCompute user,
Effective March 11, 2023 (Saturday), the following adjustments will be applied to MaxCompute PyODPS (version 0.11.3 and later). Please take note of these changes.
MaxCompute has introduced a new object level, Schema (for details, see Schema operations). Its name conflicts with the existing table schema field. Therefore, we recommend using
table_schemato refer to a table's schema. Usingschemato describe a table's schema will now produce a warning, but its usage is not affected. Example:--# Get the schema of a table. t = o.get_table('pyodps_iris') print(t.table_schema) # Get the schema of the pyodps_iris tableTime fields on Resource and Function objects, such as
creation_time, will now use the local time zone by default instead of UTC. This makes their behavior consistent with time attributes on Table and Partition objects. If you wish to continue using UTC, you can configure the following:options.use_legacy_parsedate = TrueThe attribute field
last_modified_timefor tables and partitions has been renamed tolast_data_modified_time. Using the old attribute name will produce a warning, but its usage is not affected.
Thank you for your cooperation.
January 18, 2023: Throttling for MaxCompute Tunnel shared resource groups
Dear MaxCompute user,
To provide a more stable Tunnel service, MaxCompute will set a maximum concurrency limit per user for Tunnel shared resource groups in each region, starting March 1, 2023 (Wednesday).
Currently, MaxCompute provides public Tunnel shared resource groups in each region for data upload and download. This measure prevents a sudden surge in usage by a single user from monopolizing shared resources in a region and blocking other users. For more information, see Data transmission service (upload) scenarios and tools.
Thank you for your cooperation.
January 17, 2023: MaxCompute storage metering data format adjustment
Dear MaxCompute user,
Starting from January 17, 2023 (Tuesday) Beijing time, MaxCompute adjusted the format of its storage metering data. The storage size metering data for projects will now be pushed as integer data. This adjustment primarily fixes an issue that prevented the billing system from processing charges because the data was converted to other formats. After this adjustment, billing for the MaxCompute storage service will be aligned with the official billing mechanism. You can view the cost details on the billing details page. For more information, see View billing details. You can also download and view the metering data from the Usage Records page under Costs and Billing. For more information, see Usage Records.
Thank you for your cooperation.