Legal disclaimer
Alibaba Cloud reminds you to carefully read and fully understand the terms of this legal disclaimer before you read or use this document. Your use of this document is deemed to be an acknowledgment and acceptance of this legal disclaimer.
-
You must obtain this document from the Alibaba Cloud website or other authorized channels and use it only for your own lawful and compliant business activities. The content of this document is considered confidential information of Alibaba Cloud, and you must strictly comply with your confidentiality obligations. Without the prior written consent of Alibaba Cloud, you must not disclose any content of this document to, or allow its use by, any third party.
No part of this document shall be excerpted, translated, reproduced, transmitted, or disseminated by any organization, company, or individual in any form or by any means without the prior written consent of Alibaba Cloud.
-
The content of this document may change due to product version upgrades, adjustments, or other reasons. Alibaba Cloud reserves the right to modify the content of this document without notice and to release updated user documents at any time through authorized Alibaba Cloud channels. You should periodically check for and download the latest version from authorized Alibaba Cloud channels.
-
This document serves only as a reference guide for your use of Alibaba Cloud products and services. Alibaba Cloud provides this document on an Current State, with all faults, and as available basis. Alibaba Cloud makes its best effort to provide relevant introductory information and operational guidance based on existing technology but expressly disclaims any warranties, express or implied, regarding the accuracy, completeness, applicability, or reliability of the content of this document. Alibaba Cloud shall not be liable for any errors or financial losses incurred by any party as a result of downloading, using, or relying on this document. In no event shall Alibaba Cloud be liable for any indirect, consequential, exemplary, incidental, special, or punitive damages, including lost profits, arising from the use of or reliance on this document, even if Alibaba Cloud has been advised of the possibility of such damages.
-
All content on the Alibaba Cloud website, including but not limited to works, products, images, archives, information, materials, website architecture, website layout, and web page design, is the intellectual property of Alibaba Cloud and/or its affiliates. This includes, but is not limited to, trademark rights, patent rights, copyrights, and trade secrets. Without the written consent of Alibaba Cloud and/or its affiliates, no one may use, modify, reproduce, publicly disseminate, alter, distribute, or publish the Alibaba Cloud website, product programs, or content. Furthermore, without the prior written consent of Alibaba Cloud, no one may use, publish, or reproduce the Alibaba Cloud name for any marketing, advertising, promotional, or other purposes. This includes, but is not limited to, brands of Alibaba Cloud and/or its affiliates that contain AliCloud, Aliyun, or HiChina, whether used alone or in combination; their associated marks and logos; or any similar company name, trade name, trademark, product or service name, domain name, logo, or mark that could cause a third party to identify it with Alibaba Cloud and/or its affiliates.
Please contact Alibaba Cloud directly if you discover any errors in this document.
Overview of data security and compliance
MaxCompute builds a comprehensive data security system based on confidentiality, integrity, and availability, and provides comprehensive data access control capabilities and a secure and trusted computing environment. The cluster high availability and disaster recovery solutions are provided to ensure business continuity. MaxCompute records detailed user operation logs and task runtime logs for in-process O&M monitoring and post-event security auditing. MaxCompute is built on top of Alibaba Cloud Infrastructure as a service (IaaS) and leverages the security capabilities of the cloud infrastructure. MaxCompute is linked with the security products of the cloud platform, such as Resource Access Management (RAM), Security Center of DataWorks, and Data Security Guard of DataWorks, to implement more security control scenarios.
Access control
Authentication
MaxCompute supports the following user identities: Alibaba Cloud accounts, RAM users, and RAM roles. MaxCompute also supports authentication based on an AccessKey pair, multi-factor authentication (MFA), and Security Token Service (STS) authorization.
You can create an AccessKey pair in the RAM console. An AccessKey pair consists of an AccessKey ID and an AccessKey secret. The AccessKey ID is public and uniquely identifies a user, whereas the AccessKey secret is private and used to authenticate a user. Before a client sends a request to MaxCompute, the client generates a string to be signed in the format that is required by MaxCompute and then generates a signature for the request by using the AccessKey secret. After MaxCompute receives the request, MaxCompute finds the AccessKey secret based on the AccessKey ID and then generates a signature. If the signature is the same as the signature that is sent by the client, the request is valid. Otherwise, MaxCompute rejects the request and returns an HTTP 403 error.
For more information, see User authentication.
Authorization.
MaxCompute provides the following access control mechanisms to perform fine-grained access control: access control list (ACL)-based access control, policy-based access control, download control, LabelSecurity, and row-level permissions. Authorization objects include projects, quotas, network connections, tables, functions, resources, instances, external tables, and external volumes.
RAM-based authorization
MaxCompute supports RAM-based authorization to grant access and management permissions on MaxCompute resources of your Alibaba Cloud account to RAM users and RAM roles. This way, you can assign minimum permissions to users based on your business requirements. This reduces information security risks for enterprises. For more information, see RAM permissions.
ACL-based access control
An ACL is used to implement object-based authorization. An ACL specifies permissions on an object and is considered as a subresource of the object. An ACL takes effect only if the object exists. If the object is deleted, the ACL of the object is also deleted. ACL-based access control is similar to the authorization mechanism that is implemented by using the GRANT and REVOKE statements defined in SQL-92. You can execute these statements to grant or revoke permissions on an object. To manage permissions, you must specify the effect (grant or revoke), object (such as a table or resource), subject (user or role), and action (such as read, write, or delete). For more information, see ACL-based access control.
Policy-based access control
A policy is used to define role permissions. After you assign a role to a user, the permissions of the role take effect on the user. Compared with ACL-based access control, policy-based access control supports not only the whitelist mechanism but also the blacklist mechanism. When you perform policy-based access control, you can specify a policy to allow a role to perform specified operations on specified objects or deny a role from performing specified operations on specified objects. If both the whitelist and blacklist mechanisms are used for the same object, the blacklist mechanism takes precedence. For more information, see Policy-based access control.
LabelSecurity
LabelSecurity allows you to flexibly control user access to column-level sensitive data by using data sensitivity level labels. Data in tables or columns needs to be classified based on data sensitivity levels and users need to be classified based on data access levels. You can set the sensitivity level to 0 to 9 to adapt to different data classification standards. The following security policies are supported:
No-ReadUp: Users cannot read data that has a higher data sensitivity level than their data access level, unless the users are explicitly granted permissions.
Trusted-User: Users are allowed to write data of all sensitivity levels. The default sensitivity level of newly written data is 0.
For more information, see Label-based access control.
Access modes
MaxCompute provides multiple endpoint-based access modes, including the Internet, virtual private cloud (VPC), and PrivateLink. You can enable endpoints for network isolation based on your business requirements. You can configure an IP address whitelist for each endpoint to limit connections from clients.
Secure and trusted computing environment
MaxCompute provides secure computing containers and Java and Python sandboxes to isolate task processes and prevent malicious code from affecting cluster computing tasks.
MaxCompute provides hybrid computing modes based on the considerations of computing flexibility and extensibility. MaxCompute supports user-defined functions (UDFs) in SQL engines and third-party computing frameworks such as Spark and Python in compute engines. However, these features may cause untrusted code, which can trigger unintended system damage and malicious code attacks. MaxCompute uses lightweight secure computing containers and language-level sandboxes to run untrusted code in secure computing containers to achieve process-level security isolation.
MaxCompute provides security assurance to meet the network communication requirements of internal data synchronization and external data access in computing tasks. MaxCompute builds an overlay virtual network for each computing task among its running security containers to achieve security isolation from the host network. This ensures that all nodes in the task can communicate by using private IP addresses but cannot access the host network. If a computing task needs to access the data service API over the Internet or in a VPC, MaxCompute supports task-level networking by using network connections. You must declare the destination network that you want to access and meet the permission check requirements when you start a job. For more information, see Network connection process.
The code runtime environment provided by MaxCompute belongs to users. Any legal liability arising from code execution shall be borne by users.
Confidentiality
MaxCompute provides data security measures for the storage, computing, and transmission status of data.
Transparent encryption at the data storage layer
MaxCompute integrates Key Management Service (KMS) and Bring Your Own Key (BYOK) to automatically encrypt and decrypt data files in storage media. Applications can meet data ciphertext storage requirements without modification, and can use keys to encrypt or decrypt persistent data files of tables and partitions. Encryption algorithms such as AES256 are supported. Automatic key rotation is supported. If the customer disables the key service, the encrypted data cannot be decrypted and accessed. This meets the requirements of data confidentiality and regulatory compliance. For more information, see Storage encryption.
Data content encryption
MaxCompute supports column-level content encryption for sensitive data, such as personally identifiable information (PII), financial information, accounts, and passwords. Applications can call encryption functions to encrypt sensitive data before writing data and decrypt the data during data reading. This prevents sensitive data leaks caused by attacks, such as SQL injection or data breach. MaxCompute can be connected to KMS to encrypt data by using keysets that contain multiple keys. MaxCompute supports the following encryption algorithms: AES-GCM-256, AES-SIV-CMAC-128, and AES-SIV-CMAC-256. These encryption algorithms provide higher encryption reliability to prevent data cracking. For more information, see Encryption and decryption functions.
Dynamic data masking
MaxCompute supports the dynamic data masking feature to protect sensitive data such as PII during business development and testing, data sharing, and O&M. Data masking policies include masking, hashing, character replacement, numeric value rounding, and date rounding. Data masking policies can be used together with the data classification feature of Data Security Guard to meet your masking requirements for sensitive data, such as identity information, bank card numbers, addresses, and phone numbers. Data masking of MaxCompute is implemented in the link that is closest to data reading from storage. This ensures that data is de-sensitive during query, download, association, and UDF computing to avoid sensitive data breach. For more information, see Dynamic data masking.
Encrypted transmission
When you connect to MaxCompute by using the MaxCompute client, SDK, or API, the HTTPS TLS v1.2 encryption protocol is used. This prevents data from being intercepted or tampered with during transmission.
Data lifecycle management
MaxCompute allows you to specify a data retention period. After you specify a data retention period, MaxCompute automatically cleans up expired data to reduce data leak risks and storage costs. You can specify the lifecycle of a table based on your business requirements and data usage frequency. MaxCompute determines whether the interval between the latest update time (LastModifiedTime) of each table or partition and the current time exceeds the lifecycle. If the interval exceeds the lifecycle, MaxCompute performs the reclaim operation. For more information, see Lifecycle management operations.
MaxCompute also supports tiered storage for cold and hot data. Infrequent Access (IA) storage and long-term storage can help you limit the access frequency of historical data and reduce storage costs. For more information, see Configure storage tiers for storage resources.
Integrity
MaxCompute provides data integrity protection measures during data processing.
ACID
MaxCompute supports atomicity, consistency, isolation, durability (ACID) for large-scale data processing jobs. Delta tables use the multiversion concurrency control (MVCC) model to ensure read/write snapshot isolation and optimistic concurrency control (OCC) to control transaction concurrency. Row-level or file-level transaction concurrency control is not supported. Instead, each batch data processing operation is managed as a separate transaction. The transaction conflict logic of some frequently performed operations is optimized based on the semantics of the operations to better support concurrency control while ensuring correctness. MaxCompute uses cyclic redundancy check (CRC) codes to verify data integrity during data storage and transmission. For more information, see ACID semantics.
Multi-replica data storage
MaxCompute uses a distributed file system to automatically create multiple replicas for stored data. By default, three replicas are created. The replicas are distributed across different physical machines and racks to prevent data loss caused by single points of failures (SPOFs) and ensure data durability and integrity.
MaxCompute stores data in the Apsara Distributed File System. The system provides a flat linear storage space and slices linear addresses. Each shard is called a chunk. Each chunk has three replicas and stores the replicas on different nodes in the cluster based on specific policies. This prevents data unavailability due to the failure of one chunk server or one rack. The operations of adding, modifying, and deleting data are synchronized to the three replicas to ensure data integrity and consistency. The file system reclaims the storage space that is released after data is deleted, prohibits other users from accessing the storage space, and erases data to ensure that the deleted data cannot be restored.
Task fault tolerance
The task scheduling system of MaxCompute has high fault tolerance and provides the task retry mechanism. When an SQL job runs, the system builds a directed acyclic graph (DAG) to allocate appropriate computing resource nodes for the job and optimizes the execution process. This eliminates unnecessary shuffles and network jitters and prevents the overall job from being affected by partial node faults. This ensures the accuracy of the execution result and execution efficiency.
MaxCompute Tunnel supports resumable transmission for batch data uploads and downloads. MaxCompute Tunnel also allows you to specify the number of data rows that can contain errors and adjust the buffer size. This ensures data integrity for data processing.
Backup and restoration
By default, the backup and restoration feature is enabled for MaxCompute tables. This feature effectively prevents data loss caused by misoperations. MaxCompute automatically backs up historical versions of data each time data is modified or deleted, or a table is deleted. Historical versions of data are retained for up to 30 days. You can quickly restore data based on the data version number.
Delta tables support time travel queries. You can query data snapshots of a delta table at any time in the previous seven days. You can also query incremental data within a specified time range or version range.
For more information, see Backup and restoration.
Availability
MaxCompute provides security measures to ensure data availability.
Data sharing
MaxCompute provides the package-based access control mechanism for cross-project data sharing scenarios. This mechanism can package the data resources and related permissions of a project. After the package is installed for a third-party project, the third-party project can access the authorized data resources of the project. After the package-based access control mechanism is used, the administrator of Project A can create a package and authorize Project B to install the package. After the administrator of Project B installs the package, you can manage whether the package needs to be further authorized to users in your own project. For more information, see Cross-project resource access based on packages.
Limits on data exchange
MaxCompute provides project protection mechanism for scenarios where data is only imported but not exported. For example, users who have access permissions on multiple projects can transfer data between different projects. If data in a project is sensitive and cannot be exported to other projects, the administrator can use the project protection mechanism to protect the data. This mechanism requires that data be written to the project but not read from the project. For more information, see Project data protection.
Disaster recovery
MaxCompute supports zone-disaster recovery and cross-region disaster recovery to provide disaster recovery capabilities for customers.
Cross-zone high availability
Zone-disaster recovery of MaxCompute extends data storage and computing services to three zones in the same city to improve the resilience of your business. Zone-disaster recovery is suitable for industries such as finance and critical infrastructure. This feature ensures that services of business systems do not stop due to the failure of a single data center. This feature can provide data redundancy backup to reduce business downtime and meet industry compliance requirements for better customer experience on upper-layer applications.
This feature allows you to extend the availability of project-level data from a single zone to three zones in the same city. The three zones are physically isolated, but the network connection has low latency. This implements fault isolation and real-time data synchronization across data centers to ensure data integrity and availability if a disaster occurs.
Cross-region disaster recovery
For cross-region disaster recovery, MaxCompute allows you to select a region more than 1,000 kilometers away from your primary region to serve as a disaster recovery cluster. This establishes a complete, geographically remote backup of your project-level data, protecting it from regional natural disasters. For more information, see Cross-region disaster recovery.
Monitoring and auditing
MaxCompute provides real-time monitoring and post-event auditing for activities such as data authorization and data usage.
Audit logging
MaxCompute provides detailed ActionTrail logs, which record information about operations related to jobs (instances), tables, users, roles, and permissions. ActionTrail logs can meet the compliance requirements for log retention and security management requirements such as real-time monitoring and backtracking analysis.
MaxCompute records all operations triggered by users. You can view and retrieve user behavior logs in ActionTrail logs and store the logs in Simple Log Service projects or specified OSS buckets for a long period of time. For more information, see Audit logging.
Information schema
MaxCompute allows you to query the project metadata and job running history. MaxCompute also allows you to analyze data in real time and periodically export user information, role information, table partition authorization information, historical job records of the previous14 days, and batch upload and download task records for security audit based on the ANSI SQL-92 Information Schema. For more information, see Information schema.