AccessKey Threat Detection and Leak Response Plan

更新时间:
复制 MD 格式

Overview

An AccessKey is a credential that authenticates your identity when you call Alibaba Cloud APIs. A leaked AccessKey threatens the security of all resources in your account. It can lead to unexpected charges and malicious extortion. This topic describes a plan for AccessKey threat detection and leak response. This plan helps you continuously monitor AccessKeys, identify threats promptly, and respond to them. If an AccessKey is leaked, the response plan helps you quickly contain the risk. This prevents widespread misuse of your identity credentials.

Advantages

Real-time open-source code detection

The AccessKey leak detection feature in Security Center scans public source code on GitHub in real time. It looks for leaked AccessKey information, which is often accidentally uploaded by employees. When a leak is found, Security Center notifies you. This helps you discover the risk immediately.

Quickly contain exposure

Emergency actions, such as restricting source IPs for AccessKey calls and denying high-risk permissions, allow you to quickly contain the exposure of a leaked AccessKey. You can limit AccessKey calls to trusted network environments and allow only low-risk operations. This reduces the damage from a leak. For a long-term solution, use Security Token Service (STS) tokens instead of AccessKeys to prevent leaks.

Scenarios

Responding to AccessKey leak risks

Scenario description

A company uses AccessKeys to call Alibaba Cloud APIs and access cloud resources. The company does not have effective control over its AccessKeys and is worried about leaks. A leak can threaten all resources in the account, cause unexpected charges, and lead to malicious extortion. The company wants to detect AccessKey risks. If a leak occurs, they need an effective emergency response to reduce the impact on their business.

Applicable customers

  • You can call Alibaba Cloud APIs with an AccessKey to access cloud resources.

Architecture

This solution uses Security Center, ActionTrail, and Cloud Governance Center to detect AccessKey risks. It analyzes attack vectors, account calls, and historical behavior to identify and respond to threats promptly. You can also configure alerts in Security Center to receive immediate notifications if an AccessKey is leaked. If an AccessKey is leaked, first use ActionTrail to check how it has been used. If an AccessKey is not in use, such as an idle key, disable it immediately. For an AccessKey that is in use, take immediate action. Restrict source IPs and deny high-risk permissions to quickly contain the exposure. This limits AccessKey calls to trusted network environments and allows only low-risk operations, which reduces the damage from the leak. After the initial response, rotate the AccessKey as soon as possible. Then, disable the leaked key. For a long-term solution, use STS tokens instead of AccessKeys to eliminate the risk of leaks.

Product pricing and terminology

Product billing

Product Name

Description

Pricing

Resource Access Management (RAM)

Resource Access Management (RAM) is an Alibaba Cloud service that lets you manage user identities and resource access permissions.

Free. For more information, see Product Billing.

ActionTrail

ActionTrail is an Alibaba Cloud service that records and delivers operation logs for your cloud resources. You can use it for security analytics, resource change tracking, and compliance auditing.

Free. For more information, see Product Billing.

Cloud Governance Center

Cloud Governance Center is a platform for centralized IT governance across multiple accounts on Alibaba Cloud. It uses guided wizards and automated processes to help you quickly set up a landing zone, establish a secure and compliant multi-account environment, and continuously govern your multi-account environment on the cloud.

Free. For more information, see Product Billing.

Security Center

Security Center is a security management platform that integrates continuous monitoring, in-depth defense, comprehensive analysis, and rapid response. Built on a cloud-native architecture, it provides features such as asset management, configuration checks, proactive defense, security hardening, cloud product configuration assessment, and security visualization. The platform can effectively detect and block threats such as viruses, cyberattacks, ransomware, vulnerability exploits, and AccessKey leaks. Security Center helps you achieve an integrated and automated security operations loop, protect workloads such as hosts, containers, and virtual machines in multicloud environments, and meet regulatory compliance requirements.

Paid. For more information, see Product Billing.

Glossary

Name

Description

AccessKey

An AccessKey is a permanent credential from Alibaba Cloud. It is a key pair that consists of an AccessKey ID and an AccessKey secret. Requests include a signature to verify identity and validate the request. This signature is generated by encrypting the request content with the AccessKey ID and AccessKey secret.

Security Token Service (STS) token

Alibaba Cloud Security Token Service (STS) is a service that manages temporary access permissions. RAM provides two types of identities: RAM users and RAM roles. A RAM role does not have permanent identity credentials. Instead, it obtains temporary credentials, known as an STS token, through STS. You can customize the validity period and access permissions of an STS token.

Programmatic identity

An identity is an entity that performs operations in a cloud environment. There are two main types of identities on the cloud: human identities and programmatic identities. A programmatic identity represents an application or service. It often uses an AccessKey or an STS token to access cloud resources and data through Alibaba Cloud OpenAPI.

Security

Cloud Governance Center service-linked role

The Cloud Governance Center service-linked role, AliyunServiceRoleForGovernance, has permissions to operate on products such as Resource Directory, RAM, and Cloud Config. This allows Cloud Governance Center to check the IT governance maturity of your account. For more information about service-linked roles, see Service-linked roles for Cloud Governance Center.

Security Center service-linked role

In certain scenarios, Security Center uses a service-linked role to obtain the permissions it needs to access other Alibaba Cloud services. For more information about service-linked roles, see Service-linked roles for Security Center.

Notes

AccessKey network access control policy

  • The AccessKey network access control policy feature is currently in invitational preview. Contact your Alibaba Cloud service manager to apply for trial access.

  • The AccessKey network access control policy applies to all Alibaba Cloud services except Message Queue for Apache RocketMQ, Message Queue for RabbitMQ, Message Queue for MQTT, EventBridge, Message Queue for Light-weight and Low-cost IoT, CloudMonitor (for reporting event monitoring data over HTTP), and Hologres. The release dates for unsupported services will be announced separately.

  • To use a VPC policy to restrict access from IP addresses within a VPC, make sure you access Alibaba Cloud APIs through a VPC endpoint. Otherwise, access will still be through the public network and will not be restricted by the VPC policy.

Cloud services and events supported by ActionTrail AccessKey audit

The events currently supported by AccessKey audit are the same as those supported by ActionTrail. New events will be added continuously. For a list of events supported by ActionTrail, see Services that work with ActionTrail.

Implementation steps

Preparations

  • This solution uses Cloud Governance Center to perform a comprehensive check of your cloud resources, including the secure use of AccessKeys. Make sure you have activated Cloud Governance Center. For more information, see Activate Cloud Governance Center.

  • This solution uses the Insights events feature of ActionTrail to analyze AccessKeys with abnormal call rates based on historical behavior and promptly detect anomalies. Make sure you have enabled the Insights events feature for ActionTrail. For more information, see Overview of Insights events.

  • To use the built-in alerts of ActionTrail to monitor abnormal AccessKey calls, make sure that all operation logs for this account are delivered to Simple Log Service (SLS). For more information, see Overview of event alerting.

Time commitment

After you complete the preparations, the estimated implementation time for this solution is 60 minutes.

Procedure

Audit AccessKey operation events

Both AccessKey risk detection and leak response require a thorough review of AccessKey usage. By querying basic information about the AccessKey, the cloud services accessed, and related IP addresses and resources, you can promptly identify abnormal call behavior. This information also helps you decide whether the AccessKey is in active use and needs to be rotated. You can use ActionTrail along with the native log auditing features of cloud products to trace AccessKey usage information in real time.

  1. First, audit the last used time of the AccessKey and the cloud services it called to make a preliminary assessment of its usage. Pay special attention to the following situations.

    1. If the AccessKey has no last used time or was last used a long time ago, it is likely idle. Confirm this and remove it as soon as possible.

    2. If the list of cloud services accessed by the AccessKey includes unexpected service calls, the key may be misused or leaked. Confirm this and take the emergency measures described below.

  2. Next, audit the operation events for each cloud service called by the AccessKey to understand its behavior. For example, determine which operations (API calls) were performed on which cloud resources from which source IPs. Pay special attention to the following situations.

    1. If the list of events called by the AccessKey includes sensitive operations, such as traversing resource lists, deleting or creating resources, or privileged operations, check the source IP and call time to determine if the behavior is expected.

    2. If the AccessKey was called outside of expected time ranges. For example, if your O&M team uses an AccessKey for local maintenance, you should investigate calls made outside of working hours. Check the specific call events and source IPs to determine if the behavior is expected.

    3. If the list of source IPs for AccessKey calls includes unexpected IP addresses, the key may be misused or leaked.

Auditing AccessKey events in Alibaba Cloud helps you trace AccessKey usage. Your O&M and business teams must work together to determine the actual use of AccessKeys. The detailed procedure for auditing AccessKey operation events is as follows.

Audit the last used time and called services of an AccessKey

You can use the AccessKey audit feature in ActionTrail to query the last used time of an AccessKey and the cloud services it called. AccessKey audit supports all cloud services that can be accessed using an AccessKey.

  1. Log on to the ActionTrail console.

  2. In the navigation pane on the left, click AccessKey Audit.

  3. On the AccessKey Audit page, enter an AccessKey ID and click the icon.

  4. In the Basic Information section, view the AccessKey ID, Account ID (Alibaba Cloud account ID or RAM user ID), Username, Last Used Time, and Last Used Service.

Audit control plane operation events

For control plane operations of a cloud service, you can still use the AccessKey audit feature in ActionTrail to view the lists of all events, IPs, and resources accessed by the AccessKey for that service.

  1. Log on to the ActionTrail console.

  2. In the navigation pane on the left, click AccessKey Audit.

  3. On the AccessKey Audit page, enter an AccessKey ID and click the icon.

  4. In the Accessed Cloud Services section, view the lists of cloud services, events, IPs, and resources accessed by the AccessKey.

Audit data plane operation events

For data plane operations of a cloud service, such as uploading or downloading files to an OSS bucket, ActionTrail does not yet support auditing these events. For example, the following figure shows Unsupported Event in the event list in ActionTrail.

Therefore, you need to use the log auditing feature of the corresponding cloud product to query these events. The following are common cloud services where AccessKeys are used for data plane operations.

Object Storage Service (OSS)

Use the real-time log query feature of OSS to query OSS access logs directly in the OSS console. This helps you perform operation audits, access statistics, anomaly backtracking, and problem diagnosis for OSS access. For more information, see Real-time log query. After you enable real-time log query for OSS, you can use the search statement * and __topic__: oss_access_log and access_id: ${Your AccessKey ID} to search for operation logs of a specific AccessKey.

Pay special attention to the following fields in the log information:

  1. client_ip: The source IP.

  2. operation: The operation that was called.

  3. bucket, object: The OSS bucket and file that were operated on.

Simple Log Service (SLS)

Use CloudLens for SLS to monitor Simple Log Service. This includes operation logs for creating, modifying, updating, and deleting all assets within a project, along with data read and write logs. For more information, see Overview of SLS monitoring methods. After you enable CloudLens for SLS, you can audit data plane operation events for SLS as follows.

  1. Open the SLS console. In the Log Application section, click CloudLens for SLS to go to SLS Data Insights.

  2. In the navigation pane on the left, click Provisioning. On the SLS Collection Rules tab, click Create Collection Rule to create a collection rule with the log type set to Detailed Log. If a rule already exists, you can skip this step.

  3. Click the Destination Logstores tab. In the list of destination databases, select a Logstore of the Detailed Log type. Click the Logstore name to view operation logs for different regions.

  4. In the Logstore, select the logstore named internal-operation_log. You can use the search statement * and AccessKeyId: ${Your AccessKey ID} to search for operation logs of a specific AccessKey.

Pay special attention to the following fields in the log information:

  1. SourceIP: The source IP.

  2. Method: The operation that was called.

  3. Project, LogStore: The SLS project and Logstore that were operated on.

Detect AccessKey risks

Perform configuration checks using Cloud Governance Center

The governance maturity check in Cloud Governance Center performs a comprehensive check of your cloud resources from dimensions such as security, efficiency, stability, and cost. This helps you promptly identify governance gaps and potential risks. It also provides user-friendly guidance to help you improve your IT governance configurations and reduce risks. This includes programmatic identity management risks, such as idle AccessKeys and AccessKeys that have not been rotated for a long time.

  1. Log on to the Cloud Governance Center console. In the navigation pane on the left, choose Well-Architected > Governance Maturity Check.

  2. In the filter bar for the check item list below, select Credential Security. This filters for check items related to AccessKey credentials.

  3. For check items with risks, click the check item name to view the list of non-compliant resources and related remediation guidance. This helps you with further governance.

Detect leaked AccessKeys in GitHub open-source code using Security Center

Security Center supports real-time detection of AccessKey information in public source code on GitHub, which is often accidentally uploaded by employees. When an AccessKey leak is found, Security Center sends you a notification to help you discover the risk promptly. This feature is provided by default in Security Center and does not require additional payment.

An AccessKey can be exploited by a third party only if both the AccessKey ID and the AccessKey secret are leaked. When Security Center detects that the AccessKey information of an Alibaba Cloud account or a RAM user has been leaked, it sends a notification to the corresponding Alibaba Cloud account based on whether the AccessKey secret is valid. The supported notification methods are as follows:

  • Alert on the AccessKey leak detection page: An alert is provided as long as an AccessKey leak is detected, regardless of whether the AccessKey secret is valid.

  • Voice call: A real-time voice call is sent to the recipient you set only when the leaked AccessKey secret is detected to be valid.

  • Console pop-up notification: When the leaked AccessKey secret is detected to be valid, a pop-up notification appears when you visit the Alibaba Cloud Management Console home page or most product consoles.

  • Alert notification based on notification settings: A notification is sent based on your configured notification methods (internal message, email, or text message) only when the leaked AccessKey secret is detected to be valid.

We strongly recommend that you configure AccessKey leak alert notifications in Security Center to ensure you receive alert messages promptly in case of a leak. You need to complete the alert configuration in two ways:

  1. Notification settings: Security Center enables AccessKey leak alert notifications by default. The notification methods include text message, voice call, email, and internal message. You can customize the Notification Method for AccessKey Leak Intelligence on the SMS/Email/Internal Message tab of the Notification Settings page in the Security Center console. After you configure this setting, you will only receive notifications through the selected methods. Due to the high security risk of AccessKey leaks, we recommend that you select all notification methods to ensure timely receipt of notifications. For more information, see Configure notifications.

  2. Contact settings: By default, only the account contact receives notifications. If you have a unified security contact who needs to receive notifications, you can add the contact manually.

    1. Text message, email, and internal message: On the Basic Recipient Management page, in the Security Message > Alibaba Cloud Security Information Notification section, modify the message recipients.

    2. Voice call: On the Voice Recipient Management page, in the Account Security Alert section, modify the message recipients.

If you have set up a multi-account architecture using Resource Directory, we recommend that you use the account factory in Cloud Governance Center to set and distribute contacts in batches. This ensures a consistent configuration baseline and improves efficiency. For more information, see Account Factory.

Continuously detect abnormal AccessKey calls

In addition to AccessKey leaks caused by open-source code on GitHub, there are many other ways for leaks to occur. We recommend that you continuously monitor for abnormal AccessKey calls to promptly detect potential leak events and respond quickly to limit their impact. You can detect abnormal AccessKey calls in the following ways.

Monitor abnormal AccessKey calls from an attack perspective using Security Center

Based on years of security attack and defense experience and large models, Security Center can detect common abnormal AccessKey calls. For example, it can detect if the IP address calling the AccessKey has recently initiated an attack, if the IP address is making batch calls to multiple users' AccessKeys on the cloud, or if the API being called is sensitive and the AccessKey has been leaked before. This feature is provided by default in Security Center, and you do not need to pay extra for it.

  1. Log on to the Security Center console. In the navigation pane on the left, choose Risk Governance > AK Leak Detection.

  2. Check for any abnormal AccessKey call alerts. If there is an alert, view the details to confirm whether the AccessKey call is normal behavior.

  3. (Optional) Configure security alert notifications. On the System Settings > Notification Settings page in the Security Center console, you can configure the notification method for Security Alerts to receive abnormal AccessKey call alerts promptly. The Free Edition only supports internal message notifications. For more information, see Configure notifications.

Configure anomaly alerts using ActionTrail

ActionTrail provides built-in alerts that you can enable in the ActionTrail console to receive notifications when abnormal AccessKey calls occur. Currently, we recommend using the following built-in alerts:

  • Abnormal AccessKey usage frequency alert: Checks every 15 minutes. An alert is triggered if the number of abnormal AccessKey uses in the past 30 minutes exceeds a specified threshold. The threshold can be configured in the rule parameters.

  • Root account AccessKey usage detection: Checks every 15 minutes. An alert is triggered if there is a record of root account AccessKey usage in the past 30 minutes. The root account should not create or use an AccessKey. If it does, an alert will be triggered.

For more information, see Enable and configure event alerts.

Detect abnormal AccessKey calls based on historical behavior

The Insights events feature of ActionTrail analyzes AccessKeys with abnormal call rates based on historical behavior to promptly detect anomalies. ActionTrail analyzes all write events in your Alibaba Cloud account. It uses the API call rate and a mathematical model to analyze whether the current API call behavior has changed significantly compared to historical behavior and generates Insights events.

  1. Log on to the ActionTrail console.

  2. In the navigation pane on the left, click Insights.

  3. On the Audit Event Insights page, click Enable Insights.

  4. After enabling it, select the region where you want to query events from the top navigation bar.

  5. On the Insights page, set Event Type to Abnormal AccessKey Call Event, and then click the icon.

  6. In the Actions column of the target Insights event, click View Event Details.

  7. In the Insights Event List section, click the target Insights event record.

    1. On the Insights Event tab, you can view the Basic Information and Multi-dimensional Aggregation Analysis of the target Insights event record.

    2. On the Related Operation Events tab, you can view all related operation events and their details for the target Insights event record.

    3. On the Insights Event Record tab, you can view the JSON-formatted event content of the target Insights event record.

Identify AccessKey leaks

When you detect an abnormal AccessKey call, you need to further confirm whether the behavior is expected. This helps you determine if the AccessKey is suspected to be leaked and if emergency measures are needed. For more information, see the Audit AccessKey operation events section above.

We recommend that you focus on the following audit information to help you determine if an AccessKey is suspected to be leaked:

  • The time of the abnormal call. Is it consistent with the call times of your actual business systems?

  • The source IP of the abnormal call. Is it the egress IP of your actual business systems?

  • The operation of the abnormal call. Is it a required operation for your actual business systems? Pay special attention to sensitive operations, such as traversing, creating, and deleting resources.

Handle a leaked Alibaba Cloud account AccessKey

If an Alibaba Cloud account AccessKey is leaked, you first need to confirm whether the AccessKey is actually in use. This requires the participation of your O&M and business teams. For more information, see the Audit AccessKey operation events section above.

You can use information such as the time of the call, the cloud product and operation, and the source IP to determine if the AccessKey call is a legitimate business call. For example, if the AccessKey starts being used only after the leak occurred, it is likely not being used in your business. With further investigation from your business team, you can quickly determine the usage status of the AccessKey.

  • If you confirm that the AccessKey is not in use, go to the AccessKey Management page to disable and delete the AccessKey directly.

  • If you confirm that the AccessKey is in use, go to the AccessKey Management page to rotate the AccessKey as soon as possible. You can create a new AccessKey first and store the AccessKey secret securely. Replace the original AccessKey with the new one. After verifying that everything is running normally, disable and delete the original AccessKey.

Because an Alibaba Cloud account AccessKey has full permissions for the account and its permissions cannot be controlled, it is difficult to respond effectively in the event of a leak. If an Alibaba Cloud account AccessKey is used by an unauthorized person due to improper storage or use, it poses an enormous security threat to your account resources. If you are currently using an Alibaba Cloud account AccessKey, we strongly recommend that you optimize your setup immediately.

  • Prioritize using the STS token temporary credential solution for programmatic access. Learn more.

  • If you must use an AccessKey, we recommend using a RAM user AccessKey with least privilege. Learn more.

Handle a leaked RAM user AccessKey

When a RAM user AccessKey is suspected to be leaked, you can handle it as follows.

  1. Emergency response. Because rotating an AccessKey requires time for review and modification, we recommend that you first take the following steps to reduce the impact of the leak.

    1. If the source IP for your AccessKey calls is fixed or enumerable, configure a network access control policy for the AccessKey as soon as possible to block untrusted access sources.

    2. Reduce the permissions of the leaked AccessKey as soon as possible. Restrict high-risk permissions to reduce the risk of business and financial losses.

    3. You need to simultaneously confirm the usage of the AccessKey. This requires the cooperation of your O&M and business teams to confirm whether the AccessKey is actually used by the business. You can use information such as the time of the call, the cloud product and operation, and the source IP to help you confirm the usage of the AccessKey.

  2. If you confirm that the AccessKey is not in use or is not used for legitimate business purposes, go to the RAM console to disable the RAM user's AccessKey directly. After observing that there is no impact on your business, delete the RAM user's AccessKey. For more information, see Disable an AccessKey pair for a RAM user and Delete an AccessKey pair for a RAM user.

    1. As a best practice, we recommend that all console access users in your account enable multi-factor authentication (MFA) to reduce the risk of sub-account theft.

    2. Check for abnormal operations performed with the leaked AccessKey and investigate other potentially leaked identities.

    3. Check the User Center for any abnormal charges. In conjunction with the abnormal operation investigation from the previous step, handle the resource instances that have generated abnormal charges. For example, if the AccessKey leak led to the creation of unexpected resource instances, delete them.

  3. Short-term solution. If you confirm that the AccessKey is being used in a real business, rotate the AccessKey as soon as possible.

  4. Long-term solution. We strongly recommend that you adopt a temporary credential solution and use STS tokens instead of AccessKeys.

You can refer to the detailed steps below to handle the AccessKey leak.

Emergency response
1. Configure a network access policy for the AccessKey

If the source IP for your AccessKey calls is not fixed, for example, if you put the AccessKey in your mobile app or miniapp, the source IP cannot be fixed. In this case, you cannot restrict the access source by IP, and this measure cannot be used for emergency response.

First, you need to identify the trusted source IPs for your AccessKey calls. For example, if you only call Alibaba Cloud from within your company, the source IP is your company's public egress IP. Note that to use a VPC policy to restrict access from IP addresses within a VPC, make sure you are accessing Alibaba Cloud APIs through a VPC endpoint. Otherwise, access will still be through the public network and will not be restricted by the VPC policy. You can use the AccessKey audit feature of ActionTrail to view all source IPs for the AccessKey, which helps you identify and confirm the trusted source IPs. After confirming the trusted source IPs, you can use the AccessKey's network access policy to restrict access sources.

  1. Log on to the RAM console as a RAM administrator.

  2. In the navigation pane on the left, choose Identities > Users.

  3. On the Users page, click the name of the target RAM user.

  4. On the Authentication tab, in the AccessKey section, click Network Access Control Policy in the Actions column of the target AccessKey.

  5. In the AccessKey-level Network Access Control Policy dialog box, set the public network policy and VPC policy for the AccessKey.

    1. Public network policy: Enter a public IP address or IP address range. Both IPv4 and IPv6 formats are supported. If you do not create a public network policy, all public IP sources are allowed. Therefore, to deny all public network access, you need to enter a non-public address, such as 127.0.0.1.

    2. VPC policy: Enter a VPC ID and an IP address or IP address range within the VPC. Both IPv4 and IPv6 formats are supported. If you do not create a VPC policy, all calls from VPCs are allowed. To create a VPC policy that allows access from all IP addresses within a VPC, enter 0.0.0.0/0 or ::/0 in the IP list.

2. Restrict high-risk permissions for the AccessKey

You need to first understand your business scenario. Without affecting the current business operations, reduce the permissions of the suspected leaked AccessKey as soon as possible. Restrict high-risk permissions to reduce the risk of business and financial losses. Do not remove this policy (restricting high-risk permissions) until the AccessKey is disabled and deleted. We recommend restricting high-risk permissions such as: prohibiting the RAM user from creating new RAM users and granting permissions in RAM, prohibiting the release of ECS, RDS, OSS, and SLS resources, and prohibiting the sending of text messages. The following is an example of a custom policy that disables high-risk permissions. Please fully assess the impact and configure it according to your business needs. For more information, see Create a custom policy and Grant permissions to a RAM user.

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Deny",
      "Action": [
        "ram:AddUserToGroup",
        "ram:AttachPolicyToGroup",
        "ram:AttachPolicyToRole",
        "ram:AttachPolicyToUser",
        "ram:ChangePassword",
        "ram:CreateAccessKey",
        "ram:CreateLoginProfile",
        "ram:CreatePolicyVersion",
        "ram:CreateRole",
        "ram:CreateUser",
        "ram:DetachPolicyFromUser",
        "ram:PassRole",
        "ram:SetDefaultPolicyVersion",
        "ram:UpdateAccessKey",
        "ram:SetPasswordPolicy",
        "ram:UpdateRole",
        "ram:UpdateLoginProfile",
        "ram:UpdateUser"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Deny",
      "Action": [
        "ecs:DeleteInstance",
        "ecs:DeleteInstances",
        "ecs:DeregisterManagedInstance",
        "ecs:ReleaseDedicatedHost"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Deny",
      "Action": [
        "rds:DeleteAccount",
        "rds:DeleteDatabase",
        "rds:DeleteDBInstance",
        "rds:DestroyDBInstance"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Deny",
      "Action": [
        "oss:DeleteBucket",
        "oss:DeleteObject",
        "oss:PutBucketAcl",
        "oss:PutBucketPolicy"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Deny",
      "Action": [
        "log:DeleteLogStore",
        "log:DeleteProject",
        "log:PutProjectPolicy",
        "log:DeleteProjectPolicy"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Deny",
      "Action": [
        "dysms:CreateProductNew",
        "dysms:CreateSmsTemplateNew",
        "dysms:AddSmsTemplate",
        "dysms:SendSms",
        "dysms:SendBatchSms"
      ],
      "Resource": "*"
    }
  ]
}
3. Check AccessKey usage and disable unused keys

While taking the emergency measures above, you can simultaneously confirm whether the AccessKey is actually in use. This requires the participation of your O&M and business teams. For more information, see the Audit AccessKey operation events section above.

You can use information such as the time of the call, the cloud product and operation, and the source IP to determine if the AccessKey call is a legitimate business call. For example, if the AccessKey starts being used only after the leak occurred, it is likely not being used in your business. With further investigation from your business team, you can quickly determine the usage status of the AccessKey.

If you confirm that the AccessKey is not in use or is not used for legitimate business purposes, you can go to the RAM console to disable the RAM user's AccessKey directly. After observing that there is no impact on your business, delete the RAM user's AccessKey. For more information, see Disable an AccessKey pair for a RAM user and Delete an AccessKey pair for a RAM user.

Regardless of whether the AccessKey is in use by your business, we recommend that you continue with the emergency measures described below to further reduce the risk of leaks and ensure cloud security.

4. Enable MFA for the RAM user

As a best practice, we recommend that all RAM users who access the console in your Alibaba Cloud account enable multi-factor authentication (MFA).

  1. Require RAM users under your main account to enable MFA for console logon. For more information, see Manage console logon settings for a RAM user.

  2. Bind an MFA device to the user. For more information, see Bind an MFA device to a RAM user.

5. Check for abnormal AccessKey operations

You need to further check for abnormal operations performed with the AccessKey and investigate other potentially leaked identities. For example, a third party might use the leaked AccessKey to create a new RAM user for their own use to bypass the restrictions you have placed on the AccessKey. For more information, see the Audit AccessKey operation events section above.

We recommend that you focus on abnormal access IPs, resource creation or deletion operations that are not required by your business, and so on. Then, you need to further investigate whether there are other RAM users and AccessKeys with abnormal operations, in addition to the one with the known leak risk. If you find abnormal behavior, confirm with the relevant personnel whether the operation was performed by them. If there is a suspected leak risk, we recommend handling it as follows:

  • If the RAM user needs to continue to be used, we recommend that you immediately change the RAM user's password and enable multi-factor authentication (MFA).

  • If the RAM user was not created normally or is confirmed to be an idle user that is no longer needed, you can delete it. A deleted RAM user is moved to the recycle bin. Observe whether your business is affected. If it is, the RAM user can be quickly recovered.

  • For abnormal AccessKey operations, handle and restrict them according to the methods described in steps 1, 2, 3, and 4 above.

6. Check for abnormal charges

In the Expenses and Costs console, check for any abnormal charges. In conjunction with the abnormal operation investigation from the previous step, handle the resource instances that have generated abnormal charges. For example, if a third party used the leaked AccessKey to purchase multiple resources such as ECS and ACK, you need to release the excess resources promptly after discovering the abnormal operations and charges to prevent further abnormal costs.

Short-term solution: Rotate the AccessKey

After completing the emergency response, you need to rotate and replace the AccessKey that is still in use by your business as soon as possible.

  1. Create a second AccessKey for rotation. For more information, see Create an AccessKey pair for a RAM user.

  2. In all applications or systems that use the AccessKey, update the AccessKey in use to the newly created second AccessKey. You can log on to the RAM console and view the last used time of the AccessKey in the user's AccessKey list on the user details page. This can help you initially determine whether the second AccessKey has been used and whether the original AccessKey is no longer in use.

  3. Disable the original AccessKey. For more information, see Disable an AccessKey pair for a RAM user.

  4. Verify that all applications or systems that use the AccessKey are running normally.

    • If they are running normally, the AccessKey update was successful, and you can safely delete the original AccessKey.

    • If they are not running normally, you need to temporarily reactivate the original AccessKey and then repeat steps 2-4 until the update is successful.

  1. Delete the original AccessKey. For more information, see Delete an AccessKey pair for a RAM user.

Long-term solution: Use STS tokens instead of AccessKeys

Creating an AccessKey for a RAM user or a root identity of an Alibaba Cloud account for programmatic calls is a type of fixed credential. Once created, the credential consists of a fixed AccessKey ID and AccessKey secret until it is deleted. Improper use of fixed credentials can lead to many risks. For example, an application developer might write a fixed AccessKey in plaintext in the code and upload it to a public repository like GitHub, causing an AccessKey leak and ultimately business losses. On Alibaba Cloud, we recommend using role assumption to obtain temporary credentials, STS tokens, instead of fixed AccessKeys whenever possible. Each STS token automatically expires after the maximum session duration of the role, which greatly reduces the possibility of credential leaks. We strongly recommend that you use STS tokens as a long-term solution, modify existing business systems, and establish it as a development standard for new business systems. Promote the use of STS tokens at the organizational level to reduce the security risks associated with using AccessKeys. For more information, see Use temporary credentials instead of fixed credentials.

Troubleshooting

AccessKey VPC restriction policy is not working

If you have configured an AccessKey VPC restriction policy but find that it is not working, the possible reason is that you are using a public endpoint to access Alibaba Cloud services. When using a public endpoint, the program's requests are sent over the public network, not through the IP address of the VPC. This prevents a match with the configured VPC IP whitelist, causing the access restriction policy to not be applied correctly.

AccessKey account-level network restriction policy is not working

If you have configured an AccessKey account-level network restriction policy but find that it is not working, check whether an AccessKey-level network access restriction policy is also configured. The AccessKey-level policy has a higher priority. If the source IP is not in the AccessKey-level network access whitelist, it will be blocked. You can refer to the AccessKey network access restriction policy evaluation rules below for troubleshooting.

Unable to find call events in ActionTrail AccessKey audit

ActionTrail supports querying all Alibaba Cloud services that an AccessKey has accessed. However, for specific events of an AccessKey accessing a cloud service, it only supports querying the cloud services and events supported by ActionTrail. For more information, see Services and events supported by ActionTrail.

If you find during a query that an AccessKey has accessed a cloud service, but there is no specific information in the event list, IP list, or resource list, or the last time does not match, it means that the cloud service or event is not yet supported.