AccessKey leak response plan
An AccessKey pair is a credential used for identity verification when you call Alibaba Cloud APIs. A leaked AccessKey pair can compromise the security of every resource under your account, causing unexpected charges, ransom demands, and, in severe cases, harm to Alibaba Cloud or other users. This topic describes how to respond to a suspected AccessKey leak to avoid a wider risk of credential misuse.
Alibaba Cloud security measures
Alibaba Cloud continuously strengthens cloud service security and helps you protect your account and assets. If Alibaba Cloud detects, based on external intelligence, that your AccessKey pair has been publicly exposed, it promptly notifies you through your registered contact channels. To safeguard your business and data security, Alibaba Cloud also applies restrictive protection to that AccessKey pair, blocking it from calling high-risk APIs of certain cloud services. For details, see Restrictive protection for AccessKeys.
Monitor the notifications sent through text messages, email, and in-console messages. Respond promptly based on your actual business needs, and watch for unusual activity of the cloud resources under your account so that your business runs normally.
Alibaba Cloud cannot monitor the security status of all your AccessKey pairs. Under the shared responsibility model, cloud security is shared between you and Alibaba Cloud. AccessKey pairs are the account and identity credentials of cloud customers, and you remain responsible for their security. Stay vigilant.
If restrictive protection prevents a RAM user from logging on to the console, see What should I do if an AccessKey leak causes a RAM user to be unable to log on or the account is frozen?.
Respond to a suspected leak
When you suspect that an AccessKey pair has leaked, respond immediately. First, determine the scope of the leak: you can confirm the specific leaked AccessKey ID through the notifications of your security email address and in-console messages. If the notification contains only one AccessKey pair and there are no other alerts, typically only that AccessKey pair has leaked.
Determine whether the leaked AccessKey pair belongs to your Alibaba Cloud account (root account) or a RAM user, and then follow the corresponding subsection. A root account AccessKey pair can access every resource under the account, so treat a root account leak with the highest urgency. A complete response covers the following phases:
Disable and delete the leaked AccessKey pair, or rotate it. If you cannot rotate it immediately, reduce its permissions first.
Investigate abnormal operations of the AccessKey pair.
Identify the usage scope of the AccessKey pair before you delete it.
Check for abnormal charges.
Restore secure usage of your account.
Handle a leaked Alibaba Cloud account (root account) AccessKey pair
The AccessKey pair is no longer in use — Go to the AccessKey management page and disable and delete the AccessKey pair immediately.
The AccessKey pair is in use — Rotate the AccessKey pair on the AccessKey management page:
Create a new AccessKey pair and securely store the AccessKey secret.
Replace the original AccessKey pair with the new one.
After you confirm that everything runs normally, disable and delete the original AccessKey pair.
Deleting the maliciously created RAM users alone does not eliminate the leak risk. The core risk is that the AccessKey pair of the Alibaba Cloud account (root account) or a high-privilege RAM user has already leaked, and an attacker can use that AccessKey pair to continuously create new users or escalate privileges. You must rotate the AccessKey pair, and investigate the leak path, such as code, configuration files, and server security. Otherwise, the risk persists.
Before you delete the original AccessKey pair, identify its usage scope as described in Identify the usage scope of the AccessKey pair. After you rotate or delete the AccessKey pair, complete the steps in Investigate abnormal operations and Check for abnormal charges.
Handle a leaked RAM user AccessKey pair
The appropriate response depends on whether the AccessKey pair is still in use and, if so, whether it can be rotated in the short term. If the AccessKey pair can be rotated, rotation is the recommended response.
The AccessKey pair is no longer in use
Go to the RAM console and disable and delete the AccessKey pair. For instructions, see Disable a RAM user's AccessKey and Delete a RAM user's AccessKey.
When you disable an AccessKey pair, the console requires you to complete security authentication (multi-factor authentication, MFA). The AccessKey pair status changes to Disabled only after the authentication succeeds. If the AccessKey pair has been used within the last 60 days, the console warns that disabling it may disrupt your business. Confirm that no business depends on the AccessKey pair before you proceed.
The AccessKey pair is in use and can be rotated (Recommended)
Rotate the AccessKey pair as soon as possible:
Click Create AccessKey. In the Scenarios dialog box, select the scenario that matches your actual use from CLI, Local Development, ECS, ACK, Third-party Application, and Other.
Create the new AccessKey pair and securely store the AccessKey secret.
Replace the original AccessKey pair with the new one.
After you confirm that everything runs normally, disable and delete the old AccessKey pair.
For instructions, see Rotate a RAM user's AccessKey.
When you delete an AccessKey pair, you must manually enter the AccessKey ID to complete a second confirmation. If the AccessKey pair has been disabled for less than seven days, the console warns that deleting it may disrupt your business. Confirm that no business depends on the AccessKey pair before you proceed. A deleted AccessKey pair is not destroyed immediately. It is moved to the recycle bin, retained for 30 days, and then automatically cleared. To restore the AccessKey pair, operate within the retention period.
The AccessKey pair is in use but cannot be rotated in the short term
If you confirm that the AccessKey pair is in use but cannot be rotated smoothly in the short term, follow the steps shown in the following figure to reduce the impact of misuse. Complete the rotation as soon as possible afterward.
Step 1: Reduce the permissions of the AccessKey pair
Identify your business scenario first. Without disrupting current operations, reduce the permissions of the suspected AccessKey pair as soon as possible. Restrict high-risk permissions to lower the risk of damage to your business and billing. Do not revoke this restrictive policy until the AccessKey pair is disabled and deleted.
Recommended high-risk permissions to restrict include: preventing the RAM user from creating new RAM users or granting permissions in RAM, blocking the release of ECS, RDS, OSS, and SLS resources, and blocking SMS sending.
The following example shows a custom policy that denies high-risk actions. Evaluate its impact thoroughly and adjust it based on your business needs.
{
"Version": "1",
"Statement": [
{
"Effect": "Deny",
"Action": [
"ram:AddUserToGroup",
"ram:AttachPolicyToGroup",
"ram:AttachPolicyToRole",
"ram:AttachPolicyToUser",
"ram:ChangePassword",
"ram:CreateAccessKey",
"ram:CreateLoginProfile",
"ram:CreatePolicyVersion",
"ram:CreateRole",
"ram:CreateUser",
"ram:DetachPolicyFromUser",
"ram:PassRole",
"ram:SetDefaultPolicyVersion",
"ram:UpdateAccessKey",
"ram:SetPasswordPolicy",
"ram:UpdateRole",
"ram:UpdateLoginProfile",
"ram:UpdateUser"
],
"Resource": "*"
},
{
"Effect": "Deny",
"Action": [
"ecs:DeleteInstance",
"ecs:DeleteInstances",
"ecs:DeregisterManagedInstance",
"ecs:ReleaseDedicatedHost"
],
"Resource": "*"
},
{
"Effect": "Deny",
"Action": [
"rds:DeleteAccount",
"rds:DeleteDatabase",
"rds:DeleteDBInstance",
"rds:DestroyDBInstance"
],
"Resource": "*"
},
{
"Effect": "Deny",
"Action": [
"oss:DeleteBucket",
"oss:DeleteObject",
"oss:PutBucketAcl",
"oss:PutBucketPolicy"
],
"Resource": "*"
},
{
"Effect": "Deny",
"Action": [
"log:DeleteLogStore",
"log:DeleteProject",
"log:PutProjectPolicy",
"log:DeleteProjectPolicy"
],
"Resource": "*"
},
{
"Effect": "Deny",
"Action": [
"dysms:CreateProductNew",
"dysms:CreateSmsTemplateNew",
"dysms:AddSmsTemplate",
"dysms:SendSms",
"dysms:SendBatchSms"
],
"Resource": "*"
}
]
}For instructions, see Create a custom policy and Manage permissions for RAM users.
Clearly define the business permissions that the AccessKey pair needs and remove all permissions that are not needed.
Step 2: Enable MFA for the RAM user
As a best practice, enable multi-factor authentication (MFA) for all RAM users under your Alibaba Cloud account (root account) who access the console.
Require MFA for RAM users under the root account to log on to the console. For instructions, see Require MFA for RAM user logon.
Bind an MFA device to the user. For instructions, see Bind an MFA device to a RAM user.
After you reduce the permissions and enable MFA, complete the steps in Investigate abnormal operations, Identify the usage scope of the AccessKey pair, and Check for abnormal charges. Then complete the rotation as soon as possible.
Investigate abnormal operations
Check whether the leaked AccessKey pair has abnormal operations, and investigate whether other identities are suspected of leakage. Focus on abnormal source IP addresses and resource creation or deletion operations that your business does not need.
Dangerous operations to investigate during the leak period include:
Identity and permission changes: creating RAM users (
CreateUser), creating AccessKey pairs (CreateAccessKey), attaching access policies (AttachPolicyToUser), creating roles (CreateRole), and enabling console logon (CreateLoginProfile). Events such asCreateLoginProfile,AttachPolicyToUser, andCreateUsertypically indicate that an attacker is creating backdoor accounts or escalating privileges. Pay close attention to the source IP addresses and UserAgents of these calls.Deletion and release of resources: deleting ECS instances (
DeleteInstance), deleting RDS instances (DeleteDBInstance), and deleting OSS buckets (DeleteBucket).Data exfiltration: modifying OSS bucket access permissions (
PutBucketAcl), modifying OSS bucket policies (PutBucketPolicy), and similar operations.Abnormal calling behavior: large numbers of API requests in a short period, operations from unusual IP addresses or regions, and batch operations outside working hours.
To check: In the RAM console, find the AccessKey list of the RAM user and view the operation records in the list, or enter the AccessKey ID on the AccessKey audit page of the ActionTrail console to query the operation records directly.
In addition to ActionTrail, you can log on to the Security Center console to view the details of AccessKey leak alerts, including the names of RAM users that were maliciously created and abnormal calling IP addresses.
The AccessKey audit feature of ActionTrail shows all API calls of a specified AccessKey pair in the last 90 days, including the call time, action name, source IP address, and accessed cloud service. For instructions, see AccessKey audit.
For data-plane operation logs that are not connected to ActionTrail, such as those of OSS and SLS, use the log feature of each cloud service to query.
Further investigate whether other RAM users and AccessKey pairs show abnormal operations besides the AccessKey pair with a known leak risk. If you find abnormal behavior, confirm with the relevant person whether the operations were performed by them. If a leak is suspected, handle it as follows:
If the RAM user must remain in use, immediately change the password of the RAM user and enable multi-factor authentication (MFA).
If the RAM user was not legitimately created, or is confirmed to be an idle user that is no longer needed, delete it. A deleted RAM user is moved to the recycle bin. Monitor whether your business is affected. If so, the RAM user can be restored quickly.
For abnormal operations of an AccessKey pair, first restrict its permissions, and then rotate it.
Identify the usage scope of the AccessKey pair
The source IP addresses recorded by ActionTrail reflect only the calls of management-type events, such as the management operations GetService and GetBucketInfo of OSS. Data-plane operations that are not connected to ActionTrail, such as OSS object uploads and downloads, are not recorded by default. Therefore, the source IP list alone cannot reveal the full usage scope of the AccessKey pair. Before you disable or delete the original AccessKey pair, check each of the following environments for the AccessKey configuration:
Local development machines — Developer environment variables, SDK configuration files, and CLI credential files.
CI/CD pipelines — Build job environment variables, credential stores, and deployment scripts.
Third-party and SaaS applications — Credentials granted to external platforms that call Alibaba Cloud APIs.
Containers and images — Container image layers, Kubernetes secrets, and credentials in orchestration templates.
Scheduled jobs and operations scripts — Server crontab entries, backup scripts, and monitoring collectors.
Messaging and notification services — Callers such as applications that call Short Message Service to send text messages.
After you replace all of these environments with the new AccessKey pair and confirm that your business runs normally, disable or delete the original AccessKey pair. This prevents business disruption caused by an overlooked caller.
Check for abnormal charges
In Expenses and Costs, check whether abnormal charges are generated. For the affected services, take protective measures based on the abnormal operations that you identified during the investigation.
Restore secure usage after an AccessKey leak
After the AccessKey pair of your Alibaba Cloud account (root account) leaks, follow these guidelines to restore secure usage:
Create a separate RAM user — Do not use the root account to create AccessKey pairs. Create a separate RAM user in the RAM console and grant it only the minimum permissions required. When you create the AccessKey pair, note the following:
On the user details page, click the Authentication tab, and then click Create AccessKey. When you select the access method, select Use permanent AccessKey pairs to access.
After you select the I confirm that I must create an AccessKey pair check box, you must complete security authentication (multi-factor authentication, MFA) to continue the creation.
The AccessKey secret is displayed only once during creation. After you close the dialog box, you cannot view it again. Save it securely right away.
On the user details page, grant permissions to the RAM user. When you grant permissions,
AdministratorAccessis marked as high risk. Follow the principle of least privilege and grant only the minimum permissions the RAM user needs, such asAliyunOSSReadOnlyAccess. Do not grantAdministratorAccess.Unless your scenario requires long-term password-free calls, use Security Token Service (STS) token temporary credentials instead of a permanent AccessKey pair to further reduce the leak risk.
Permission scope of the AccessKey pair — An AccessKey pair of a RAM user has only the permissions within the scope of the attached access policies. It does not inherit all permissions of the root account.
Check the AccessKey configuration in your code — Check all AccessKey configuration parameters in your code and remove AccessKey pairs in plain text. If your code has been uploaded to a third-party code hosting platform such as GitHub, use the AccessKey leak detection feature of Security Center to check for exposure.
Store credentials securely — Use environment variables or Key Management Service (KMS) to store AccessKey pairs and avoid hard-coding them. For instructions, see Best practices for using access credentials to call Alibaba Cloud OpenAPI.
Reclaim all AccessKey pairs when an employee leaves
After an employee leaves, promptly check and reclaim the AccessKey pairs under their name to prevent misuse of the leftover credentials. Follow these steps:
Log on to the RAM console. In the Users list, locate the RAM user of the departed employee.
Click the user to open the user details page, and then click the Credentials tab.
Check the status of all AccessKey pairs under this user.
For each AccessKey pair in the Enabled status, disable it first. Disabling requires security authentication (multi-factor authentication, MFA). If the AccessKey pair has been used within the last 60 days, the console warns that disabling it may disrupt your business.
After you confirm that no business depends on the AccessKey pair, delete it. You must manually enter the AccessKey ID to complete a second confirmation. The deleted AccessKey pair is moved to the recycle bin, retained for 30 days, and then automatically cleared.
FAQ
What should I do if an AccessKey leak causes a RAM user to be unable to log on or the account is frozen?
When the Alibaba Cloud security system detects that an AccessKey pair is at risk of leakage, it automatically triggers protective mechanisms that restrict the associated RAM user from logging on to the console. You may encounter error messages such as Authentication.Block.AccountFrozen or "The current account is not allowed to log on."
To resolve this issue:
Log on to the RAM console with the Alibaba Cloud account (root account). Immediately disable or delete the leaked AccessKey pair, and then create a new AccessKey pair to complete the rotation. Disabling the AccessKey pair alone may not lift the restriction. You must delete or rotate it.
After you handle the AccessKey pair, if the RAM user still cannot log on, submit a ticket to request removal of the security risk control restriction.
If neither your secure phone nor Alipay can receive the verification code, and you therefore cannot log on to the console to complete the preceding operations, use the account appeal process to change your secure phone number. After you regain access, log on and disable the leaked AccessKey pair. For instructions, see How to submit an appeal if you cannot complete identity verification?.
Why is my AccessKey pair detected as leaked even though it is stored in a private repository, Nacos, or an offline database?
Even when you believe your AccessKey pair is stored securely, it can still be detected or stolen through the following common paths:
Code repositories — If a private repository was previously public, has been forked, or a collaborator uploaded the code to another location, the AccessKey pair can still be detected by scanning services.
Configuration centers (such as Nacos) — If the configuration center lacks access control, has vulnerabilities, or the server has been compromised, AccessKey pairs stored in plain text may be stolen.
Offline storage — If the application or script that reads the AccessKey pair connects to the internet to call APIs, the key may be intercepted during transmission or written to a log record.
Application vulnerabilities — Remote code execution (RCE) or arbitrary file read vulnerabilities can allow attackers to steal AccessKey pairs from configuration files on the server.
To prevent future leaks:
Use the Credentials tool to manage credentials instead of hard-coding AccessKey pairs.
Configure network access restrictions (IP whitelist) for AccessKey pairs.
Avoid writing AccessKey pairs directly into source code or configuration files.
Can I request emergency disabling of an AccessKey pair that belongs to a different account?
Due to privacy protection and permission isolation, Alibaba Cloud cannot directly disable, based on a leak report, an AccessKey pair that does not belong to your currently logged-on account. Handle it as follows:
If you know the account that owns the AccessKey pair, contact the account holder and ask them to handle it.
If you have access to the account that owns the AccessKey pair, use the
DeleteAccessKeyAPI or the RAM console to delete the key yourself.For leaked AccessKey pairs that belong to other accounts you cannot access, coordinate handling through legal or security compliance channels.