Troubleshoot cluster access issues in the ACK console

Cluster API server request errors: ErrorQueryClusterNamespace or APIServer.500

Symptom

Cause

Abnormal configuration or status of the API server load balancer breaks connectivity between the Container Service control plane and the cluster API server.

Solution

1. Log on to the ACK console. In the left navigation pane, click Clusters.

2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Cluster Information.

3. On the Basic Information tab, click the API server SLB instance ID to open the SLB console.

   • If the UI shows that the load balancer ID does not exist, the API server load balancer was deleted or released and the cluster cannot be recovered. See Create an ACK managed cluster to recreate it.

   • If the instance exists, continue to the next step.

4. Check whether the Status of the load balancer is Running.

   • If not, check whether the instance is overdue or expired. If so, renew and enable it. See Overdue payments for SLB billing details.

   • If yes, continue to the next step.

5. Click Listeners and verify that frontend and backend TCP:6443 listeners exist and Status is Running.

   • If not, the API server load balancer listener was modified.

     • If the listener is in the Disable state, select the listener and then click Enable.

     • If the listener does not exist:

       • For an ACK managed cluster: Submit a ticket.

       • For an ACK dedicated cluster: Add all cluster master nodes to the default server group, then add a TCP listener with frontend and backend set to TCP:6443. Set the backend to the Default Server Group and enable the listener.

   • If yes, continue to the next step.

6. Check whether the Health Check Status of the listener is Healthy.

   • If not, the API server load balancer backend is abnormal.

     • For an ACK managed cluster: Submit a ticket.

     • For an ACK dedicated cluster: Troubleshoot as described below. If the issue persists, submit a ticket.

       • In the ACK console, go to Nodes > Nodes. For each master node, click its ECS instance ID to open the ECS console and verify Running status.

       • In the ECS console, remotely log on to each master node and verify that the API Server container is running.

         1. To log on to a master node, see Connection methods.

         2. Check API server container status based on the container runtime.

            • Docker runtime: Run docker ps | grep kube-apiserver, then use docker inspect to check container status from the output.

            • containerd runtime: Run crictl ps | grep kube-apiserver, then use crictl inspect to check container status from the output.

   • If yes, continue to the next step.

7. Check whether access control is enabled for the listener.

   • If so, the access control whitelist for the API server load balancer listener is misconfigured. Add address block 100.104.0.0/16 to the whitelist. This block is the source address range the ACK control plane uses to access the API server internally.

   • If no, continue to the next step.

8. If none of the above apply, submit a ticket.

API server request errors when accessing pod logs

Account lacks cluster RBAC permissions for the operation: ForbiddenQueryClusterNamespace or APISERVER.403

Symptom

Cause

Your account lacks the required cluster RBAC permissions.

Solution

1. Log on to the ACK console as an Alibaba Cloud account or administrator. In the left navigation pane, choose Authorizations.

2. On the RAM Users tab, find the affected user and click Modify Permissions next to the username.

3. Click Add Permissions, select the cluster, namespace, and permissions, then click Submit.

Account lacks RAM permissions for the operation StatusForbidden

Symptom

Cause

Your account lacks the required RAM permissions.

Solution

1. Log on to the RAM console as an Alibaba Cloud account or an account with RAM permissions.

2. Add the required permission, such as cs:DescribeKubernetesVersionMetadata, based on the CS information on the error page. See Grant permissions on clusters and cloud resources by using RAM.