Troubleshoot remote connection failures to a Windows instance

Use the self-service troubleshooting tool

The Alibaba Cloud self-service troubleshooting tool helps you quickly check security group configurations, the firewall on your instance, and the listener status of common application ports. The tool provides a detailed diagnostic report.

If the self-service troubleshooting tool does not identify the issue, use the following steps to troubleshoot it manually.

Manual troubleshooting

First, check the ECS instance status. Then, use Cloud Assistant to send commands or VNC to log on to the instance.

Step 1: Test remote logon with Workbench

Use Workbench to test the remote logon. It provides specific error messages and solutions if the logon fails.

1. Go to ECS console - Instances.

2. In the upper-left corner of the page, select a region and resource group.

3. On the Instances page, find the instance that you want to connect to and click Remote Connection in the Actions column.

4. In the Remote Connection dialog box that appears, click Log On Now that corresponds to Remote connect via Workbench.

5. Workbench automatically fills in the basic information required to log on to the target instance. Confirm that the information is correct and enter your username and credentials. Then, take the appropriate action based on the result:

   • If you still cannot log on, Workbench returns an error message and a solution. Follow the on-screen instructions to resolve the issue and then use Workbench to test the remote logon again. You can connect to the instance by using VNC to resolve common exceptions that occur when you use Workbench.

   • If you can connect by using Workbench but not from your local client, the remote connection port and service on the instance are working correctly. You must troubleshoot your local client.

Step 2: Check for blackhole notifications

Check whether you have received a blackhole filtering notification for the instance. During blackhole filtering, the instance cannot be accessed over the Internet. For more information, see Alibaba Cloud blackhole filtering policy.

Step 3: Check ports and security groups

Check whether security group rules are blocking the connection. To do so, perform the following steps:

1. Go to ECS console - Instances.

2. In the upper-left corner of the page, select a region and resource group.

3. On the Instances page, click the ID of the instance.

4. Click the Security Groups tab. In the Security Groups section, click Manage Rules in the Actions column for the target security group.

5. Select the direction of the security group rules.

6. On the Access Rule tab, use one of the following methods to add a security group rule.

   • Method 1: Quickly add a security group rule

     • Select RDP remote connection to Windows instance.

   • Method 2: Manually add a security group rule

     • Action: Allow

     • Priority: 1 (A lower value indicates a higher priority. 1 is the highest priority.)

     • Protocol: Custom TCP

     • Source > /Destination IPv4 CIDR Block: 0.0.0.0/0 (This value indicates all IP addresses.)

     • Destination Port > Range: Set this parameter to the RDP port (default: 3389).

7. Connect to the instance by using Remote Desktop Connection in the IP:Port format.

   In the Remote Desktop Connection dialog box, enter the address in the IP:Port format, such as 192.168.0.1:4389, in the Computer field and click Connect.

8. Run the following command to test whether the port is working as expected:

   telnet <IP> <Port>

   Note

   • <IP> indicates the IP address of the Windows instance.

   • <Port> indicates the RDP port number of the Windows instance.

   If you run the telnet 192.168.0.1 4389 command and the connection is successful, a result similar to the following one is returned.

   Trying 192.168.0.1 ...
   Connected to 192.168.0.1  4389.
   Escape character is '^]'

   If the port test fails, see Check port availability when the ping command succeeds but the port is unreachable for troubleshooting.

Step 4: Check the firewall configuration

1. Connect to the Windows instance by using VNC.

2. In the menu bar, choose Start > Control Panel.

3. Set View by to Small icons and click Windows Firewall.

4. In the Windows Firewall window, click Advanced settings.

5. Enable the firewall configuration.

   1. In the Windows Firewall with Advanced Security window, click Windows Firewall Properties.

   2. Select On (recommended) and click Apply(A).

      We recommend that you enable the firewall on the Domain Profile, Private Profile, and Public Profile tabs.

6. In the Windows Firewall with Advanced Security window, click Inbound Rules. In the right-side pane, scroll to the bottom, right-click Remote Desktop - User Mode(TCP-In), and then select Enable Rule(E).

Step 5: Check Remote Desktop Services

Check whether Remote Desktop Services is enabled on the Windows server.

1. Connect to the Windows instance by using VNC.

2. Right-click the Start menu and click System.

3. In the System window, click Remote settings.

4. In the Remote Desktop section, select Allow remote connections to this computer(L) and click OK.

5. Start the Remote Desktop Services service.

   From the Start menu, choose Administrative Tools > Component Services > Services (Local). In the right-side pane, find the Remote Desktop Services service and check whether it is running. If not, start the service.

6. Load the drivers and services on which Remote Desktop Services depends.

   For security purposes, some key services on which Remote Desktop Services depends may be disabled by mistake, which causes Remote Desktop Services to fail. You can perform the following steps to check for this issue.

   1. Right-click the Start menu, click Run, enter msconfig, and then click OK.

   2. In the System Configuration dialog box, click the General tab, select Normal startup(N), and then click OK.

   3. Restart the ECS instance.

Step 6: Check the remote terminal service configuration

You may be unable to connect to the remote desktop of a Windows instance due to an incorrect remote terminal service configuration.

Step 7: Check the network

If you cannot remotely connect to a Windows instance, you need to check whether the network is working as expected.

1. Use computers in other network environments, such as on different network segments or from different carriers, for a comparative test. This helps determine whether the issue lies with the local network or the server.

   • If the issue is with your local network or carrier, contact your local IT personnel or the carrier to resolve it.

   • If the network interface card (NIC) or its driver is not working correctly, ensure that the NIC is available and update its driver. To do so, perform the following steps:

     Resolve failures from a faulty NIC or driver

     Note

     This operation uses Windows Server 2016 as an example. The UI may differ on other versions. Adjust the steps accordingly for your operating system version.

     1. Connect to the Windows instance by using VNC.

     2. In the notification area, right-click the icon and select Open Network and Sharing Center.

     3. Click Change adapter settings to check whether the NIC is enabled.

        • If the NIC is disabled, right-click the NIC and select Enable(A). Check whether the remote connection to the Windows instance is restored.

        • If the NIC is enabled but still unavailable, proceed to the next step.

     4. Open the Run window, enter regedit, and then click OK.

     5. In Registry Editor, navigate to HKEY_LOCAL_MACHINE > SYSTEM > CurrentControlSet > Control > Class > {4d36e972-e325-11ce-bfc1-08002be10318} and check whether the following information exists in the right-side pane. If not, right-click a blank area and choose New(N) to add it.

        Important

        After you modify information in Registry Editor, you must restart the system for the changes to take effect.

        • Name: Installer32

        • Type: REG_SZ

        • Data: NetCfgx.dll,NetClassInstaller

     6. Open the Run window, enter devmgmt.msc, and then click OK.

     7. On the Device Manager page, expand Network adapters, right-click the NIC device, and then select Update Driver Software(p)....

     8. In the dialog box that appears, click Search automatically for updated driver software(S). After the update is complete, click Close.

     9. Check again whether the remote connection to the Windows instance is restored.

2. Run the ping command on the local client to test the network connectivity to the instance.

   • If a network exception occurs, see Use a packet capture tool to capture network packets for troubleshooting.

   • If ping packets are lost or the ping command fails, use MTR to analyze network links for troubleshooting.

   • If intermittent packet loss occurs and the network of your ECS instance is unstable, see Link interruption to resolve the issue.

3. If the General failure error is reported when you ping the client from the instance, see The "General failure" error is reported when you ping an external network from a Windows instance to resolve the issue.

Step 8: Check CPU, bandwidth, and memory

High CPU load, insufficient bandwidth, or low memory can cause remote connection failures.

1. Check for high CPU load and take appropriate action.

   • If the CPU load is not high, proceed to the next step.

   • If the CPU load is high, resolve the issue by performing the following checks:

     • Log on to the instance from the terminal on the Instance Details page and check whether Windows Update is running in the background. If Windows Update is running, a high CPU load is expected. Wait for the update to complete.

     • If the applications on an instance perform many disk read/write operations, initiate many network requests, or generate compute-intensive workloads, a high CPU load is expected. We recommend that you upgrade the instance type to resolve resource bottlenecks.

       Note

       For more information about how to resolve high CPU load, see Troubleshoot high CPU utilization on Windows ECS instances.

2. Check for insufficient public bandwidth.

   Remote connection failures may be caused by insufficient public bandwidth. Perform the following steps to troubleshoot the issue:

   1. Go to ECS console - Instances.

   2. In the upper-left corner of the page, select a region and resource group.

   3. On the Instances page, click the ID of the instance. On the Instance Details tab, check the Public Bandwidth in the Configuration Information section.

      If the server bandwidth is 0 Mbit/s, no public bandwidth was purchased when you created the instance. You can upgrade the public bandwidth to resolve this issue.

3. Check for insufficient memory.

   After you remotely connect to a Windows instance, the desktop may not be displayed and the connection is terminated without an error message. This issue may be caused by insufficient server memory. You need to check the memory usage of the server. To do so, perform the following steps:

   1. Connect to the Windows instance by using VNC.

   2. Choose Start > Administrative Tools > Event Viewer and check for warning logs that indicate insufficient memory resources. For more information, see How do I troubleshoot insufficient virtual memory on a Windows Server 2003 or 2008 ECS instance?.

Step 9: Check the system security policy settings

Check whether any security policies on the Windows server are blocking remote desktop connections. To do so, perform the following steps:

1. Connect to the Windows instance by using VNC.

2. Choose Start > Control Panel > Administrative Tools and double-click Local Security Policy.

3. In the Local Security Policy window, click IP Security Policies on Local Computer and take the appropriate action based on whether a relevant security policy exists.

   1. If a relevant security policy exists, delete or edit it.

      • To delete the security policy, right-click the policy and select Delete(D). In the dialog box that appears, click Yes.

      • Double-click the IP security policy to reconfigure it to allow remote desktop connections, and then try to connect again.

   2. If no relevant security policies exist, proceed to Step 9: Check the system's security policy settings to continue troubleshooting.

Step 10: Check the antivirus software

Third-party antivirus software settings can cause remote connection failures. This section provides troubleshooting methods and two case studies involving Server Safe Dog.

• If antivirus software is running in the background, log on to the instance by using VNC and upgrade the antivirus software to the latest version or uninstall it. For more information about how to log on to an instance by using VNC, see Select a method to connect to an ECS instance.

• Use a commercial antivirus software or the free Microsoft Safety Scanner to scan for and remove viruses in safe mode. For more information about security scanners, see Microsoft Safety Scanner.

Step 11: Fix incorrect registry configurations

Incorrect Windows registry configurations can block RDP connections. Use the following steps to fix them:

1. Connect to the Windows instance by using VNC.

2. In the Run dialog box, enter regedit and click OK to open Registry Editor.

3. In Registry Editor, modify the following parameter configurations:

   • Change the value of the fEnableWinStation parameter in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp to 1.

   • Change the value of the fDenyTSConnections parameter in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server to 0.

Step 12: Renew an expired RDP self-signed certificate

An expired RDP self-signed certificate can cause remote logon failures. Use the following steps to fix this issue:

1. Connect to the Windows instance by using VNC.

2. Run Windows PowerShell as an administrator.

3. In the Windows PowerShell dialog box, run the following command to check whether the current certificate has expired:

   Get-Item 'Cert:\LocalMachine\Remote Desktop\*' | Select-Object NotAfter

4. If the certificate has expired, run the following command to delete the self-signed certificate and restart the TermService service:

   Remove-Item -Path 'Cert:\LocalMachine\Remote Desktop\*' -Force -ErrorAction SilentlyContinue
   Restart-Service TermService -Force

   After the TermService service is restarted, the system automatically generates a new self-signed certificate.

5. Run the following command to confirm that the new self-signed certificate is updated:

   Get-Item 'Cert:\LocalMachine\Remote Desktop\*' | Select-Object NotAfter

   Note

   The default validity period of an RDP self-signed certificate is six months.

A specific error message is returned

• Protocol error when you remotely connect to a Windows ECS instance

• Authorization-related issues:

  • Enabling password-protected sharing causes the 'Your credentials did not work' error during remote access to a Windows system

  • The "The connection was denied because the user account is not authorized for remote login" error is reported when you connect to an ECS instance by using Remote Desktop

  • Solution for an unspecified Remote Desktop license server in a Windows instance

  • What do I do if the "To log on to this remote computer, you must be granted the Allow log on through Terminal Services right" error is reported when I connect to a Windows instance by using Remote Desktop?

• Connection count-related issues:

  • "The terminal server has exceeded the maximum number of allowed connections" error is reported when you remotely connect to a Windows instance

Related documents

• Troubleshoot remote connection failures to a Linux instance

• VNC connection issues

• SSH connection issues

• RDP connection issues