Troubleshoot connection and network issues

更新时间:
复制 MD 格式

This topic helps you troubleshoot issues when connecting to an ApsaraDB for MongoDB instance. We recommend that you first check the Common errors section to quickly identify and resolve the issue. If you cannot find a matching error message, follow the Troubleshooting steps to diagnose and resolve the connection issue.

Troubleshooting steps

Step 1: Check network connectivity

Troubleshooting method

Use the following commands to test connectivity to the ApsaraDB for MongoDB instance.

  • Run the ping <domain> command. For example, run ping dds-xxxx.mongodb.rds.aliyuncs.com.

    ping dds-xxx.mongodb.rds.aliyuncs.com
    PING dds-xxx.mongodb.rds.aliyuncs.com (xxx): 56 data bytes
    64 bytes from xxx: icmp_seq=0 ttl=90 time=7.698 ms
    64 bytes from xxx: icmp_seq=1 ttl=90 time=7.085 ms
    64 bytes from xxx: icmp_seq=2 ttl=90 time=11.006 ms
    64 bytes from xxx: icmp_seq=3 ttl=90 time=12.154 ms
    64 bytes from xxx: icmp_seq=4 ttl=90 time=9.149 ms

    The sample output indicates that the client can connect to the ApsaraDB for MongoDB instance.

  • Run the telnet <domain> <port> command. For example, run telnet dds-xxxx.mongodb.rds.aliyuncs.com 3717.

    root@mongotest:~# telnet xxx.mongodb.rds.aliyuncs.com 3717
    Trying xxx...
    Connected to xxx.mongodb.rds.aliyuncs.com.
    Escape character is '^]'.

    The sample output indicates that the domain name is resolved correctly and that port 3717 is accessible.

Checklist

If the network test fails, use the following checklist to identify and resolve the issue.

Check the instance status. The instance must be in the Running state.

Check the whitelist settings. The client's IP address must be added to the whitelist of the ApsaraDB for MongoDB instance.

  • How to check:

    • (Recommended) Use the Connection Diagnosis feature of ApsaraDB for MongoDB to check IP connectivity. For instructions on how to run the check and resolve issues, see Connection diagnosis.

    • Temporarily add 0.0.0.0/0 to the instance's whitelist. If you can connect to the instance after this change, the original whitelist settings were incorrect.

      Important

      The IP address 0.0.0.0/0 allows any device to access the ApsaraDB for MongoDB instance, which poses a security risk. Use this IP address for testing purposes only and remove it from the whitelist immediately after testing.

  • Solution:

    Add the correct client IP address to the whitelist of the ApsaraDB for MongoDB instance. For information about how to find the public IP address of your local client, see FAQ.

Check the network environment and instance endpoint. ApsaraDB for MongoDB supports connections over a private network and the internet. You must use the appropriate endpoint for your network environment.

You can log on to the ApsaraDB for MongoDB console and view the endpoints of the instance on the Database Connection page.

Step 2: Authorization and authentication

Verify the account name and password, and check for special characters in the password.

  • Verify that the provided account name and password are correct.

  • Change the password. If you need to change the password, see Reset password.

    If you change the password after connecting to the ApsaraDB for MongoDB instance by using Data Management Service (DMS), go to the DMS console, right-click the target instance, and then select Edit to log on again.

  • Special characters. When you use the MongoDB Shell or application code to connect to an ApsaraDB for MongoDB instance, you must escape any special characters such as !@#$%^&*()_+= in the account name or password within the connection string. For more information, see How do I resolve a connection failure that is caused by special characters in a username or password?.

Check the authentication database. A database account must be authenticated against the database to which it belongs.

Check the instance endpoint. For a replica set instance that requires write operations, we recommend that you connect by using the ConnectionString URI endpoint.

  • Symptom: The instance suddenly becomes inaccessible, even though the endpoint has not been changed.

  • Cause: Only the primary node can perform write operations. If your application was previously connected by using the endpoint of the primary node, a failover can change the connected node to a secondary node, which causes write operations to fail.

  • Solution:

    • For production environments, we recommend that your application connects to the database by using a ConnectionString URI. This way, if a node fails, read and write operations are unaffected by failover. For endpoint details, see the following topics:

    • Manually switch node roles. Promote the node that corresponds to the single-node endpoint to the primary node.

    We recommend that you implement a reconnection mechanism and exception handling in your application. This ensures that the application can automatically reconnect after a transient disconnection and maintain stability.

Step 3: Other checks

Check if the instance performance meets your business requirements.

Check third-party tools.

If a third-party tool, such as Navicat or Compass, cannot connect to your ApsaraDB for MongoDB instance, the issue might be caused by incorrect instance or tool settings. Follow these steps to troubleshoot:

  1. Test the connection by using the MongoDB Shell.

    We recommend that you use the MongoDB Shell to connect to the instance by using its high-availability ConnectionString URI.

  2. Troubleshoot the issue based on the test result.

    • Connection successful: The instance is accessible. The issue is likely caused by incorrect settings in the third-party tool. Check the tool's configuration.

    • Connection failed: If the MongoDB Shell cannot connect, the instance settings may be incorrect. Follow the steps in this topic to check the network connectivity, authorization, authentication, and other settings.

Common errors

Error message

Cause and solution

network error while attempting to run command 'isMaster' on host 'dds-xxxx.mongodb.rds.aliyuncs.com:3717' :exception: connect failed

  • Timed out after 3000ms while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}. exception=(com.mongodb.MongoSocketReadException: Prematurely reached end of stream)

  • Socket recv() errno:54 Connection reset by peer x.x.x.x:27017

The instance may have reached its maximum connection limit and cannot establish new connections.

Solution:

  1. Check whether the number of connections is exhausted. For more information, see How do I query the number of connections?.

  2. Optimize connection usage. For more information, see What do I do if the number of connections to my instance reaches its limit?.

MongoDB.Driver.MongoWaitQueueFullException: The wait queue for acquiring a connection to server xxx is full.

The wait queue of the MongoDB driver is full. This can happen if the connection pool size is too small or if there are high-concurrency requests, which leaves no connections available.

Solution:

  1. Check the connection pool configuration in the application that reports the error. For example, make sure the connection pool is appropriately sized. For more information, see How do I limit the number of client connections?.

  2. If the issue persists after you adjust the application-side configuration, check whether the number of connections to the ApsaraDB for MongoDB instance is exhausted. For more information, see the following topics:

    1. How do I query the number of connections?

    2. What do I do if the number of connections to my instance reaches its limit?

(TooManyLogicalSessions) Unable to add session into the cache because the number of active sessions is too high.

Excessive concurrent connections can exhaust available sessions.

Solution:

  1. Troubleshoot the connection failure that is caused by exhausted connections.

    1. How do I query the number of connections?

    2. What do I do if the number of connections to my instance reaches its limit?

  2. If the number of connections is normal, check whether other performance metrics of the instance meet your business requirements.

    1. Use node monitoring to check resource usage, such as CPU and memory, to determine if the instance type meets your business requirements.

    2. If the instance type is too small for the workload, change the instance configuration during off-peak hours.

  • getaddrinfo failed.

  • No suitable servers found (`serverSelectionTryOnce` set).

Check whether the instance endpoint is correct. For more information about endpoints, see the following topics:

  • Failed to connect to 10.*.*.8:3717 after 5000 milliseconds, giving up.Error: couldn't connect to server 10.*.*.8:3717 (10.*.*.8), connection attempt failed

  • pymongo.errors.ServerSelectionTimeoutError: dds-xxxx.mongodb.rds.aliyuncs.com:3717: [Errno 113] No route to host,dds-xxxx.mongodb.rds.aliyuncs.com:3717

  • InvalidInstanceId.NotFound: The instance not in current vpc.

An ECS instance fails to connect to an ApsaraDB for MongoDB instance over a private network.

Solution:

  1. Check the whitelist settings. Make sure that the private IP address of the ECS instance is added to the whitelist of the ApsaraDB for MongoDB instance.

  2. Make sure that the ECS instance and the ApsaraDB for MongoDB instance can communicate with each other over the network.

    If the ECS instance and the ApsaraDB for MongoDB instance are in the same VPC, they can connect directly over the private network. If they are in different VPCs, use one of the following methods:

org.springframework.data.mongodb.UncategorizedMongoDbException: Timeout while receiving message; nested exception is com.mongodb.MongoSocketReadTimeoutException: Timeout while receiving message

  • "errmsg": "not master", "code": 10107, "codeName": "NotMaster"

  • "errmsg": "not master", "code": 10107, "codeName": "NotWritablePrimary"

  • Time out after 30000ms while waiting for a server that matches writableServerSelector.

  • Command failed with error 10107 (NotWritablePrimary): 'not primary' on server xxx.

  • Explain's child command cannot run on this node. Are you explaining a write command on a secondary?

  • not master and slaveOk=false.

  • MongoNotPrimaryException: Command failed with error 10107 (NotMaster): 'not master' on server xxx.

  • reason: TopologyDescription { type: 'ReplicaSetNoPrimary',...}

The node to which data is being written is not a primary node.

Cause: Write operations can be performed only on the primary node. If your application was previously connected by using the endpoint of the primary node, a failover can change the connected node to a secondary node, which causes write operations to fail.

Solution:

  • For production environments, we recommend that your application connects to the database by using a ConnectionString URI. This way, if a node fails, read and write operations are unaffected by failover. For endpoint details, see the following topics:

  • Manually switch node roles. Promote the node that corresponds to the single-node endpoint to the primary node.

We recommend that you implement a reconnection mechanism and exception handling in your application. This ensures that the application can automatically reconnect after a transient disconnection and maintain stability.

[Unauthorized] cloud instance error, disk locked, plz check and upgrade your disk quota,

Exhausted disk space has locked the instance.

Solution: Resolve instance locking or write failures caused by exhausted disk space.

(AuthenticationFailed) Authentication failed.

  • !xxx@dds-xxx.mongodb.rds.aliyuncs.com: event not found

The password of the database account contains special characters, such as !@#$%^&*()_+=.

Solution: How do I resolve a connection failure that is caused by special characters in a username or password?.

error getting cluster ID: (CommandNotFound) replSetGetConfig is forbidden by cloud provider for security reason

ApsaraDB for MongoDB does not support the replSetGetConfig command. For more information, see Limits on commands in ApsaraDB for MongoDB.

FAQ

Querying connections

The maximum number of connections varies by instance type.

Note

The maximum number of connections applies to each node in an instance. For example, if you purchase a three-node replica set instance with a 1-core CPU and 2 GB of memory, the maximum number of connections for both the primary node and each secondary node is 500. The hidden node does not provide services by design.

Console

  1. Go to the ApsaraDB for MongoDB Replica Set Instances page or the ApsaraDB for MongoDB Sharded Cluster Instances page. In the top navigation bar, select a region. Then, click the ID of the target instance.

  2. In the left-side navigation pane, click Monitoring Information.

  3. On the Monitoring Information page, view the Connections information for the instance.

    If the instance is a sharded cluster instance, you must select the Mongos that your service currently uses in the upper-right corner of the page.

Command

  1. Connect to the instance by using the MongoDB Shell.

  2. View the number of connections.

    Run the db.serverStatus().connections command.

    mgset-123456:PRIMARY> db.serverStatus().connections
    {
            "current" : 1,
            "available" : 999,
            "internal_current" : 10,
            "internal_available" : 990,
            "totalCreated" : 632
    }             

    Parameter description:

    • "current": the number of established connections.

    • "available": the number of available connections.

  3. Query the current connection sources.

    1. Switch to the admin database.

      use admin
    2. Run the db.runCommand({currentOp: 1, $all: true}) command.

      mgset-123456:PRIMARY> db.runCommand({currentOp: 1, $all:[{"active" : true}]})                    

    Analyze the command output to query the source IP address of each connection to determine the number of connections from each client. For more information, see the official MongoDB documentation about db.currentOp().

Limiting client connections

When you connect to a database by using a Connection String URI, you can set the number of connections in a connection pool by appending &maxPoolSize=<integer> to the URI. The following example shows how to use the MongoDB Shell to connect to a database and set the connection pool size to 10. In this example, the database account is test, and the authentication database is admin:

mongosh "mongodb://test:****@dds-bp19f409d7512****-pub.mongodb.rds.aliyuncs.com:3717,dds-bp19f409d7512****-pub.mongodb.rds.aliyuncs.com:3717/admin?replicaSet=mgset-6108****&maxPoolSize=10"
Note

For information about how to limit the connection pool size for clients in different languages, see the MongoDB Drivers documentation.

Handling connection limit

You can restart the instance, its components, or nodes to temporarily release all connections.

Important

When you restart an instance, its nodes are restarted in sequence. Each node experiences a transient disconnection of about 30 seconds. If the instance contains a large number of collections (more than 10,000), the disconnection may last longer. Before you restart the instance, plan accordingly and ensure that your application has a reconnection mechanism.

To prevent this issue from recurring, we recommend that you make the following adjustments after you restart the instance:

Special characters in credentials

When you connect to an ApsaraDB for MongoDB instance by using the MongoDB Shell or application code, the connection fails if the account name or password in the connection string contains unrecognized special characters. These special characters include !@#$%^&*()_+=.

Solution: Escape the special characters in the connection string. The following table describes the encoding rules.

! : %21
@ : %40
# : %23
$ : %24
% : %25
^ : %5e
& : %26
* : %2a
( : %28
) : %29
_ : %5f
+ : %2b
= : %3d

Example: If the password is ab@#c, you must escape the special characters in the connection string. The escaped password is ab%40%23c.

Specifying an authentication database

The authentication database stores database accounts and passwords. A database account must be authenticated against the database to which it belongs.

Assume that the username and password are both test, and the test account belongs to the testdb database.

High-availability endpoint

When using a ConnectionString URI or a read-only ConnectionString URI endpoint, you must set the authSource parameter to specify the database to which the database account belongs. If you do not configure this parameter, the database that is specified by <database> is used as the authentication database.

Examples:

  • After the test account is authenticated against the testdb database, the instance switches to the <database> database.

mongosh "mongodb://test:test@<host1>:<port1>,<host2>:<port2>,...,<hostN>:<portN>/<database>?replicaSet=<replicaSet_value>&authSource=testdb"
  • After the test account is authenticated against the testdb database, the instance switches to the testdb database.

mongosh "mongodb://test:test@<host1>:<port1>,<host2>:<port2>,...,<hostN>:<portN>/testdb?replicaSet=<replicaSet_value>"

Single-node endpoint

You can run one of the following commands to log on:

mongosh --host $myhost --port $myport -u test -p test --authenticationDatabase testdb

or

    mongosh --host $myhost --port $myport
    mongo> use testdb
    mongo> db.auth("test", "test")

For more information about the endpoint parameters, see Connect to a replica set instance and Connect to a sharded cluster instance.

Logging on with DMS

Supported client languages

ApsaraDB for MongoDB is fully compatible with MongoDB and supports all official MongoDB clients.

Supported languages include C, C++, C#, Java, Node.js, Python, PHP, and Perl. For more information, see the official documentation.

For sample code, see the following topics:

Note

To ensure proper authentication when you connect to an ApsaraDB for MongoDB instance, use a MongoDB driver of version 3.0 or later.

Public network access

Password-free access

ApsaraDB for MongoDB no longer supports enabling password-free access over a VPC.

If you have already enabled password-free access for an instance, you can disable this feature. This action cannot be reversed. For information about how to disable password-free access, see Disable password-free access over a private network.

Mongo Shell on Windows

  • Download and install the MongoDB Shell from the MongoDB Shell Download page.

  • Use the MongoDB Shell.

    • Open the Command Prompt application.

    • Connect to the ApsaraDB for MongoDB instance.

      Example:

      mongosh "mongodb://test:****@dds-bp19f409d7512****.mongodb.rds.aliyuncs.com:3717,dds-bp19f409d7512****.mongodb.rds.aliyuncs.com:3717,dds-bp19f409d7512****.mongodb.rds.aliyuncs.com:3717/admin?replicaSet=mgset-6108****"    

      For more information about the connection string parameters, see Connect to a replica set instance and Connect to a sharded cluster instance.

readPreference connection failure

In a shell environment, the ampersand (&) character is interpreted as a command to run a process in the background. This can cause the connection string to be truncated, causing the readPreference parameter to be unrecognized. To resolve this issue, enclose the entire connection string in quotation marks. The following code provides a correct example:

mongosh "mongodb://root:****@dds-bp10a572fae7a****.mongodb.rds.aliyuncs.com:3717,dds-bp10a572fae7a****.mongodb.rds.aliyuncs.com:3717/admin?replicaSet=mgset-9202****&readPreference=secondary"

Common connection scenarios