Chained replication performance test-ApsaraDB for MongoDB(MongoDB)-阿里云帮助中心

Chained replication can reduce primary node load but may increase replication lag. This document benchmarks its performance impact on ApsaraDB for MongoDB replica set instances with 3, 7, and 15 nodes to help you choose the optimal configuration.

Chained replication

What is chained replication?

MongoDB supports chained replication. Chained replication occurs when a secondary node in a replica set syncs data from another secondary node. This can reduce the load on the primary node but may increase replication lag in certain network topologies. For more information, see Self-Managed Chained Replication.

Chained replication does not require all nodes to form a single chain. A secondary node can select a sync source other than the primary node based on metrics such as round-trip time (RTT). The diagram below shows several five-node replica set topologies that use chained replication.

Configure chained replication

You can enable or disable chained replication by adjusting the settings.chainingAllowed parameter on the Parameter Settings page in the console. For more information about how to adjust parameters, see Set database parameters.

If chained replication causes replication lag, you can disable it to optimize sync performance.

Note

For security reasons, you cannot run the replSetReconfig command on ApsaraDB for MongoDB instances. You must modify parameters in the console.

Instance performance impact test

Test environment

Create an ECS instance and an ApsaraDB for MongoDB replica set instance. For more information, see Create a replica set instance and Create an ECS instance by using the wizard.

ApsaraDB for MongoDB instance architecture: a standard three-node replica set consisting of one primary node, one secondary node, and one hidden node. Larger configurations include additional secondary and read-only nodes.
Network round-trip time (RTT) between the ECS and ApsaraDB for MongoDB instances: 0.103 ms on average when both instances are in the same region and zone.

The following table lists the instance configurations used in this test.

Parameter	ECS instance	ApsaraDB for MongoDB instance
Region and zone	China (Beijing) zone H	China (Beijing) zone H
Network type	VPC	VPC
Instance family	Compute-optimized c6	Dedicated
Instance type	ecs.c6.xlarge (4 vCPU, 8 GiB)	ecs.c7.xlarge (4 vCPU, 8 GiB)
Storage type	ESSD PL0	ESSD PL1
Operating system	Alibaba Cloud Linux 3.2104 LTS 64-bit	4.19.91-26.al7.x86_64
MongoDB version	N/A	Major version: 5.0 Minor baseline version: 5.0.30

Test tool

This test uses the open-source YCSB 0.17.0 benchmark tool.

Note

Yahoo! Cloud Serving Benchmark (YCSB) is a Java-based tool for benchmarking database performance. For installation and usage instructions, see the official YCSB repository.

Procedure

Add an IP address to the whitelist. Log on to the ECS console, find the primary private IP address of your ECS instance on the instance details page, and add it to the whitelist of your ApsaraDB for MongoDB instance.
Connect to the ECS instance.
Use YCSB to load the test data.
```
./bin/ycsb.sh load mongodb -s -p workload=site.ycsb.workloads.CoreWorkload -p recordcount=5000000 -p mongodb.url="mongodb://test:****@dds-bp13e84d11****.mongodb.rds.aliyuncs.com:3717/admin" -p table=test -threads 8
```
Parameters:
- recordcount: the total number of records to load into the ApsaraDB for MongoDB instance.
- mongodb.url: the connection string of the ApsaraDB for MongoDB instance. In this test, the database account is test and the database is admin. You can find the connection string on the Database Connection page under the Internal Connections - VPC section in the ApsaraDB for MongoDB console.
- threads: the number of concurrent client threads.
View the YCSB test results and instance monitoring data. On the Monitoring Information page, go to the Node Monitoring tab. Select the time range for the test and view the CPU utilization, QPS, and average response time metrics for the primary node. For more information, see Node Monitoring (formerly basic monitoring).

Test results

Parameters

Write Concern: the level of acknowledgment requested from MongoDB for write operations. This test uses the following values:

{w:"majority"}: the default setting. A write is acknowledged after it is replicated to a majority of the voting nodes in the replica set.
{w: 1}: a write is acknowledged after the primary node confirms it.

Detailed test results

3-node instance

Replica set topology: one primary node, one secondary node, and one hidden node.

Write concern: `{w:"majority"}`

Item	Chained replication enabled	Chained replication disabled
Throughput (ops)	5277	5241
CPU utilization	65%	65%
QPS (count)
Average response time (μs)

Write concern: `{w:1}`

Item	Chained replication enabled	Chained replication disabled
Throughput (ops)	15075	14785
CPU utilization	93%	93%
QPS (count)
Average response time (μs)

7-node instance

Replica set topology: one primary node, five secondary nodes, and one hidden node.

Write concern: `{w:"majority"}`

Item	Chained replication enabled	Chained replication disabled
Throughput (ops)	3005	4312
CPU utilization	56%	85%
QPS (count)
Average response time (μs)

Write concern: `{w:1}`

Item	Chained replication enabled	Chained replication disabled
Throughput (ops)	14414	11492
CPU utilization	91%	93%
QPS (count)
Average response time (μs)

15-node instance

Replica set topology: one primary node, five secondary nodes, one hidden node, and eight read-only nodes.

This configuration has 7 voting nodes and 8 non-voting nodes (read-only nodes do not vote).

Write concern: `{w:"majority"}`

Item	Chained replication enabled	Chained replication disabled
Throughput (ops)	2932	3123
CPU utilization	58%	91%
QPS (count)
Average response time (μs)

Write concern: `{w:1}`

Item	Chained replication enabled	Chained replication disabled
Throughput (ops)	14093	7500
CPU utilization	90%	94%
QPS (count)
Average response time (μs)

Performance comparison and summary

In the performance comparison charts, "true_majority" means chained replication is enabled and the write concern is set to {w:"majority"}.

For a fixed number of nodes, the write concern setting determines whether disabling chained replication degrades write performance.
Write concern: {w:1}
- For a 3-node instance, the performance degradation from disabling chained replication is negligible.
- For a 7-node instance, disabling chained replication causes a performance degradation of approximately 20.3%.
- For a 15-node instance, performance degrades by 46.8%, and the primary node's CPU utilization increases significantly.
Write concern: {w:"majority"}
- For a 3-node instance, the performance degradation from disabling chained replication is negligible.
- For 7-node and 15-node instances, disabling chained replication can improve performance by approximately 6.5% to 43.5%.
  
  Reason for performance improvement: disabling chained replication shortens the sync path for all nodes in the replica set, making it easier to meet the majority condition and reducing the latency of each write operation.
  
  Impact of non-voting nodes: with a fixed number of 7 voting nodes, the performance gain from disabling chained replication diminishes as more non-voting nodes are added because of the increased replication load on the primary node.
With the same chainingAllowed and writeConcern settings, write performance decreases as the number of nodes increases. The degradation is more significant when chained replication is disabled.
- With the default configuration (chainingAllowed:true and writeConcern:{w:"majority"}), the performance degradation is not significant when expanding from 7 to 15 nodes because a replica set can have a maximum of 7 voting members, so the majority condition does not change.
- Performance with writeConcern:{w:1} is significantly better than with writeConcern:{w:"majority"}, regardless of whether chained replication is enabled. This is expected given how write concerns work.
- With a fixed write concern setting, disabling chained replication increases the primary node's CPU utilization more significantly as the number of nodes grows than when it is enabled.
Grouped by configuration

Grouped by node count

Best practices

For deployments with a small number of nodes, enabling or disabling chained replication has no significant impact on instance performance or CPU utilization.
For deployments with a large number of nodes:
- If writeConcern is {w:1}, we recommend that you enable chained replication.
- If writeConcern is {w:"majority"}, you must balance primary node load (such as CPU utilization) against instance performance. Disabling chained replication improves write performance but significantly increases the load on the primary node.