Reclaim disk fragments to free up disk space

Which situation are you encountering?

Use the following table to quickly identify your situation and navigate to the relevant section:

Trigger conditions and rules

The system automatically reclaims fragments only from collections that simultaneously meet the following conditions:

• The combined size of index space and data space exceeds 1 GB.

• The fragmentation rate exceeds 20%.

Additional rules:

• The upper limit for a single round of reclamation is 100 GB. Any excess is reclaimed in subsequent rounds.

• Only Hidden nodes are reclaimed. Hidden nodes do not serve business traffic, so reclamation does not affect the primary or secondary nodes.

• The thresholds above are built into the system and cannot be customized.

Configuration entry point

1. Log on to the MongoDB console and navigate to the target instance.

2. In the left-side navigation pane, choose CloudDBA > Storage Analysis.

3. In the Data Space list, locate the Fragmentation Rate column and click the Recycle link.

4. In the Fragment Recycling Plan dialog box that appears, click Create Plan and confirm.

Procedure

1. Navigate to the target instance. In the left-side navigation pane, choose CloudDBA > Storage Analysis.

2. In the Data Space list, view the fragmentation rate and reclaimable space for each collection.

   Important

   The reclamation operations in the console storage analysis, as well as the fragmentation rate and reclaimable space displayed on the page, target Hidden nodes by default.

3. For collections with high fragmentation rates that you want to reclaim, click the corresponding Recycle button to execute.

Verify reclamation success

After reclamation is complete, run storage analysis again and check whether the fragmentation rate of the target collection has decreased. Make sure you are viewing the same node where reclamation was performed (see FAQ and troubleshooting).

Permission requirements

Running compact requires the account to have dbAdmin or hostManager (version > 8.0) permissions. Otherwise, a permission error will be reported:

not authorized on <database> to execute command { compact: ... }

Impact of compact on workloads

• Read/write blocking and performance impact

  • Before MongoDB 4.4: The compact command locks the database that contains the collection, blocking all read and write operations on that database. When fragmentation is severe, compact may take a long time to complete, which can cause replication lag on Hidden nodes. We recommend running it during off-peak hours, increasing the oplog size based on your write workload, or Upgrade database major version upgrading to MongoDB 4.4 or later before reclaiming fragments.

  • MongoDB 4.4 and later: The compact command no longer blocks read and write operations, but may affect performance during execution. We recommend running it during off-peak hours.

• Node rebuilding

  • MongoDB 3.4 (all versions), MongoDB 4.0 (all versions), MongoDB 4.2 early minor versions (4.0.22 or earlier), and MongoDB 4.4 early minor versions (5.0.6 or earlier): The node running compact enters the RECOVERING state. If this state persists for too long, the health check component may flag the node as unhealthy and trigger an automatic rebuild. For MongoDB version information, see MongoDB minor versions.

  • For instances running later versions: The node executing compact remains in the SECONDARY state and does not trigger a rebuild.

• Regardless of the version, always prioritize reclaiming fragments from Hidden or Secondary nodes to avoid impacting the primary node. Before reclaiming disk fragments, we recommend backing up your database.

compact execution time

The execution time depends on the data volume, system load, and other factors, and cannot be precisely estimated. Larger collections with more fragmentation take longer. Very large collections (hundreds of GB) may take significantly longer. We recommend using dryRun: true to estimate reclaimable space first, and running the operation during off-peak hours.

The following data is provided for reference only:

Real-world test (MongoDB 4.4): Running compact on a collection of approximately 2.5 GB with 50% fragmentation took about 5 seconds and freed approximately 1.9 GB of space.

Scenarios where compact is ineffective

The following scenarios may cause the compact command to have no effect:

• The physical collection size is less than 1 MB.

• The fragmentation rate is below 20%.

• Less than 20% of free space exists in the first 80% of the file, or less than 10% of free space exists in the first 90% of the file.

For more information, see block_compact.

Replica set instances

Connect to a Secondary node and run the following commands:

use <database>

// Before reclamation: check database space usage for comparison
db.stats()

// Dry run: estimate reclaimable space without actually reclaiming
db.runCommand({ compact: "<collection>", dryRun: true })

// Actual reclamation
db.runCommand({ compact: "<collection>" })

// After reclamation: check again to compare space
db.stats()

The dryRun: true parameter activates dry run mode, which returns only estimatedBytesFreed (the estimated number of bytes that can be freed) without modifying any files. After reviewing the dry run results, remove this parameter and run the actual reclamation. Run db.stats() before and after reclamation to compare the database space usage and confirm the result.

will not run compact on an active replica set primary as this will slow down
other running operations. use force:true to force

Sharded cluster instances

For sharded clusters, you only need to reclaim fragments from the corresponding nodes in the Shard component. Mongos and ConfigServer components do not store user data and do not require reclamation.

Read-only nodes in sharded clusters do not support the compact command, so you cannot reclaim fragments from read-only (ReadOnly) nodes.

Reclaiming Secondary node fragments: When running runCommandOnShard, you must set the read preference to Secondary. The syntax varies by client. Select the appropriate method for your client:

db.runCommand({runCommandOnShard:"<Shard ID>","command":{compact:"<collection_name>"}},{readPreference: "secondary"})

db.getMongo().setReadPref('secondary')
db.runCommand({runCommandOnShard:"<Shard ID>","command":{compact:"<collection_name>"}})

db.runCommand({runCommandOnShard:"<Shard ID>","command":{compact:"<collection_name>"},$queryOptions: {$readPreference: {mode: 'secondary'}}})

Reclaiming Primary node fragments (not recommended):

db.adminCommand({
  runCommandOnShard: "<shardId>",
  dbName: "<database>",
  command: { compact: "<collection>", force: true }
})

Reclamation executed but no visible effect

Check the following in order:

1. Node mismatch between operation and verification (most common cause): Console storage analysis reclaims fragments from Hidden nodes, and the page displays the fragmentation rate for Hidden nodes by default. Command-line compact affects the node you connected to. If you ran compact on the primary or secondary node but checked the fragmentation rate in the console storage analysis, the numbers will not change. Correct approach: Go to the Monitoring Information page and switch to the node where you actually performed the operation to view the disk space utilization.

2. Internal file fragmentation: compact can only truncate contiguous free space at the end of a file. Reusable space within the file cannot be reclaimed. If fragments are distributed inside the file, disk space may remain nearly unchanged after compact. This is expected behavior of the WiredTiger storage engine, not a failure. Run db.<collection>.stats() and check the ratio of freeStorageSize to storageSize to estimate the reclaimable space.

3. Below reclamation threshold: If the fragmentation rate is below 20%, the physical collection size is less than 1 MB, or the collection is small, the absolute reclaimable space is very small and reclamation may have no visible effect.

4. Primary node protection blocked the operation: If you see use force:true to force, this is the protection mechanism indicating that compact was run on the primary node. Do not use force. Instead, run compact on a Hidden or Secondary node, or use console storage analysis to reclaim Hidden nodes.

Why doesn't the fragmentation rate drop to 0%?

This is expected behavior. compact only reclaims contiguous free space at the end of a file. Reusable space within the file is preserved for future writes. Therefore, the fragmentation rate typically cannot drop to 0%. This is a design choice of the WiredTiger storage engine.

Error: not authorized / Unauthorized

Running compact requires dbAdmin or hostManager (version > 8.0) permissions. Otherwise, a permission error will be reported. See Permission requirements.

Error: Interrupted ... cache eviction pressure

Cause: During compact execution, the WiredTiger storage engine experiences cache eviction pressure. Older versions and small-specification instances have limited memory resources. When cache pressure becomes too high and the engine cannot evict pages in time, compact is interrupted and exits early.

Resolution: Retry during off-peak hours, trigger a primary/secondary failover and try again, or upgrade the instance specification before reclaiming.

Can I run compact when the instance is locked due to full disk?

Yes. When the disk is full, the instance enters a locked state. Write (insert) and delete (delete) operations are rejected with the error cloud instance error, disk locked..., but find, compact, and drop operations can still be executed.

Fastest recovery decision

Console utilization does not match alerts (for example, console shows 60% but alert triggers at 90%)

This is caused by different display dimensions, not a data error:

• Alerts trigger on the "highest single node": In a replica set, the disk utilization of Primary, Secondary, and Hidden nodes may differ (Secondary/Hidden nodes are often higher than Primary due to oplog, reclamation timing, temporary files, and other reasons). An alert triggers when any single node exceeds the threshold.

• The console does not show the "highest node" by default: The instance Basic Information page shows an aggregated value, and the Storage Analysis page defaults to showing the Hidden node. Both may be lower than the node that triggered the alert.

How to view the actual utilization of each node: Navigate to the instance Monitoring Information page, switch the view mode to Sub-node independent, and you can view the disk space utilization of each Primary / Secondary node individually to find the node with high usage that triggered the alert.

For more information on why primary and secondary node disk utilization differs and how to address it, see High disk space usage of ApsaraDB for MongoDB instances.

Can I downsize the disk to save money?

Downsizing is not supported. ApsaraDB for MongoDB does not support reducing the purchased disk space for any version. Even if you have cleaned up a large amount of data and reclaimed fragments, you can only maintain or increase the disk specification. You cannot directly reduce it.

If you need a smaller disk specification, the only option is to create a new instance with a smaller specification, migrate your data using DTS, and then release the original instance. Evaluate the migration cost and downtime window before proceeding.

For configuration change and specification limits, see Modify replica set instance configuration.

Why do disk fragments occur?

When data is deleted using the delete command or TTL expiration, it is only marked as deleted. The space it occupied is not immediately returned to the operating system. Instead, it is retained as free blocks for future writes. When deletions outpace writes, these free blocks remain unused for extended periods, forming fragments. This is why disk utilization remains high even after the data volume decreases.

When to reclaim disk fragments

Consider reclaiming disk fragments in the following situations:

• After deleting a large volume of data: When you delete a large number of documents, the freed space is not returned to the operating system but reserved for future writes, leaving significant fragmented space on the disk.

  Important

  Manual deletion (delete) and TTL expiration do not automatically reclaim disk fragments. Manual reclamation is required.

• After prolonged high-write workloads: Sustained high-write workloads (frequent inserts, updates, and deletes) gradually accumulate fragmented space on the disk.

• When disk space is low and fragmentation exceeds 20%: When disk utilization reaches 85% to 90% or higher, reclaiming fragments can free up space and reduce storage pressure.

View and estimate reclaimable space

Connect to the instance (for replica set instances, connect to a Secondary node to minimize service impact) and run db.runCommand({collStats: "<collection>"}) to view the storage status of a collection. Pay attention to the following fields:

• size: The logical storage size of the collection.

• storageSize: The physical storage size of the collection.

• freeStorageSize: The reclaimable free space within the collection (available in MongoDB 4.4 and later).

After deleting documents with the remove command, size decreases, but storageSize may not change. A higher ratio of freeStorageSize to storageSize indicates a higher fragmentation rate.

You can also run the following command to estimate the reclaimable fragment space of a collection (returns the number of reusable bytes):

db.<collection>.stats().wiredTiger["block-manager"]["file bytes available for reuse"]