Metadata cache-Object Storage Service(OSS)-阿里云帮助中心

Background information

When you query file metadata using ossfs 2.0, the system converts the query into multiple OSS requests to simulate local file system semantics. As the file path depth increases, the number of OSS requests grows exponentially. This can lead to high queries per second (QPS) consumption, which affects overall performance and increases the server-side load.

To reduce the request load and improve access efficiency, ossfs 2.0 uses multiple caching mechanisms, including a file attribute cache and a negative cache. The client caches positive results (the file exists) and negative results (the file does not exist). This caching reduces repetitive requests to OSS, which significantly lowers the request frequency and improves metadata access performance.

File attribute cache

When you first access a file or folder using a command such as stat, ossfs 2.0 sends an OSS request to retrieve its metadata. If metadata caching is enabled, ossfs 2.0 caches the retrieved metadata in memory. Subsequent metadata queries for that path are read from the cache first. This reduces repetitive requests to OSS.

The file attributes that can be cached include file size, modification time, and file type. For the file type attribute, only regular files and folders are supported.

The attr_timeout parameter controls the cache validity period. The default value is 60 s. During this period, access requests return the cached result. After the cache expires, ossfs 2.0 sends a new OSS request to retrieve the latest metadata.

By default, the operating system manages the cache. When memory usage reaches a certain level, which is dynamically calculated by the operating system, the OS instructs ossfs 2.0 to release some cached metadata. You can also control the number of cached file attributes by setting the max_inode_cache_count parameter.

Scenarios

This cache is suitable for read-heavy scenarios with controlled changes. You can enable the file attribute cache in the following scenarios:

The mounted bucket is read-only: No write or modification operations occur. The local cache and OSS data remain consistent.
Only one ossfs 2.0 process writes to the bucket: A single writer ensures that the source of metadata changes is unique, which prevents cache inconsistency issues.

Important
If multiple users modify the same bucket at the same time, you can enable the file attribute cache only if each user's operation path is isolated. The paths must not overlap or conflict. For example, you can create separate directory structures for each user or task.

Configuration items

ossfs 2.0 provides flexible configuration items to manage the file attribute cache for metadata.

Configuration item	Description
attr_timeout	The time in seconds before the metadata cache expires. The default value is 60 s. After expiration, a new request is sent to OSS for the latest metadata.
max_inode_cache_count	Enables active control over the number of metadata cache entries. The default value is 0. This means active control is disabled, and the operating system automatically evicts metadata. If set to a positive integer, active control is enabled. When the number of cache entries exceeds the set value, ossfs 2.0 evicts the excess entries based on specific rules.
readdirplus	Specifies whether to preload the metadata cache when you perform a readdir operation. true (default): Builds the metadata, such as size and modification time, for subdirectory items during a readdir operation. This consumes more memory because the metadata for subdirectory items is maintained in memory, but it significantly improves the performance of subsequent access. false: Retrieves only the list of filenames during a readdir operation without pre-building any metadata. Disabling this feature can lead to frequent metadata query requests, which affects performance.
close_to_open	Specifies whether to support close-to-open semantics. false (default): When you open a file, ossfs 2.0 decides whether to send a request to OSS for the latest file attributes based on the metadata cache expiration time. true: Sends an OSS request to get the file information from OSS when you open a file. This ensures real-time metadata. Enabling this option forces a request to OSS every time a file is opened. This significantly increases access latency in scenarios where many small files are read.

Consistency

After you enable the file attribute cache, the metadata in the ossfs 2.0 mounted directory may become inconsistent with the data in the OSS bucket. By default, with attr_timeout set to 60 s, if a file in the bucket changes, you may have to wait up to 60 seconds to see the change when you use the stat command with ossfs 2.0.

In read-write scenarios, if files in OSS are updated frequently and you need real-time metadata when you open files, you can enable the close_to_open option. This bypasses the local metadata cache to ensure strong consistency.

Negative cache

A negative cache records the "file not found" status. This avoids repeated OSS requests for paths that are frequently accessed but do not exist.

When you query for a file that does not exist using ossfs 2.0, the system sends a request to OSS and receives a 404 Not Found response. If negative caching is enabled, ossfs 2.0 caches this result in memory instead of discarding it. During the cache validity period, subsequent queries for that path immediately return a "file or folder not found" error. No new OSS request is sent.

The oss_negative_cache_timeout parameter controls the expiration time of the negative cache. The default value is 0. During this period, access requests return the cached negative result. After the cache expires, ossfs 2.0 sends a new OSS request to update the cache.

The maximum number of negative cache entries is controlled by the oss_negative_cache_size parameter, which defaults to 10,000. If the number of entries exceeds this value, entries are automatically evicted based on a priority system.

Scenarios

You can enable negative caching in the following scenarios:

The mounted bucket is read-only: No write or modification operations occur. The local cache and OSS data remain consistent.
Many exploratory accesses occur before you create files in a batch: A program uses the stat command to probe multiple paths for configurations, resources, or plugins, and some of these paths may not exist.

Configuration items

ossfs provides flexible configuration items to manage the negative cache for metadata.

Configuration item

Description

oss_negative_cache_timeout

The expiration time for the negative cache, in seconds.

The default value is 0, which means negative caching is disabled.
If set to a positive integer, negative caching is enabled and the cache expires after the specified time.

oss_negative_cache_size

The maximum number of entries in the negative cache. The default value is 10000. When the number of cache entries exceeds this value, ossfs 2.0 evicts the excess entries based on specific rules.

Consistency

After you enable the metadata negative cache, if ossfs 2.0 caches a "not found" result (a 404 response) for a file, such as object-A, that file cannot be accessed during the cache validity period, even if the file is later created on the OSS server. You must wait for the oss_negative_cache_timeout period to expire. After the cache entry expires, commands such as stat can retrieve the file. Therefore, do not enable the negative cache feature in scenarios that require high data consistency.