Random write configuration and working principles for the ossfs 2.0 local temporary directory-Object Storage Service(OSS)-阿里云帮助中心

ossfs 2.0.9 and later support random writes based on a local temporary directory. By configuring the --temp_dir mount option, ossfs 2.0 stages written data on a local disk, giving the mount point full POSIX random write semantics such as writes at arbitrary offsets, truncation to any size, and read-while-write. This topic describes the working principles, configuration methods, usage constraints, and recommendations for random writes.

Background information

Once an object is created in OSS, it cannot be partially modified. Any content change requires the entire object to be overwritten. In the default mode of ossfs 2.0, the write path uses streaming upload: data is uploaded to OSS in sequential parts as it is written, which has the following limitations:

Only sequential writes are supported. pwrite calls at positions beyond the end of the file are not supported.
truncate only supports truncating to 0. Truncating to any other non-zero size returns ENOTSUP.
A file cannot be read while it is being written (before it is closed). Doing so returns EBUSY.

Scenarios such as AI agent code execution, in-place script editing, and intermediate results of scientific computing commonly rely on random write semantics. The random write feature of ossfs 2.0 stages written data in a local temporary directory and, combined with chunk management and incremental upload, implements random writes on top of object storage.

The random write mode of ossfs 2.0 has the following characteristics:

Full POSIX random write semantics: Supports pwrite at arbitrary offsets, O_APPEND, and truncate to any size (grow or shrink).
Read-while-write: A file can be read while it is being written. Modified parts are read from the local staging area, and unmodified parts are read directly from OSS.
Incremental upload: On flush or close, only modified data blocks are uploaded from the local disk. Unmodified parts are handled through server-side copy on OSS, with zero local I/O and zero upload bandwidth consumption.
Disk space protection: A built-in global disk budget returns ENOSPC for writes when the staging disk is low on space. You can reserve a free space threshold at mount time to prevent the system disk from being filled up.

Note

The local temporary directory serves only the write path. It is independent of the local data cache (--disk_data_cache_dir, a read cache). You can enable both at the same time (using different directories) without any interference between them.

By default, ossfs 1.0 uses the /tmp directory as the local temporary directory. To keep the path consistent with the default path of ossfs 1.0, configure --temp_dir=/tmp.

Important

After random writes are enabled, write performance is limited by the read/write performance of the disk where the local temporary directory resides. In scenarios that involve bulk sequential writes, such as copying large files, performance is far lower than in the default mode. Enable this feature only when your workload genuinely requires random write semantics.

Working principles

Local staging and merged upload

After random writes are enabled, ossfs 2.0 creates a staging file in the local temporary directory for each file that is randomly written, saving the latest modifications locally at a fixed data block (chunk) granularity:

Written data lands directly in the local staging file. If a write only partially overlaps a data block that already exists in the cloud, ossfs first fetches that block from OSS to ensure the locally staged data is complete.
On fsync or close, locally staged modifications are automatically merged and uploaded with the data in the cloud: modified parts are uploaded from the local disk, and unmodified parts are handled through server-side copy on OSS, without retransmitting the entire file or consuming local disk read bandwidth.
After a successful upload, the staging disk space is automatically released.

Read-while-write

A file can also be read while it is being written: modified parts are read from the local staging area, and unmodified parts are read directly from OSS.

Disk space protection

At mount time, ossfs checks whether the free space on the disk where the temporary directory resides meets the reserved requirement, and rejects the mount if it does not. If the reserved disk space is insufficient, writes return ENOSPC instead of the process crashing due to a full disk.

Configuration methods

Configuration item	Required	Description	Default value
temp_dir	No	The path of the local temporary directory. Must be an absolute path. Setting a non-empty value enables random writes. Leave it empty to disable this feature.	Empty
temp_dir_free_bytes	No	The minimum free disk space reserved on the disk where the temporary directory resides, in bytes (supports units such as K, M, and G). ossfs2 will not use this reserved space for staging data: when the free disk space drops below this threshold, new writes return ENOSPC, leaving a safety margin for other workloads on the same disk. If the free disk space is already below this value at mount time, the mount fails. Minimum value: 64 MiB.	1 GiB
random_write_max_file_size	No	The maximum logical size of a single file in random write mode, in bytes (supports units such as K, M, and G). Writes or truncations that exceed this limit return EFBIG. The physical upper bound is the OSS multipart upload capacity (10,000 parts × 5 GiB, about 48.8 TiB).	100 GiB

Constraints:

If the directory that --temp_dir points to does not exist, it is created automatically. We recommend using an independent partition or disk to avoid contending for space with the system disk or other workloads.
Data for files being written is stored in the local temporary directory. If the disk lacks capacity, writes return ENOSPC. The remaining free disk space minus the space reserved by temp_dir_free_bytes is referred to as the writable disk space. ossfs2 only uses this portion of space to store data that is being written. The writable disk space must be greater than the total size of all files being written concurrently (in the worst case, an entire file may be written locally).
When multiple ossfs2 instances are mounted on the same machine, they are allowed to share the same temporary directory, because staging file names do not collide. However, multiple instances will contend for the disk's space and I/O, so we recommend using independent directories.
--temp_dir and --enable_appendable_object are mutually exclusive and cannot be enabled at the same time.

Example mount command:

ossfs2 mount /mnt/oss/ \
  --oss_bucket <your-bucket> \
  --oss_endpoint <your-endpoint> \
  --oss_access_key_id <ak> \
  --oss_access_key_secret <sk> \
  --temp_dir /mnt/disk/ossfs2/temp \
  --temp_dir_free_bytes 1G

Environment requirements

Parameter	Requirement
Disk type of the temporary directory	Elastic ephemeral disk, local NVMe, or ESSD is recommended. Disk write throughput directly affects random write latency and upload speed.
Disk available space	Not less than `temp_dir_free_bytes`. In addition, the writable disk space (see the constraints above) must be able to accommodate the data of files being written concurrently.

Effects of the feature

Comparison of write capabilities before and after enabling --temp_dir:

Operation	Default mode	Random write mode
Sequential append write	Supported	Supported
pwrite at an arbitrary offset (beyond the end of the file)	Not supported	Supported
truncate to 0	Supported	Supported
truncate to any size (grow or shrink)	Not supported (ENOTSUP)	Supported
O_APPEND write to an existing file	Supported	Supported
Reading a file while it is being written	Not supported (EBUSY)	Supported
Multiple handles writing to the same file concurrently	Not supported	Supported

Durability semantics:

By default, random write mode follows fsync semantics: fsync triggers a merged upload of local modifications with the data in the cloud. A successful return means the data has been persisted to OSS. Remaining data is also uploaded when the file is closed.
An upload failure is returned to the application as an error, and the application can retry through fsync. If the upload ultimately fails and the file is closed, the unpersisted data is lost, and the file content reverts to the last successfully uploaded version on the remote end.
If the process exits abnormally, unuploaded staged data is lost, but the object in OSS remains in the state of its last successful upload. No corrupted object is produced.

Usage recommendations

Random writes based on a local temporary directory are suitable for scenarios that require local-filesystem-like write semantics on an OSS mount point.

Scenarios suitable for enabling random writes:

Scenario	Usage recommendation
AI agent code execution / sandbox environments (agents and code interpreters perform arbitrary reads and writes, and in-place edits, on files)	Enable random writes. We recommend using an independent, high-performance disk for the temporary directory, and reserving writable disk space based on the total size of files being written concurrently.
Intermediate result files of scientific computing or simulation applications	Enable random writes. Plan writable disk space based on the total size of files being written concurrently.

Scenarios where enabling random writes is not recommended:

Scenario	Usage recommendation
Copying large files (bulk writes to the mount point using cp, rsync, and similar tools)	In this scenario, write performance is limited by the local disk and is far lower than in the default mode. Use the default streaming upload mode instead.
Pure sequential append writes (streaming log writes)	The default streaming upload path has lower overhead, so enabling random writes is unnecessary. If read-while-write is required, consider `--enable_appendable_object` instead.

Disk budget recommendations:

Keep the temporary directory separate from the system disk to prevent staged data from filling the system disk and affecting host stability. If the machine has only one disk, we recommend setting temp_dir_free_bytes to a larger value (for example, 10G) to reserve sufficient free space for the system.
The writable disk space (see the constraints above) must be greater than the total size of files being written concurrently.
For a single file larger than 100 GiB, in addition to explicitly increasing --random_write_max_file_size (for example, --random_write_max_file_size=500G), you also need to ensure that the writable disk space exceeds the size of that file.