Store application logs with ossfs

Background

ossfs is a FUSE-based tool provided by Alibaba Cloud that mounts an OSS bucket as a local file system. ossfs 1.0 uses a single-part PUT model: it writes incoming data to a local disk cache and triggers the upload to OSS only when the file handle is closed. For details, see File uploads.

An application log file opens when the application starts and closes only when the application exits. This open-file lifecycle is incompatible with ossfs 1.0's upload-on-close behavior, which is why a bridging strategy is required.

Recommended practice: log locally and archive with ossfs

Write application logs to a local directory, rotate them periodically into closed files, then move the rotated files to the ossfs mount path. ossfs uploads each closed file to the OSS bucket automatically.

1. Configure your application to write logs to a standard local directory—not the ossfs mount path. This keeps logs immediately readable for real-time troubleshooting during runtime.

2. Set up a log rotation utility such as logrotate to split logs periodically by time or size. Use the create directive instead of copytruncate so that each rotated file is fully closed before the next write cycle begins. The copytruncate directive keeps the original file open and may produce incomplete objects in OSS.

3. Move or copy the rotated log files to the ossfs 1.0 mount path. Because rotated files are closed and complete, this step aligns with the upload-on-close behavior of ossfs 1.0, ensuring timely and durable storage in OSS. To automate this step, add a postrotate or lastaction block to your logrotate config:

   /var/log/myapp/*.log {
       daily
       rotate 7
       create
       compress
       postrotate
           mv /var/log/myapp/*.log.1 /mnt/ossfs/logs/
       endscript
   }

   Replace /var/log/myapp/ with your application's log directory and /mnt/ossfs/logs/ with the ossfs mount path.

This pattern keeps active logs on the local disk for real-time access and archived logs in an OSS bucket for long-term retention and cross-node access, while eliminating the three risks described below.

Why direct logging to ossfs is not recommended

Writing logs directly to a path mounted by ossfs 1.0 produces three failure modes:

1. Logs are not visible in OSS during runtime. An application log stays open from startup to exit. Because ossfs 1.0 uploads only when the file is closed, log data remains in the local cache throughout the entire runtime and never appears in the OSS bucket, making real-time troubleshooting impossible.

2. Local disk fills up. As the application writes logs, ossfs 1.0 accumulates temporary files in its cache directory. Over time, this consumes all available local disk space.

3. Data loss on abnormal exit. Before a file is closed, data written through ossfs 1.0 exists only in temporary local files and has not been persisted to OSS. If the host crashes or loses power and the ossfs 1.0 process exits abnormally, all unclosed log files are lost.