Logtail limits

Limits

Limitations

Architecture

• Logtail for Linux supports x86_64 and arm64.

• Logtail for Windows supports x86_32 and x86_64.

Compute resources

• CPU: Reserve at least 0.1 core.

• Memory: 30 MB minimum for hosts, 150 MB for Kubernetes, 100 MB for Kubernetes Controller/Operator.

Actual usage depends on collection rate, monitored directory and file count, and send blockage. Keep usage below 80% of the limit.

System environment

Supported operating systems are listed in System types.

Kubernetes

• DaemonSet log collection requires Kubernetes 1.10.0 or later with HostToContainer mount propagation support.

• CRD-based log collection (alibaba-log-controller) requires Kubernetes 1.16.0 or later.

  Kubernetes 1.7.0 and later also supports CRDs via apiextensions.k8s.io/v1beta1, but the stability of this beta API varies by version.

Docker

Limits for container standard output collection:

• You must add "log-driver": "json-file" to the Docker configuration file daemon.json.

• For CentOS 7.4 and later (except CentOS 8.0), you must set fs.may_detach_mounts=1. For more information, see Bug 1468249, Bug 1441737, and issue 34538.

Storage medium

Avoid shared network storage (NAS, OSS), which can cause data truncation, content inconsistency, or collection suspension. Use Elastic Block Storage (EBS).

Limits

Limitations

Size of a single log

Default: 512 KB, adjustable up to 8 MB with the max_read_buffer_size startup parameter. Set Logtail startup parameters.

Multi-line logs split by a first-line regex are still subject to the 512 KB limit per log. Logs exceeding 512 KB are forcibly split. For example, a 1,025 KB log is split into 512 KB + 512 KB + 1 KB, resulting in multiple incomplete logs.

File encoding

UTF-8 and GBK encodings are supported. UTF-8 is recommended for optimal performance.

Log file size

Unlimited.

Log file rotation

Default rotation queue size: 20, adjustable with the logreader_max_rotate_queue_size startup parameter. Set Logtail startup parameters.

If more than 20 files are unprocessed, new logs are lost. Check whether the Logstore shard write quota is exceeded, then adjust Logtail concurrency. Recommended parameter values.

Related technical articles.

Collection behavior when log parsing is blocked

When log parsing is blocked, Logtail keeps the log file descriptor open to prevent file deletion and data loss.

If the log file is rotated multiple times while parsing is blocked, Logtail places the file in the rotation queue.

Regular expression

Perl-compatible regular expressions are supported.

JSON

Standard JSON, as defined in RFC 7159 and ECMA-404, is fully supported. Non-standard JSON, such as {"name": "\xE5\xAD\xA6"}, is not supported.

Multiple Logtail configurations for one file

A file matches only one Logtail configuration by default. How to collect multiple copies of logs from a single file.

File opening behavior

Logtail keeps collected and pending files open for data integrity. A file closes when:

• The file has not been modified for more than 5 minutes.

• The file is rotated and collection is complete.

• The Logtail collection configuration changes.

To release file handles on a schedule after deletion, set the force_release_deleted_file_fd_timeout startup parameter. Set Logtail startup parameters.

Initial log collection behavior

Logtail collects only incremental data. On first detecting a file modification, if the file exceeds 1 MB (512 KB for container stdout), collection starts from the last 1 MB. Otherwise, collection starts from the beginning.

Adjust this with the tail_size_kb parameter. Logtail configurations (Legacy).

Logtail skips unmodified files after configuration is applied. Import historical log files.

File overwriting behavior

Logtail identifies files by inode and the hash of the first 1,024 bytes. If either changes after an overwrite, the file is treated as new and collected from the beginning. Otherwise, it is skipped.

File moving behavior

A moved file that matches a previously unmatched Logtail configuration is treated as new and collected from the beginning. Otherwise, it is skipped.

File collection history

Logtail tracks file collection history in memory to collect only incremental data. Writing beyond the retention scope causes duplicate collection.

• By default, the history of files from the last month is retained.

• If there are more than 5,000 historical files in the same directory, only records from the last week are retained.

• If there are more than 10,000 historical files in the same directory, only records from the last day are retained.

Non-standard text logs

For log lines containing \0, versions later than 2.1.10 and 3.0.12 retain only the mid-log \0 and discard leading and trailing \0. Earlier versions may truncate at the first \0 or retain it entirely. Upgrade Logtail for consistent behavior. Other escape characters (ASCII color codes, non-printable characters) are reported as-is.

Limits

Limitations

Initial log collection behavior

For container stdout, when a file modification is first detected, collection starts from the last 512 KB if the file exceeds 512 KB, or from the beginning otherwise. Adjust with the StartLogMaxOffset parameter. Collect container standard output in DaemonSet mode using the console.

Symbolic link

Symbolic links for directories and files are not supported in container file collection.

Container lifecycle

Minimum container lifecycle: 10 seconds. For container file collection, Logtail limits container updates to 10 times per 3 minutes. Adjust with the docker_config_update_interval and max_docker_config_update_times startup parameters. Set Logtail startup parameters.

Standard output log rotation

Docker or kubelet rotates container stdout files. Default rotation size: 10 MB for kubelet (100 MB for Docker on ACK). If stdout exceeds 10 MB/s, files rotate rapidly. To prevent data loss, collect from container files or increase the kubelet containerLogMaxSize parameter.

Standard output log driver

If you use Docker as a container runtime, you must add "log-driver": "json-file" to the Docker configuration file daemon.json.

Limitations

Limitations

Checkpoint timeout

Checkpoints are deleted after 30 days of file inactivity. With preserve:false, checkpoints are deleted after 30 minutes of inactivity.

Checkpoint storage policy

Checkpoints are saved every 15 minutes and on program exit. Adjust with the check_point_dump_interval startup parameter. Set Logtail startup parameters.

Checkpoint storage location

Default path: /tmp/logtail_checkpoint. Change with the check_point_filename startup parameter. Set Logtail startup parameters.

Handling during downtime

Recovery after downtime starts from the last saved checkpoint, which may cause duplicate collection. Adjust the checkpoint save policy to minimize duplicates.

Limits

Limits

Configuration update latency

Configuration updates from the console or API take about 30 seconds to reach the Logtail client.

Dynamic loading of configurations

Supported. An update to one configuration does not affect others.

Total number of loadable configurations for a single Logtail instance

No hard limit. Recommended maximum: 100 configurations per server.

Third-party flusher output

Console or API configurations are tied to a Logstore. When using a third-party flusher, Logtail also sends a copy to the associated Logstore by default.

Multi-account and cross-account

Supported. Configure a user identifier and Use Logtail to collect container logs across Alibaba Cloud accounts.

Multi-region

Not supported by default. To enable, submit a ticket.

Global Accelerator

Supported. Enable on the server side, then configure on the client. Enable Global Accelerator.

Limitations

Limitations

Number of machines

No hard limit. Maximum recommended: 100,000. Exceeding this causes incorrect heartbeat detection.

Number of applied configurations

No hard limit. Recommended maximum: 1,000.

Limits

Limitations

Log processing throughput

Default raw log traffic limit: 20 MB/s (typical compression ratio: 5-10x). Exceeding this limit may cause data loss. Adjust with the max_bytes_per_sec startup parameter. Set Logtail startup parameters.

Maximum performance

Single-core throughput:

• Simple mode: The maximum log processing capacity is 100 MB/s.

• Regex mode: The default maximum processing capacity is 20 MB/s. This depends on the complexity of the regular expression.

• The throughput for separator-delimited logs is capped at 40 MB/s.

• JSON mode: The maximum log processing capacity is 30 MB/s.

Multiple processing threads via the process_thread_count parameter can increase throughput by 1.5 to 3x.

Maximum number of monitored directories and files

Depends on the mem_usage_limit parameter (default: 384 MB for hosts, 2,048 MB for containers). Four levels apply:

• Maximum total number of monitored directories: ⌊mem_usage_limit/100⌋ × 5,000. This does not include directories specified in the Logtail collection configuration blacklist.

• Maximum total number of monitored directories and files: ⌊mem_usage_limit/100⌋ × 50,000. This does not include directories specified in the Logtail collection configuration blacklist, but includes files not matched by any Logtail collection configuration.

• Total number of monitored directories and files for a single collection configuration: ⌊mem_usage_limit/100⌋ × 5,000. This does not include directories specified in the Logtail collection configuration blacklist, but includes files not matched by the Logtail collection configuration.

• Total number of monitored subdirectories and files in a single directory: ⌊mem_usage_limit/100⌋ × 5,000. This includes directories specified in the Logtail collection configuration blacklist and files not matched by any Logtail collection configuration.

When any level is reached, Logtail stops monitoring additional entries. Increase limits by narrowing monitored directories or raising mem_usage_limit. Configure the mem_usage_limit parameter in Set Logtail startup parameters.

On Linux, Logtail uses inotify to reduce collection latency. Maximum inotify-monitored directories (including subdirectories): 3,000.

Resource limit handling policy

If Logtail's resource usage exceeds the maximum limit for 5 minutes, Logtail forcibly restarts. This may cause data loss or duplication.

Multi-tenant data isolation

Logtail provides isolation at the configuration level. An exception in one Logtail collection configuration does not affect others. Related technical articles.

Log collection latency

Under normal conditions, latency from log write to Logtail collection is less than 1 second.

Log upload policy

Logtail aggregates and uploads logs per file. Upload triggers when log count exceeds 4,000, total size exceeds 512 KB, or 3 seconds elapse, whichever comes first.

Limitations

Limitations

Network error handling

On network errors, Logtail retries with automatic interval adjustment. Extreme cases may cause duplicate collection or data loss:

• If the server receives a packet but the client does not receive the response within 15 seconds, the retry causes duplicate collection.

• If 5 consecutive corrupted packets are received, the data is discarded.

Resource quota exceeded

If the send rate exceeds the Logstore quota, Logtail blocks and retries. Increase the number of Logstore shards to resolve. Related technical articles.

Client time exception

If the client-server time difference exceeds 15 minutes, retries fail after 5 attempts and data is discarded. Correct the client machine time.

Project or Logstore does not exist

Data is discarded after 5 failed retries. This can occur if the Logstore was deleted via API. Delete the corresponding Logtail configuration to resolve.

Authentication failed

Data is discarded after 5 failed retries. Common causes:

• Unstable network during Logtail startup prevents authentication.

• The machine cannot establish an HTTPS connection with the SLS endpoint for authentication.

Other unknown errors

After 5 unsuccessful retries, the data is discarded.

Maximum retry time before timeout

If data sending fails continuously for more than 6 hours, the data is discarded.

Status self-check

Logtail automatically restarts on abnormal exits or when resource usage exceeds configured limits.

Exceeding the maximum number of monitored directories and files

Logtail cannot locate collection paths promptly, which may cause data loss.

Significant collection delay

Log collection progress lags behind log generation. If more than 20 unprocessed log files are rotated, data loss occurs.