Resolve common collection issues by modifying configurations

Symptoms include log loss or duplication, frequent retries when LoongCollector sends logs, and frequent network disconnections or fluctuations. Possible causes are:

• Insufficient network bandwidth: If the data volume of collected logs is large, insufficient network bandwidth can cause log backlogs and increased latency.

• Unstable network environment: Cross-region or cross-network connections can cause network latency or packet loss.

Solution

Use Alibaba Cloud Content Delivery Network (CDN) edge nodes to accelerate log collection. This method offers significant advantages in network latency and stability compared to collection over the public network. Enabling this feature incurs additional charges based on the actual data volume transferred.

1. Enable the cross-domain transfer acceleration feature for the project.

2. Stop LoongCollector. Log on to the server where LoongCollector is installed and run the sudo /etc/init.d/loongcollectord stop command.

3. Modify the acceleration endpoint. In the /usr/local/ilogtail/ilogtail_config.json file, replace the value of the endpoint_list parameter under data_servers with log-global.aliyuncs.com. This changes the domain name used for data transmission to the acceleration endpoint.

4. Start LoongCollector. Run the sudo /etc/init.d/loongcollectord start command.

If a single log is too long, it is truncated after collection. This happens because the maximum size for a single log entry is 512 KB by default. Simple Log Service truncates any log that exceeds this limit. After multiline logs are split based on the regular expression for the first line, each resulting log entry is still subject to the 512 KB limit. If this limit is exceeded, the log is split into multiple entries for collection.

Solution

1. In the /usr/local/ilogtail/ilogtail_config.json file, modify the value of the max_read_buffer_size parameter. If the parameter does not exist, add it. The maximum value for this parameter is 8 MB.

   Note that memory usage increases linearly. You must reserve an amount of memory equal to max_read_buffer_size × file concurrency.

2. Restart LoongCollector. Run the sudo /etc/init.d/loongcollectord restart command.

Log collection and transmission may appear normal, but some logs are lost. This occurs if the timestamp of a lost log differs from the current Simple Log Service server-side time by more than 15 minutes. For data security, Simple Log Service validates the timestamp of each request, including those from LoongCollector. It rejects requests with timestamps that are more than 15 minutes different from the server-side time. If the local time on the server is changed, for example, to a future time for testing, requests are rejected, and data cannot be written.

Solution

1. In the /usr/local/ilogtail/ilogtail_config.json file, set the value of the enable_log_time_auto_adjust parameter to true. If the parameter does not exist, add it. This setting allows the log time to automatically adjust to the server's local time.

   When this feature is enabled, the offset between the server-side time and the server's local time is added to the log time. Because the offset is updated only when a request is rejected by Simple Log Service, the log time queried in Simple Log Service may not match the actual time the log was written.

2. Restart LoongCollector. Run the sudo /etc/init.d/loongcollectord restart command.

For various reasons, you may need to store logs collected on a server in a new project.

Solution

1. To reuse the collection configuration, save the content of the collection configuration file before proceeding. The uninstallation process deletes this file.

2. Uninstall LoongCollector from the server.

3. Select the appropriate LoongCollector installation method based on your new project and server setup.

4. To transfer historical data to the new project, use one of the following two methods:

   • Download the logs to your local machine and use the Import Historical Log Files feature to transfer the logs to the new project.

   • Use data transformation to replicate data from the old project to the new project.