systemd service issue in Alibaba Cloud Linux 2-Alibaba Cloud Linux(Alinux)-阿里云帮助中心

If the systemd service cannot run normally in your Alibaba Cloud Linux 2 operating system, you can troubleshoot the problem based on the solution described in this topic.

Symptoms

Services deployed on an Alibaba Cloud Linux 2 instance with the following configurations may fail to start.

Image: Alibaba Cloud Linux 2.1903 LTS 64-bit public image.
systemd service: systemd-219-78.4.al7 and earlier versions.

This issue causes the systemd service process to run abnormally or generates the following log message:

Failed to activate service 'org.freedesktop.systemd1': timed out

Running the systemctl daemon-reexec command does not fix the issue. The output of the pstack 1 command shows that the systemd service is in the process shown in the following figure.

Cause

The open source systemd service has a vulnerability that can cause heap memory corruption, which in turn causes the systemd service process to crash. After systemd crashes, it enters a freeze state and cannot be restored.

Note

This issue is not resolved in the latest open source version of the systemd service, and the root cause of the heap memory corruption is still under investigation. However, Alibaba Cloud has enhanced the systemd service in new public image versions. If your systemd version is later than systemd-219-78.4.al7 and the issue persists, you can run the kill -15 1 command to restore the systemd service.

Solution

Note

Before you proceed, create a snapshot for the ECS instance to back up your data. If an incorrect operation causes data loss, you can use the snapshot to roll back the cloud disks of the instance. For more information, see Snapshot overview.

The ECS instance with remote connection issues.
For more information, see Choose an ECS remote connection method.
Run the following command to check the systemd version:
```
 rpm -q systemd
```
A sample command output is shown below:
```
 systemd-219-78.4.al7.3.x86_64
```
Perform the following operations to solve the problem based on the systemd version.
- If the systemd version is systemd-219-78.4.al7.3.x86_64 or earlier, run the following commands in order:
  1. Upgrade the systemd service.
```
yum update systemd
```
  2. Restart the ECS instance.
```
reboot
```
- If the systemd version is later than systemd-219-78.4.al7.3.x86_64 and the issue persists, run the following command to restore the systemd service.
```
kill -15 1
```