Flow Runner-Alibaba Cloud DevOps(Alibaba Cloud DevOps)-阿里云帮助中心

Apsara DevOps Flow Runner is a local service that you install on your build machines and deployment hosts. The Runner uses long polling to communicate with the Apsara DevOps Flow server, pulls build tasks and deployment tasks, and reports runtime logs and task status back to the Flow server.

1. Install a Runner

A Runner is the channel through which Flow runs builds and deployments. The host that runs a Runner must belong to a build cluster or a host group, so create one first:

Build cluster — On the Flow homepage, choose Global Settings > Build Cluster Management, and then click New Build Cluster. A build cluster provides the build environment for Flow tasks.
Host group — On the Flow homepage, choose Global Settings > Host Group Management, and then click New Host Group. A host group runs host deployment tasks in Flow.
You add a host to a private build cluster in much the same way as you add one to a host group. The following sections use a build cluster to describe how to work with a Runner.

1.1 Network connectivity requirements

The Runner uses long polling to communicate with the Apsara DevOps Flow server, so the host that runs the Runner must be able to reach the Flow server. If you install the Runner with the Manual Installation method, the host must be able to reach the following services over the Internet:

https://devops-build-new.aliyuncs.com
https://gray-devops-build.aliyuncs.com
If your host is an Alibaba Cloud ECS instance in Beijing, Hangzhou, Zhangjiakou, Shenzhen, Shanghai, Qingdao, Hohhot, Singapore, Heyuan, Guangzhou, Ulanqab, or Chengdu, and you install the Runner with the Alibaba Cloud ECS method, the Runner connects to the Flow server over an internal endpoint in that region. In this case, the ECS host does not need Internet access.

Note

When you add a host to a build cluster, the host needs Internet access with either the Alibaba Cloud ECS method or the Manual Installation method, because the Runner might pull images from public addresses during a build.

1.2 Host requirements

The Flow Runner supports Linux, Windows, and macOS hosts, with the following requirements:

The Linux operating system must support Systemd, Upstart, or System V.
The Windows operating system must support Windows services.
All of these operating systems support both the amd64 and arm64 chip architectures.

To install a Runner, use the root user on Linux and the Administrator user on Windows. On macOS, you can use any user.

1.3 Install a Runner on a host

On the Flow homepage, choose Global Settings > Build Cluster Management, open the details page of the target build cluster, and then click Add Node to add a build host.

Select the operating system that you need, and then select an installation method:

Manual Installation — Copy the installation command and run it on the host.
Alibaba Cloud ECS:
- Follow the on-screen instructions to create an ECS service connection for the corresponding Alibaba Cloud account.
- Select the ECS host. Apsara DevOps Flow sends the installation command through Cloud Assistant. After the installation finishes, you can view the installation log on the page.
After the host is added, the Host List section of the build cluster details page shows the Instance ID, IP, Hostname, Region, Operating System, Runner Status, and Installation Progress of the new host. You can also click the Log link next to the installation progress to view the full installation log.

On Linux and Windows hosts, you can install a Runner either manually or with the Alibaba Cloud ECS method. On macOS hosts, only manual installation is supported.

On every supported operating system, the Runner is installed as a service:

On Linux, the Runner supports the Systemd, Upstart, and System V service types. If the host supports more than one of them, the Runner selects the first available type in that order. The Runner installation log shows which service type was used:

[INFO] install runner system service
/usr/local/share/yunxiao-runner/v0.1.1/runner install --tenant=be-xxx
INFO[2024-02-28T11:09:17+08:00] service type linux-systemd
INFO[2024-02-28T11:09:17+08:00] Aliyun yunxiao runner-v0.1.1-be-xxx service, linux-systemd installed
[INFO] start runner service
/usr/local/share/yunxiao-runner/v0.1.1/runner start --tenant=be-xxx
INFO[2024-02-28T11:09:17+08:00] service type linux-systemd
INFO[2024-02-28T11:09:17+08:00] Aliyun yunxiao runner-v0.1.1-be-xxx service started
INFO[2024-02-28T11:09:17+08:00] Aliyun yunxiao runner-v0.1.1-be-xxx service status running

On Windows, the Runner runs as a Windows service.
On macOS, the Runner runs through Launchd.

1.4 Runner service naming

Apsara DevOps is a multi-tenant service, so you can install the Runners of different Apsara DevOps organizations on the same host.

Each Apsara DevOps organization has a namespace, which is a random 24-character string, such as aaaaaaaaaaaaaaaaaaaaaaaa. If the installed Runner version is v0.1.0, the service name is runner-v0.1.1-be-aaaaaaaaaaaaaaaaaaaaaaaa. If another organization has the namespace bbbbbbbbbbbbbbbbbbbbbbbb, the host can run both services at the same time:

runner-v0.1.1-be-aaaaaaaaaaaaaaaaaaaaaaaa
runner-v0.1.1-be-bbbbbbbbbbbbbbbbbbbbbbbb

Although you can install different Runner versions of the same organization on one host, do not do this. After you install a Runner, it checks the server for a new version and updates itself automatically, so only one Runner service exists for each organization. You therefore do not need to run the installation command more than once on the same host.

2. Configure a Runner

After you install a Runner, you can view its configuration on the host. In most cases, you do not need to change the default configuration file. How you locate the configuration file and apply your changes depends on the operating system.

2.1 Configuration items

You can set the following Runner configuration items:

scanInterval — The interval, in seconds, at which the Runner sends a heartbeat to the Flow server. In most cases, you do not need to change this value.
concurrency — The maximum number of concurrent tasks allowed.
autoUpgrade — Specifies whether to enable automatic updates. If this setting is enabled, the Runner tries to update itself. For the update rules, see Automatic updates.
upgradeInteval — The interval, in seconds, at which the Runner checks for a Runner version update.

2.2 Modify the configuration file

Assume that the host runs the following two services:

runner-v0.1.1-be-aaaaaaaaaaaaaaaaaaaaaaaa
runner-v0.1.1-be-bbbbbbbbbbbbbbbbbbbbbbbb

Linux

Use the service command of your operating system to list the services:

#Systemd
$ systemctl | grep "runner-v" | awk '{print $1}'
runner-v0.1.1-be-aaaaaaaaaaaaaaaaaaaaaaaa.service
runner-v0.1.1-be-bbbbbbbbbbbbbbbbbbbbbbbb.service

#Upstart
$ initctl list | grep 'runner-v' | awk '{print $1}'
runner-v0.1.1-be-aaaaaaaaaaaaaaaaaaaaaaaa
runner-v0.1.1-be-bbbbbbbbbbbbbbbbbbbbbbbb

#System V
$ service --status-all 2>&1 | grep 'runner-v' | awk '{print $4}'
runner-v0.1.1-be-aaaaaaaaaaaaaaaaaaaaaaaa
runner-v0.1.1-be-bbbbbbbbbbbbbbbbbbbbbbbb

Select the service that you want to inspect, and then parse its service name and organization namespace:

$ SERVICE_NAME=runner-v0.1.1-be-aaaaaaaaaaaaaaaaaaaaaaaa.service
# For System V or Upstart, use SERVICE_NAME=runner-v0.1.1-be-aaaaaaaaaaaaaaaaaaaaaaaa
$ NAMESPACE=`echo $SERVICE_NAME | awk -F '-' '{print $3"-"$4}' | awk -F "." '{print $1}'`
$ echo $NAMESPACE
be-aaaaaaaaaaaaaaaaaaaaaaaa

View the configuration file of the service:

$ vi /root/yunxiao/${NAMESPACE}/runner/config/config.yml

Windows

# List the services.
PS C:\> Get-Service  | Select-Object -Property Name | findstr runner
runner-be-aaaaaaaaaaaaaaaaaaaaaaaa 
runner-be-bbbbbbbbbbbbbbbbbbbbbbbb

# Set SERVICE_NAME and NAMESPACE.
PS C:\> $Env:SERVICE_NAME = "runner-be-bbbbbbbbbbbbbbbbbbbbbbbb"
PS C:\> $Env:NAMESPACE = "be-aaaaaaaaaaaaaaaaaaaaaaaa"

# View the configuration file of the service.
PS C:\> cat C:\yunxiao\data\$Env:NAMESPACE\runner\config\config.yml

macOS

# List the services.
$ launchctl list | grep runner-v | awk '{print $3}'
runner-v0.1.1-be-aaaaaaaaaaaaaaaaaaaaaaaa
runner-v0.1.1-be-bbbbbbbbbbbbbbbbbbbbbbbb

# Set SERVICE_NAME and NAMESPACE.
$ SERVICE_NAME=runner-v0.1.1-be-aaaaaaaaaaaaaaaaaaaaaaaa
$ NAMESPACE=`echo $SERVICE_NAME | awk -F '-' '{print $3"-"$4}'`
$ echo $NAMESPACE
be-bbbbbbbbbbbbbbbbbbbbbbbb

# View the configuration file of the service.
cat ~/.yunxiao-runner/$NAMESPACE/config/config.yml

After you change the configuration file, restart the Runner to apply the change. For instructions, see Start, stop, disable, and enable a Runner.

3. Operate and maintain a Runner

3.1 Start, stop, disable, and enable a Runner

Linux

Set SERVICE_NAME by listing the services on Linux, as described in Modify the configuration file. Then use the commands for your service type.

For Systemd:

# Start
$ systemctl start $SERVICE_NAME
# Stop
$ systemctl stop $SERVICE_NAME
# Restart
$ systemctl restart $SERVICE_NAME
# Enable
$ systemctl enable $SERVICE_NAME
# Disable
$ systemctl disable $SERVICE_NAME

For Upstart:

# Start
$ start $SERVICE_NAME
# Stop
$ stop $SERVICE_NAME
# Restart
$ restart $SERVICE_NAME
# Disable
# Not applicable
# Enable
# Not applicable

For System V:

# Start
$ service $SERVICE_NAME start
# Stop
$ service $SERVICE_NAME stop
# Restart
$ service $SERVICE_NAME stop && service $SERVICE_NAME start
# Disable
$ update-rc.d $SERVICE_NAME defaults
# Enable
$ update-rc.d $SERVICE_NAME remove

Windows

Perform these operations in Services on the Windows operating system:

In the service list, find the Aliyun yunxiao runner-be-{namespace} service, right-click it, and then select the action that you need from the shortcut menu, such as Start, Stop, Pause, Resume, or Restart.

macOS

Set SERVICE_NAME by listing the services on macOS, as described in Modify the configuration file.

# Start
$ launchctl start $SERVICE_NAME
# Stop
# Not applicable. If you no longer need the service, uninstall it.
# Restart
$ launchctl stop $SERVICE_NAME # The service restarts immediately after it stops.
# Disable
# Not applicable
# Enable
# Not applicable

3.2 View Runner logs

Linux

Set NAMESPACE by listing the services on Linux, as described in Modify the configuration file.

$ tail -f /root/yunxiao/$NAMESPACE/runner/__flow_logs/runner.log

Windows

PS C:\> Get-Service  | Select-Object -Property Name | findstr runner
runner-be-aaaaaaaaaaaaaaaaaaaaaaaa
PS C:\> $Env:NAMESPACE_SHORT="aaaa" # The first four characters of the service name after the be- prefix
PS C:\> Get-Content -Tail 10  C:\yunxiao\d\$Env:NAMESPACE_SHORT\runner\__flow_logs\runner.log -Wait

macOS

Set NAMESPACE by listing the services on macOS, as described in Modify the configuration file.

$ tail -f ~/.yunxiao-runner/$NAMESPACE/__flow_logs/runner.log

3.3 Clean up temporary files on a host

A Runner generates the following temporary files while it runs:

The runner.log file described in View Runner logs.
Temporary storage for build logs.
Temporary storage for the local cache.
Images that are pulled for containerized builds.
Temporary layers that image build tasks generate.
The Runner cleans up some of these temporary files automatically, based on the following rules:

A single runner.log file can grow to 500 MB. Beyond 500 MB, the Runner keeps up to 14 archived log files and retains all log files for a maximum of 7 days. Older logs are deleted automatically.
Temporary storage for build logs is retained for 3 days and then deleted automatically.
Clean up the remaining items manually by running the following command. The command performs these operations:

Deletes build caches that have not been updated for 15 days.
Deletes stopped containers created by containerized builds.
Deletes dangling images that have no tag.
Deletes intermediate images generated by custom environment builds.

docker run -v /var/run/docker.sock:/var/run/docker.sock -v /root/yunxiao:/root/yunxiao registry.cn-beijing.aliyuncs.com/build-steps/cache-cleaner:0.0.3-20230914183240

3.4 Automatic updates

A Runner version uses the v<major>.<minor>.<patch> format. By default, a Runner periodically asks the Flow server whether a version update is available. If the minor version number of the latest version on the server differs from the local minor version number, the Runner updates itself automatically.

3.5 Manual updates

Manual updates are not supported yet. To upgrade the patch version of a Runner, uninstall the Runner and then install the latest version.

4. Delete or uninstall a Runner

Linux

Set SERVICE_NAME and NAMESPACE by listing the services on Linux, as described in Modify the configuration file. Then use the commands for your service type.

For Systemd:

$ systemctl stop $SERVICE_NAME
$ systemctl disable $SERVICE_NAME
$ rm /etc/systemd/system/$SERVICE_NAME
$ rm -rf /root/yunxiao/$NAMESPACE
# To delete the Runner services of all tenants on the host, run the preceding commands for each service, and then run the following command.
$ rm -rf /root/yunxiao

For Upstart:

$ initctl stop $SERVICE_NAME
$ mv /etc/init/$SERVICE_NAME.conf /etc/init/$SERVICE_NAME.conf.disabled
$ rm -rf /root/yunxiao/$NAMESPACE
# To delete the Runner services of all tenants on the host, run the preceding commands for each service, and then run the following command.
$ rm -rf /root/yunxiao

For System V:

$ service $SERVICE_NAME stop
$ for i in /etc/rc*.d/S*$SERVICE_NAME; do rm -f $i; done
$ rm -rf /etc/init.d/$SERVICE_NAME
$ rm -rf /root/yunxiao/$NAMESPACE
# To delete the Runner services of all tenants on the host, run the preceding commands for each service, and then run the following command.
$ rm -rf /root/yunxiao

Windows

Set SERVICE_NAME and NAMESPACE by listing the services on Windows, as described in Modify the configuration file. Set NAMESPACE_SHORT as described in View Runner logs.

First, stop the Runner service in the service management console.
Then, run the following commands to delete the corresponding directories.

PS C:\> C:\yunxiao\d\ohce\runner
PS C:\> Remove-Item -Path "C:\yunxiao\d\$Env:NAMESPACE_SHORT\runner" -Recurse -Force
# To delete the Runner services of all tenants on the host, run the preceding commands for each service, and then run the following command.
PS C:\> Remove-Item -Path "C:\yunxiao" -Recurse -Force

macOS

Set SERVICE_NAME and NAMESPACE by listing the services on macOS, as described in Modify the configuration file.

$ sudo launchctl bootout gui/`id -u`/$SERVICE_NAME
$ rm -f ~/Library/LaunchAgents/$SERVICE_NAME.plist
$ rm -rf ~/.yunxiao-runner/$NAMESPACE
# To delete the Runner services of all tenants on the host, run the preceding commands for each service, and then run the following command.
$ rm -rf ~/.yunxiao-runner

5. Other limitations and known issues

5.1 Two hosts have the same config.yml content

Do not copy the config.yml file from one host to another and then start the Runner service. Both Runner services then try to pull tasks, but the target host does not exist in the build cluster or the host group, which can lead to unexpected build behavior.

5.2 Runner installation fails on an ECS instance created from an image

When you create a server from an ECS image, such as a custom image or a shared image, the image also carries the original Runner process and installation directory to the new server. As a result, a conflict or an error occurs when you install the Runner again. Clean up the old Runner as follows, and then install the Runner again:

Check whether a leftover Runner process exists:

ps -ef | grep runner

If a Runner process exists, record its process ID and stop the process:

kill -9 <process ID>

Delete the installation directory and the data directory of the Runner. /usr/local/share/yunxiao-runner/ holds the Runner binary file, and /root/yunxiao/${NAMESPACE}/ holds the Runner data and configuration. NAMESPACE is the namespace of your organization. For information about how to obtain it, see Runner service naming.

rm -rf /usr/local/share/yunxiao-runner/
rm -rf /root/yunxiao/${NAMESPACE}/

After you confirm that no leftover process remains, run the Runner installation command again.