Use GitLab CI to run GitLab Runner and execute a Pipeline

This topic describes how to install and register GitLab Runner in a Kubernetes cluster. You can add a Kubernetes-type Executor to run builds. This guide walks you through a full continuous integration and continuous delivery (CI/CD) process for a sample Java project. The process covers compiling the source code, packaging an image, and deploying the application.

Background information

This topic uses an example of building a Java project and deploying it to an Alibaba Cloud Container Service for Kubernetes (ACK) cluster. It describes how to use GitLab CI to run GitLab Runner on ACK, configure a Kubernetes-type Executor, and execute a Pipeline.

Create a GitLab source project and upload the sample code

Create a GitLab source project.
The GitLab source project address used in this example is:
```
http://xx.xx.xx.xx/demo/gitlab-java-demo.git
```

Run the following commands to obtain the sample code and upload it to GitLab.

git clone https://github.com/haoshuwei/gitlab-ci-k8s-demo.git
git remote add gitlab http://xx.xx.xx.xx/demo/gitlab-java-demo.git
git push gitlab master

Install GitLab Runner in a Kubernetes cluster

Obtain the registration information for GitLab Runner.
1. Obtain the registration information for a project-specific Runner.
  1. Log on to GitLab.
  2. In the top navigation bar, choose Projects > Your projects.
  3. On the Your projects tab, select the project.
  4. In the navigation pane on the left, choose Settings > CI / CD.
  5. Click Expand to the right of Runners.
  6. Obtain the URL and registration token.
2. Obtain the registration information for Group Runners.
  1. In the top navigation bar, choose Groups > Your groups.
  2. On the Your groups tab, select the group.
  3. In the navigation pane on the left, choose Settings > CI / CD.
  4. Click Expand to the right of Runners.
  5. Obtain the URL and registration token.
3. Obtain the registration information for Shared Runners.
  Note
  Only an administrator can perform this step.
  1. In the top navigation bar, click the icon to go to the Admin Area page.
  2. In the navigation pane on the left, choose Overview > Runners.
  3. Obtain the URL and registration token.

Run the following command to obtain and modify the Helm Chart for GitLab Runner.

git clone https://github.com/haoshuwei/gitlab-runner.git

Replace the gitlabUrl and runnerRegistrationToken fields. The following code provides an example:

## GitLab Runner Image
##
image: gitlab/gitlab-runner:alpine-v11.4.0

## Specify an imagePullPolicy
##
imagePullPolicy: IfNotPresent

## Default container image to use for the init container
init:
  image: busybox
  tag: latest

## The GitLab server URL (with protocol) to register the runner against
##
gitlabUrl: http://xx.xx.xx.xx/

## The registration token for adding new runners to the GitLab server. You must
## retrieve this token from your GitLab instance.
##
runnerRegistrationToken: "AMvEWrBTBu-d8czE****"
## Unregister all runners before termination
##
unregisterRunners: true

## Configure the maximum number of concurrent jobs
##
concurrent: 10

## Defines in seconds how often to check GitLab for new builds
##
checkInterval: 30

## For RBAC support:
##
rbac:
  create: true
  clusterWideAccess: false

## Configure the integrated Prometheus metrics exporter
##
metrics:
  enabled: true

## Configuration for the pods that the runner launches for each new job
##
runners:
  ## Default container image to use for builds when none is specified
  ##
  image: ubuntu:16.04

  ## Specify the tags associated with the runner. This is a comma-separated list of tags.
  ##
  tags: "k8s-runner"

  ## Run all containers with the privileged flag enabled
  ## This allows the docker:dind image to run if Docker commands are needed.
  ## Read the documentation before enabling this setting.
  ##
  privileged: true

  ## The namespace where Kubernetes jobs run. Defaults to the same namespace as this release.
  ##
  namespace: gitlab

  cachePath: "/opt/cache"

  cache: {}
  builds: {}
  services: {}
  helpers: {}

resources: {}

Run the following commands to install GitLab Runner.
- Package the application.
```
helm package .
```
  Expected output:
```
Successfully packaged chart and saved it to: /root/gitlab/gitlab-runner/gitlab-runner-0.1.37.tgz
```
- Install the application.
```
helm install --namespace gitlab gitlab-runner *.tgz
```
Check whether the related deployment and pod started successfully. If they started, the registered GitLab Runner appears on GitLab, as shown in the following figure.

Configure the cache

GitLab Runner has limited support for caching schemes. Therefore, you must mount a volume for caching. In the previous example, the /opt/cache directory is used as the default cache space when you install GitLab Runner. To change the cache directory, modify the runners.cachePath field in the values.yaml file.

For example, to create a Maven cache, add the MAVEN_OPTS variable under variables and specify the local cache directory:

variables:
  KUBECONFIG: /etc/deploy/config
  MAVEN_OPTS: "-Dmaven.repo.local=/opt/cache/.m2/repository"

Modify the following fields in the templates/configmap.yaml file to mount docker.sock and the cache volume.

cat >>/home/gitlab-runner/.gitlab-runner/config.toml <<EOF
            [[runners.kubernetes.volumes.pvc]]
              name = "gitlab-runner-cache"
              mount_path = "{{ .Values.runners.cachePath }}"
            [[runners.kubernetes.volumes.host_path]]
              name = "docker"
              mount_path = "/var/run/docker.sock"
              read_only = true
              host_path = "/var/run/docker.sock"
    EOF

Set global variables

In the top navigation bar, choose Projects > Your projects.
On the Your projects tab, select the project.
In the navigation pane on the left, choose Settings > CI / CD.
Click Expand next to Variables. Add the environment variables for GitLab Runner. In this example, add the following three variables.
- REGISTRY_USERNAME: The username for the image repository.
- REGISTRY_PASSWORD: The password for the image repository.
- kube_config: The encoded string of the KubeConfig file.
Run the following command to generate the encoded string for the KubeConfig file:
```
echo $(cat ~/.kube/config | base64) | tr -d " "
```

Create the .gitlab-ci.yml file

Create the .gitlab-ci.yml file to compile the source code, build and push the image, and deploy the application for the Java demo project. For more information, see the .gitlab-ci.yml.example file in the gitlabci-java-demo source project.

The following code provides an example of a .gitlab-ci.yml file.

image: docker:stable
stages:
  - package
  - docker_build
  - deploy_k8s
variables:
  KUBECONFIG: /etc/deploy/config
  MAVEN_OPTS: "-Dmaven.repo.local=/opt/cache/.m2/repository"
mvn_build_job:
  image: maven:3.6.2-jdk-14
  stage: package
  tags:
    - k8s-runner
  script:
    - mvn package -B -DskipTests
    - cp target/demo.war /opt/cache
docker_build_job:
  image: docker:latest
  stage: docker_build
  tags:
    - k8s-runner
  script:
    - docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD registry.cn-beijing.aliyuncs.com
    - mkdir target
    - cp /opt/cache/demo.war target/demo.war
    - docker build -t registry.cn-beijing.aliyuncs.com/haoshuwei24/gitlabci-java-demo:$CI_PIPELINE_ID .
    - docker push registry.cn-beijing.aliyuncs.com/haoshuwei24/gitlabci-java-demo:$CI_PIPELINE_ID
deploy_k8s_job:
  image: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/kubectl:1.16.6
  stage: deploy_k8s
  tags:
    - k8s-runner
  script:
    - mkdir -p /etc/deploy
    - echo $kube_config |base64 -d > $KUBECONFIG
    - sed -i "s/IMAGE_TAG/$CI_PIPELINE_ID/g" deployment.yaml
    - cat deployment.yaml
    - kubectl apply -f deployment.yaml

The .gitlab-ci.yml file defines a Pipeline that runs in three stages:

image: docker:stable  # The docker:stable image is used by default if a build image is not specified for a stage.
stages:
  - package                # Source code packaging stage
  - docker_build         # Image building, packaging, and pushing stage
  - deploy_k8s           # Application deployment stage
variables:
  KUBECONFIG: /etc/deploy/config   # Defines the KUBECONFIG global variable

Maven source code packaging stage.

mvn_build_job:     # Job name
  image: maven:3.6.2-jdk-14  # The build image for this stage
  stage: package      # The associated stage name
  tags:                     # The GitLab Runner tag
    - k8s-runner
  script:
    - mvn package -B -DskipTests  # Run the build script
    - cp target/demo.war /opt/cache   # Save the build output to the cache

Image building, packaging, and pushing stage.

docker_build_job:  # Job name
  image: docker:latest # The build image for this stage
  stage: docker_build  # The associated stage name
  tags:                      # The GitLab Runner tag
    - k8s-runner
  script:
    - docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD registry.cn-beijing.aliyuncs.com   # Log on to the image repository
    - mkdir target
    - cp /opt/cache/demo.war target/demo.war
    - docker build -t registry.cn-beijing.aliyuncs.com/haoshuwei24/gitlabci-java-demo:$CI_PIPELINE_ID .     # Package the Docker image. The tag is the ID of the current Pipeline.
    - docker push registry.cn-beijing.aliyuncs.com/haoshuwei24/gitlabci-java-demo:$CI_PIPELINE_ID      # Push the Docker image

Application deployment stage.

deploy_k8s_job:   # Job name
  image: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/kubectl:1.16.6   # The build image for this stage
  stage: deploy_k8s   # The associated stage name
  tags:                      # The GitLab Runner tag
    - k8s-runner
  script:
    - mkdir -p /etc/deploy
    - echo $kube_config |base64 -d > $KUBECONFIG   # Configure the config file to connect to the Kubernetes cluster
    - sed -i "s/IMAGE_TAG/$CI_PIPELINE_ID/g" deployment.yaml  # Dynamically replace the image tag in the deployment file
    - cat deployment.yaml
    - kubectl apply -f deployment.yaml

Execute the Pipeline

After you commit the .gitlab-ci.yml file, the gitlab-java-demo project automatically detects the file and executes the Pipeline, as shown in the following figures.

Access the service

If a namespace is not specified in the deployment file, the application is deployed to the gitlab namespace by default:

 kubectl -n gitlab get svc

Expected output:

NAME        TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE
java-demo   LoadBalancer   172.19.9.252   xx.xx.xx.xx   80:32349/TCP   1m

Access xx.xx.xx.xx/demo in a browser to verify the deployment.

For more information about container service, see Container Service.

For more information about GitLab CI, see GitLab CI.