Samba CI on gitlab/Under the hood
How GitLab CI works in Samba
Running remote scripts, displaying the output
Like the Samba build farm of old, GitLab CI is best seen as a system for running scripts on remote hosts against a git checkout.
Samba uses a feature called GitLab Pipelines] to orchestrate our CI.
In Samba's case, the remote script is autobuild.py plus some housekeeping before and after. The details is recorded in the .gitlab-ci*.yml files in the Samba tree (so it is maintained with the code).
- See also introduction to setting up GitLab CI
.gitlab-ci-private.yml vs .gitlab-ci.yml
We have two different CI configurations, one using the default name .gitlab-ci.yml (so picked up by default by forks of our repo) and one that we specify in the Common development repo (.gitlab-ci-private.yml)
The .gitlab-ci-private.yml file includes .gitlab-ci.yml to as to avoid duplication.
The motivation here is to use the shared runners where possible as these are provided by gitlab.com at no cost to Samba Team.
To get a consistent build environment docker images are used, so the scripts described above all run inside a docker container.
The image used is defined in the .gitlab-ci.yml file.
GitLab CI is best thought of as a fancy way to run commands in Docker containers and report their results.
A bit like running in a chroot
The way docker is used by GitLab CI is very much akin to downloading a tarball (the image), unpacking it and calling chroot() into it (entering the docker container). Modern container concepts like namespaces etc are used to make it more seamless, but this conceptualisation may assist those struggling with the concepts.
On a private VM
To allow us to accept and test code from a broader range of contributors, and to enable scaling at times of peak load, the docker container is started in a private VM using Docker Machine]. This applies for both the private and shared (provided by gitlab.com) runners.
Multiple VMs in parallel
Each section in the .gitlab-ci*.yml file is a job, and each job is distributed to an independent VM, allowing execution in parallel.
Providing the private VMs
- The Samba team provides the private VMs using credit in the Rackpace cloud.
Ansible management scripts
- The scripts used to configure and operate this service are available.