Difference between revisions of "Samba CI on gitlab/Under the hood"

(How GitLab CI works at a High Level)
(Multiple VMs in parallel)
Line 33: Line 33:
 
==Multiple VMs in parallel==
 
==Multiple VMs in parallel==
  
Each section in the '''.gitlab-ci*.yml''' file is a job, and each job is distributed to an independent VM, allowing execution in parallel.
+
Each section in the '''.gitlab-ci*.yml''' file is a [https://docs.gitlab.com/ee/ci/yaml job], and each job is distributed to an independent VM, allowing execution in parallel.

Revision as of 21:18, 12 February 2019

How GitLab CI works in general

Running remote scripts, displaying the output

Like the Samba build farm of old, GitLab CI is best seen as a system for running scripts on remote hosts against a git checkout.

Pipelines

Samba uses a feature called GitLab Pipelines] to orchestrate our CI.

In-repo configuration

In Samba's case, the remote script is autobuild.py plus some housekeeping before and after. The details is recorded in the .gitlab-ci*.yml files in the Samba tree (so it is maintained with the code).

.gitlab-ci-private.yml vs .gitlab-ci.yml

We have two different CI configurations, one using the default name .gitlab-ci.yml (so picked up by default by forks of our repo) and one that we specify in the Common development repo (.gitlab-ci-private.yml)

The .gitlab-ci-private.yml file includes .gitlab-ci.yml to as to avoid duplication.

The motivation here is to use the shared runners where possible as these are provided by gitlab.com at no cost to Samba Team.

Wrapping docker

To get a consistent build environment docker images are used, so the scripts runs in a docker container. The image used is defined in the .gitlab-ci.yml file.

On a private VM

To allow us to accept and test code from a broader range of contributors, and to enable scaling at times of peak load, the docker container is started in a private VM using Docker Machine]. This applies for both the private and shared (provided by gitlab.com) runners.

Multiple VMs in parallel

Each section in the .gitlab-ci*.yml file is a job, and each job is distributed to an independent VM, allowing execution in parallel.