EN-SMM-APC

SU Development Forum

Author: Rémi DucceschiSupervisor: Francis Klumb

2018-08-06

Gitlab CI

2EN-SMM-APC

Gitlab CI – Continuous Integration

● Automatic build

● Automatic tests

● Automatic deployment– Desktop installers– Mobile APK– Web deployment

What for?

3EN-SMM-APC

Gitlab CI – Continuous Integration

● Run on each push (configurable)

● Responsible notified by e-mail in case of failure

● Well integrated in the Gitlab Web-UI– Logo on branches / PR / commits…– Possibility to prevent merge if tests don’t pass

● Nomenclature:– Job: list of commands executed in a safe environment– Stage: group of jobs that are run in parallel

● Stages are run in sequence– Pipeline: all jobs triggered for one event

● If one job fails, the whole pipeline fails

How does it work?

4EN-SMM-APC

Gitlab CI – Continuous Integration

● Easy:– Write script

● .gitlab-ci.yml

● To go further:– Set up runners

● Linux● Windows

– (default Linux runners exist)

How does it work?

5EN-SMM-APC

Gitlab CI – Writing script

● Documentation: https://docs.gitlab.com/ee/ci/yaml/

● Define your jobs:– Stages– Trigger option– Artifacts– Script (mandatory)

.gitlab-ci.yml

6EN-SMM-APC

Gitlab CI – Writing script

● Documentation: https://docs.gitlab.com/ee/ci/yaml/

● Stages:– By default, jobs are run in parallel

● If sequence needed → define stages● Only and except:

– By default, jobs are run for each push● Artifacts:

– Upload files (Windows installer, DLL...)

.gitlab-ci.yml

7EN-SMM-APC

Gitlab CI – Writing script

● Documentation: https://docs.gitlab.com/ee/ci/yaml/

● Script:– Each line is executed as is by the default runner’s terminal– Use correct script language!

.gitlab-ci.yml

#$*@&!

8EN-SMM-APC

Gitlab CI – Writing script

● Script example:

.gitlab-ci.yml

9EN-SMM-APC

Gitlab CI – RunnersWhat is it?

● Environment where the script is run● Default are empty Linux CC7 Docker containers

– No Windows– Nothing installed

● → Create customized runners– Use image from Docker Hub– Easy for Linux: create Docker image– Complex for Windows: OpenStack VM

● See https://indico.cern.ch/event/689678/

10EN-SMM-APC

Gitlab CI – RunnersLinux

1. Create a Git repo on gitlab– See gitlabci-examples/build_docker_image

2. Add a Dockerfile to generate an image– Inherit from gitlab-registry.cern.ch/ci-tools/ci-worker:cc7

– Configure your image– No CMD or ENTRYPOINT– See https://docs.docker.com/engine/reference/builder/

3. Add a .gitlab-ci.yml4. Use the image in your project

11EN-SMM-APC

Gitlab CI – RunnersLinux

1. Create a Git repo on gitlab

2. Add a Dockerfile to generate an image3. Add a .gitlab-ci.yml

– Add the tag docker-image-build to a job● Automatically build and upload the image

4. Use the image in your project

12EN-SMM-APC

Gitlab CI – RunnersLinux

1. Create a Git repo on gitlab

2. Add a Dockerfile to generate an image3. Add a .gitlab-ci.yml4. Use the image in your project

– Add the image in all jobs, and the tag docker

13EN-SMM-APC

Gitlab CI – RunnersWindows

1. Create VM– See https://indico.cern.ch/event/689678/

2. Install everything needed– Install Git for Windows

● See https://indico.cern.ch/event/697540/● Use Git and optional Unix tools from CMD

14EN-SMM-APC

Gitlab CI – RunnersWindows

3. Install Gitlab Runner– See installation doc– See registration doc

● Gitlab CI coordinator URL: https://gitlab.cern.ch/● Gitlab CI token: in the settings of your Gitlab repo

15EN-SMM-APC

Gitlab CI – RunnersWindows

3. Install Gitlab Runner– See installation doc– See registration doc

● Gitlab CI coordinator URL: https://gitlab.cern.ch/● Gitlab CI token: in the settings of your Gitlab repo● Enter meaningful tags

– windows, x64…● Use shell as executor

– Description and tags can be changed later from Gitlab– File config.toml created

16EN-SMM-APC

Gitlab CI – RunnersWindows

3. Install Gitlab Runner– File config.toml created

17EN-SMM-APC

Gitlab CI – RunnersWindows

4. Add the runner in your Gitlab projects– Already activated in your project (with token)– Need to manually activate them for other projects

18EN-SMM-APC

Gitlab CI – RunnersWindows

5. Update your .gitlab-ci.yml– In your projects– Add correct tags

● NO docker● windows, x64…

19EN-SMM-APC

Gitlab CI – RunnersMaintenance

● Linux– Easy: just rebuild image

● Dockerfile should include a yum update -y● Windows

– Manuall update everything● Software● Libraries● …

20EN-SMM-APC

Gitlab CI – Web interfaceTests

● Jobs are executed on every push– Unless told otherwise

● Live console

21EN-SMM-APC

Gitlab CI – Web interfaceTests

● UI automatically updated to show the status– Commits– Branches– Pull Requests

● Rules to not merge if tests don’t pass…● Artifacts:

– Download a zip– URL to directly access to one file

● See documentation

22EN-SMM-APC

Gitlab CI – Web interfaceBonus: Scheduled pipelines

23EN-SMM-APC

Gitlab CI - Conclusion

● Always run tests– Automatic = free– Try to do all tests in less than 10 minutes

● Bad availibility of Linux runners at CERN…● Build artifacts only on versions

– Master– Tags

24EN-SMM-APC

Questions