Tuto pour le déploiement des sites avec Docker
How to deploy on docker on BR infra
Note: all this tutorial is particularly tricky, don't hesitate to contact a BRman to get some help.
Requirements
- Basics of git (videos here or the beginning of this awesome online book) ;
- Basics of containers (see a really good explanation here) and of docker-compose (see the getting-started here) ;
- Basics of web (see an introduction on MDN) ;
- Basics of gitlab-ci (reading the getting-started is recommended) ;
Existing architecture
A physical server called endalcher
deploys and runs your container. It uses the following services to connect your service to the Internet:
- docker-compose to deploy the correct environment for your containers ;
- traefik as a reverse proxy to serve your site behind your endpoint, it is configured here ;
- nginx to serve your site to the outside world (as all the binets sites).
Deployment
To deploy a service you need to trigger a pipeline of br/docker-services. This pipeline has the rights to deploy on endalcher
, and will push a docker-compose.yaml
file. The pipeline will have to choose which docker-compose.yaml
file to use, for that matter you will need to correctly set the variable PROJECT
: the ci runner will deploy the file named docker-compose.${PROJECT}.yaml
. You can also pass any parameter needed in the compose file.
In order to create the correct docker-compose.${PROJECT}.yaml
file, you will need to create a Merge Request on br/docker-services, or send a docker-compose.yaml
file to a BRman.
Triggering a pipeline is particularly adapted for CI jobs. For example, you could write a ci job in your repo that triggers the pipeline of br/docker-services.
Example
You want to deploy the QDJ site with docker, and bind it to the server name qdj.binets.fr
. Your site is running on flask and uses a postgresql database.
Pre-existant configuration
You have a little Dockerfile
:
# Base image: python3 FROM python:3 WORKDIR /usr/src/app # Install dependacies COPY requirements.txt ./ RUN pip install --no-cache-dir -r requirements.txt # Copy application files COPY . . EXPOSE 5000/tcp # the command runs the migrations and then launches gunicorn to serve the app CMD python -m flask db upgrade && gunicorn -b :5000 qdj:app
You have the following docker-compose.yaml
:
version: '3.1'
services:
db:
# Use a small postgres image, as we don't need anything on this container
image: postgres:alpine
environment:
POSTGRES_USER: qdj
POSTGRES_PASSWORD: somepassword
POSTGRES_DB: qdj
# store the data on a persistent file
volumes:
- "qdj-data:/var/lib/postgresql/data"
restart: always
app:
# Use the image from Dockerfile
image: ${DOCKER_IMAGE}
environment:
FLASK_APP: qdj.py
FLASK_ENV: production
FLASK_DEBUG: 0
DATABASE_URI: "postgresql://qdj:somepassword@db:5432/qdj"
SECRET_KEY:
# Tells to docker to start app after db (otherwise flask crashes)
depends_on:
- db
# Always restart on crash
restart: always
volumes:
qdj-data:
Production docker-compose.yaml
We will adapt your docker-compose.yaml
file in order to bind your site to the hostname :
- First let's connect your
app
container to the global network. We won't bind thedb
container to the global network to protect it from intruders. Network graph in endalcher We will add network configuration to thedocker-compose.yaml
:to enable the network
web
, we add at the bottom of the file:networks: web: external: true
to connect
app
to the networkweb
, we add to theapp
configuration:networks: - web - default
Then we can add labels to tell
traefik
to match the hostname to a packet going toapp
:labels: - "traefik.docker.network=web" - "traefik.frontend.rule=Host:qdj.binets.fr" - "traefik.port=5000"
This gives us the following docker-compose.qdj.yaml
:
version: '3.1'
services:
db:
image: postgres:alpine
restart: always
environment:
POSTGRES_USER: qdj
POSTGRES_PASSWORD: qdjpw
POSTGRES_DB: qdj
volumes:
- "qdj-data:/var/lib/postgresql/data"
app:
image: ${DOCKER_IMAGE}
environment:
FLASK_APP: qdj.py
FLASK_ENV: production
FLASK_DEBUG: 0
DATABASE_URI: "postgresql://qdj:qdjpw@db:5432/qdj"
SECRET_KEY:
depends_on:
- db
restart: always
networks:
- web
- default
labels:
- "traefik.docker.network=web"
- "traefik.frontend.rule=Host:qdj.binets.fr"
- "traefik.port=5000"
volumes:
qdj-data:
networks:
web:
external: true
This file goes into the br/docker-services repository, it will be used by the triggered pipeline to create your containers.
Triggering pipelines with Gitlab CI
In this pipeline, we will need to add 2 jobs:
- one to build the docker image using the
Dockerfile
- one to trigger the br/docker-services pipeline to create the containers.
To build the docker image, we will use the template do so:
# Build docker image and upload to registry
docker:build:
stage: build
only:
- master
- stable
image: docker:stable
services:
- docker:dind
variables:
DOCKER_HOST: tcp://docker:2375/
DOCKER_DRIVER: overlay2
# For disabling TLS in docker-in-docker (not needed)
DOCKER_TLS_CERTDIR: ''
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build --pull --build-arg version=$CI_PIPELINE_IID -t $IMAGE_TAG -f deployment/Dockerfile .
- docker push $IMAGE_TAG
The second job is rather tricky. We will use curl
to trigger the pipeline, using Gitlab Pipeline API. This call is secured by a trigger token, which should be saved into your project variables as $TRIGGER_TOKEN
. This is what the job looks like:
docker:deploy:
stage: deploy
when: manual
script:
- apt-get update && apt-get install -y curl
- curl --request POST --form token=$TRIGGER_TOKEN --form ref=master --form variables[PROJECT]=qdj --form variables[DOCKER_IMAGE]=$IMAGE_TAG https://gitlab.binets.fr/api/v4/projects/560/trigger/pipeline
Pay attention to this syntax because debugging this api call takes a lot of time...
If you want to add variables that will be used in your docker-compose.yaml
, add other arguments to curl
like --form variables[WELCOME_MESSAGE]="Hello World!"
.
You can see an example of a CI using this in the br/qdj repo.
Note: for BR project, you can use the trigger
syntax in the API call since you don't need autorisation.
History
- In 2017, the BR16 set up a Kubernetes cluster. They managed to set up a workflow to deploy seamlessly from gitlab on that cluster. However, the configuration was hard to maintain and the BR17 was not able to keep it going.
- In 2018, the BR17 (Oliver Facklam + Hadrien Renaud) set up a small docker service on a computer from the Salle Informatiques