Building the docker image

Building the docker image is pretty straightforward as it closely resembles the steps of manual deployment. The docker file is probably terribly inefficient as it is to large and should be build in stages. Consider this a working example, not a best practice. Also feel free to give me pointers on how to improve it. Specifics I want to point out are:

static files are collected when building the image
pip install -e . is used to install the python package. Without -e the apps static files will not be collected correctly. I haven't figured out why.
the CMD ilmo is executed when starting the container and maps to the script in docker/ilmo.bash (see below).

FROM python:3-slim
MAINTAINER Julian-Samuel Gebühr

ENV DOCKER_BUILD=true

RUN apt update
RUN apt install gettext -y
ENV VIRTUAL_ENV=/var/ilmo/venv
RUN python -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
COPY src/requirements.txt requirements.txt
RUN pip install -r requirements.txt
WORKDIR /var/ilmo
COPY . .
RUN pip install -e .  # Without the -e the library static folder will not be copied by collectstatic!
RUN mkdir /ilmo
RUN mkdir /ilmo/static
RUN ilmo-manage collectstatic --noinput
RUN ilmo-manage compilemessages --ignore venv

COPY docker/ilmo.bash $VIRTUAL_ENV/bin/ilmo

EXPOSE 8345
CMD ["ilmo"]

The standard command of the container is a small bash script located at docker/ilmo.bash that

activates the virtual environment
sets a number of workers based on the available CPU cores
applies migrations to the database
executes gunicorn as WSGI HTTP Server on port 8345

#!/bin/bash

set -eux

cd /var/ilmo/src
export DATA_DIR=/var/ilmo/
source /var/ilmo/venv/bin/activate

AUTOMIGRATE=${AUTOMIGRATE:-yes}
NUM_WORKERS_DEFAULT=$((2 * $(nproc --all)))
export NUM_WORKERS=${NUM_WORKERS:-$NUM_WORKERS_DEFAULT}

if [ "$AUTOMIGRATE" != "skip" ]; then
  ilmo-manage migrate --noinput
fi

exec gunicorn ilmo.wsgi \
    --name ilmo \
    --workers $NUM_WORKERS \
    --max-requests 1200 \
    --max-requests-jitter 50 \
    --log-level=info \
    --bind 0.0.0.0:8345

Using WhiteNoise to serve static files

Django apps usually put their static files in the directory you define in STATIC_ROOT after running python manage.py collectstatic and expect a webserver like nginx to serve theses files. Now as discussed before traefik does not easily serve static files. Luckily there is a solution for that: WhiteNoise. It allows a django app to serve it's own static files pretty efficiently while it also takes care of best-practices for you, for instance:

Serving compressed content (gzip and Brotli formats, handling Accept-Encoding and Vary headers correctly)
Setting far-future cache headers on content which won’t change (useful if working with CDNs).

To get it to work we have to:

add WhiteNoise to the dependencies (see my pyproject.toml)
add the WhiteNoise middleware directly after the SecurityMiddleware

MIDDLEWARE = [
    # ...
    "django.middleware.security.SecurityMiddleware",
    "whitenoise.middleware.WhiteNoiseMiddleware",
    # ...
]

define the storage backend (this is new for django >4.2, for previous version use STATICFILES_STORAGE). This is not strictly necessary but improves performance.

STORAGES = {
    "staticfiles": {
        "BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage",
    },
}

When testing if the new configuration works you should test with DEBUG=False. Otherwise django will serve static files by itself (which is not safe for production). If you encounter problems check the Whitenoise Documentation.

Traefik as webserver

Traefik is a HTTP(S) reverse proxy and load balancer. It is focused on containers and supports dynamic configuration. This means we can spin up a docker container with the --label /path/to/label_file flag and traefik will use the configuration in the label file to register a new service and router, obtain SSL certificates and start routing traffic to your application.

For ILMO our traefik configuration adds some sensible response headers, defines an entrypoint (web-secure stands for HTTPS via port 443), add a SSL certificate resolver (default is here LetsEncrypt) and tells traefik where to send traefik to traefik.docker.network=traefik and traefik.http.services.mash-ilmo.loadbalancer.server.port=8345. It assumes traefik and the application are both in the docker network called traefik.

Everything together looks like this:

traefik.docker.network=traefik

traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.X-XSS-Protection=1; mode=block
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.X-Frame-Options=SAMEORIGIN
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.X-Content-Type-Options=nosniff
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.Content-Security-Policy=frame-ancestors 'self'
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.Permission-Policy=interest-cohort=()
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.Strict-Transport-Security=max-age=31536000; includeSubDomains

traefik.enable=true
traefik.http.routers.mash-ilmo.rule=Host("ilmo.example.com")
traefik.http.routers.mash-ilmo.middlewares=mash-ilmo-add-response-headers
traefik.http.routers.mash-ilmo.service=mash-ilmo
traefik.http.routers.mash-ilmo.entrypoints=web-secure
traefik.http.routers.mash-ilmo.tls=true
traefik.http.routers.mash-ilmo.tls.certResolver=default
traefik.http.services.mash-ilmo.loadbalancer.server.port=8345

Ansible to deploy

The ansible role will set up everything we did so far on the server. I will not discuss the inner workings of the role in detail as the role is mostly derived from the generic role layout we use in MASH for a large variety of services.

The role features: Install, uninstall and creating the first user. It does so by installing a config and data path, configuring the traefik labels and configuration file, pulling the docker image and finally setting up a systemd service to start the container.

Used together with the MASH playbook it will also set up a database user and database and install traefik.

The full role can be found at ansible-role-ilmo

Final thoughts

The process of deploying a django app via docker sure is somewhat complicated. In the end I am still glad to have done it as I think it a) will make deployment more reliable & easier to maintain b) encouraged me to make some design decisions that improved the app itself.

Reach out if you have questions or think this blog post could be improved!

8.2 KiB Raw Blame History Unescape Escape

Building the docker image

Using WhiteNoise to serve static files

Traefik as webserver

Ansible to deploy

Final thoughts

8.2 KiB

Raw Blame History