Add blog post docker ansible django traefik

This commit is contained in:
moanos [he/him] 2023-07-25 09:31:51 +02:00
parent 9feb9df73b
commit b47ff31d43
2 changed files with 157 additions and 0 deletions

View File

@ -0,0 +1,157 @@
---
title: "Deploying a django app with docker, ansible and traefik"
date: 2023-07-24T22:10:10+02:00
draft: false
image: "uploads/docker-ansible-django-traefik/django_docker_ansible_traefik.png"
categrories: ['English']
tags: ['MASH', 'django', 'ilmo', 'ansible', 'traefik', 'docker']
---
This blog post will try to outline the process of deploying [ILMO](https://github.com/moan0s/ILMO2) (a [Django](https://www.djangoproject.com/) app) by building a [docker](https://www.docker.com/) image, using [ansible](https://www.ansible.com/) to install&configure it on our server and use [Traefik](https://traefik.io/) as webserver that is readily configured and obtains certificates for us.
I will go through the steps one by one and link more extensive documentation.
# Building the docker image
Building the docker image is pretty straightforward as it closely resembles the steps of [manual deployment](https://ilmo2.readthedocs.io/en/latest/dev/deployment.html#manual-deployment). The docker file is probably terribly inefficient as it is to large and should be build in stages. Consider this a working example, not a best practice. Also feel free to give me pointers on how to improve it. Specifics I want to point out are:
* static files are collected when building the image
* `pip install -e .` is used to install the python package. Without `-e` the apps static files will not be collected correctly. I haven't figured out why.
* the CMD `ilmo` is executed when starting the container and maps to the script in `docker/ilmo.bash` (see below).
```Dockerfile
FROM python:3-slim
MAINTAINER Julian-Samuel Gebühr
ENV DOCKER_BUILD=true
RUN apt update
RUN apt install gettext -y
ENV VIRTUAL_ENV=/var/ilmo/venv
RUN python -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
COPY src/requirements.txt requirements.txt
RUN pip install -r requirements.txt
WORKDIR /var/ilmo
COPY . .
RUN pip install -e . # Without the -e the library static folder will not be copied by collectstatic!
RUN mkdir /ilmo
RUN mkdir /ilmo/static
RUN ilmo-manage collectstatic --noinput
RUN ilmo-manage compilemessages --ignore venv
COPY docker/ilmo.bash $VIRTUAL_ENV/bin/ilmo
EXPOSE 8345
CMD ["ilmo"]
```
The standard command of the container is a small bash script located at `docker/ilmo.bash` that
* activates the virtual environment
* sets a number of workers based on the available CPU cores
* applies migrations to the database
* executes [gunicorn](https://gunicorn.org/) as [WSGI](https://de.wikipedia.org/wiki/Web_Server_Gateway_Interface) HTTP Server on port 8345
```bash
#!/bin/bash
set -eux
cd /var/ilmo/src
export DATA_DIR=/var/ilmo/
source /var/ilmo/venv/bin/activate
AUTOMIGRATE=${AUTOMIGRATE:-yes}
NUM_WORKERS_DEFAULT=$((2 * $(nproc --all)))
export NUM_WORKERS=${NUM_WORKERS:-$NUM_WORKERS_DEFAULT}
if [ "$AUTOMIGRATE" != "skip" ]; then
ilmo-manage migrate --noinput
fi
exec gunicorn ilmo.wsgi \
--name ilmo \
--workers $NUM_WORKERS \
--max-requests 1200 \
--max-requests-jitter 50 \
--log-level=info \
--bind 0.0.0.0:8345
```
# Using WhiteNoise to serve static files
Django apps usually put their static files in the directory you define in `STATIC_ROOT` after running `python manage.py collectstatic` and expect a webserver like nginx to serve theses files. Now as [discussed before](/post/static-sites-with-mash/) traefik does not easily serve static files. Luckily there is a solution for that: [WhiteNoise](https://whitenoise.readthedocs.io). It allows a django app to serve it's own static files [pretty efficiently](https://whitenoise.readthedocs.io/en/latest/#isn-t-serving-static-files-from-python-horribly-inefficient) while it also takes care of best-practices for you, for instance:
* Serving compressed content (gzip and Brotli formats, handling Accept-Encoding and Vary headers correctly)
* Setting far-future cache headers on content which wont change (useful if working with CDNs).
To get it to work we have to:
* add WhiteNoise to the dependencies (see my [pyproject.toml](https://github.com/moan0s/ILMO2/blob/main/pyproject.toml))
* add the WhiteNoise middleware directly after the SecurityMiddleware
```python
MIDDLEWARE = [
# ...
"django.middleware.security.SecurityMiddleware",
"whitenoise.middleware.WhiteNoiseMiddleware",
# ...
]
```
* define the [storage backend](https://docs.djangoproject.com/en/4.2/ref/settings/#storages) (this is new for django >4.2, for previous version use [`STATICFILES_STORAGE`](https://docs.djangoproject.com/en/4.2/ref/settings/#staticfiles-storage)). This is not strictly necessary but improves performance.
```python
STORAGES = {
"staticfiles": {
"BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage",
},
}
```
When testing if the new configuration works you should test with `DEBUG=False`. Otherwise django will serve static files by itself (which is not safe for production). If you encounter problems check the [Whitenoise Documentation](https://whitenoise.readthedocs.io/en/latest/django.html).
# Traefik as webserver
[Traefik](https://traefik.io/) is a HTTP(S) reverse proxy and load balancer. It is focused on containers and supports dynamic configuration. This means we can spin up a docker container with the `--label /path/to/label_file` flag and traefik will use the configuration in the label file to register a new service and router, obtain SSL certificates and start routing traffic to your application.
For ILMO our traefik configuration adds some sensible response headers, defines an entrypoint (`web-secure` stands for HTTPS via port 443), add a SSL certificate resolver (`default` is here LetsEncrypt) and tells traefik where to send traefik to `traefik.docker.network=traefik` and `traefik.http.services.mash-ilmo.loadbalancer.server.port=8345`. It assumes traefik and the application are both in the docker network called `traefik`.
Everything together looks like this:
```cfg
traefik.docker.network=traefik
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.X-XSS-Protection=1; mode=block
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.X-Frame-Options=SAMEORIGIN
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.X-Content-Type-Options=nosniff
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.Content-Security-Policy=frame-ancestors 'self'
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.Permission-Policy=interest-cohort=()
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.Strict-Transport-Security=max-age=31536000; includeSubDomains
traefik.enable=true
traefik.http.routers.mash-ilmo.rule=Host("ilmo.example.com")
traefik.http.routers.mash-ilmo.middlewares=mash-ilmo-add-response-headers
traefik.http.routers.mash-ilmo.service=mash-ilmo
traefik.http.routers.mash-ilmo.entrypoints=web-secure
traefik.http.routers.mash-ilmo.tls=true
traefik.http.routers.mash-ilmo.tls.certResolver=default
traefik.http.services.mash-ilmo.loadbalancer.server.port=8345
```
# Ansible to deploy
The ansible role will set up everything we did so far on the server. I will not discuss the inner workings of the role in detail as the role is mostly derived from the generic role layout we use in [MASH](https://github.com/mother-of-all-self-hosting) for a large variety of services.
The role features: Install, uninstall and creating the first user. It does so by installing a config and data path, configuring the traefik labels and configuration file, pulling the docker image and finally setting up a systemd service to start the container.
Used together with the [MASH playbook](https://github.com/mother-of-all-self-hosting/mash-playbook) it will also set up a database user and database and install traefik.
The full role can be found at [ansible-role-ilmo](https://github.com/moan0s/ansible-role-ilmo)
# Final thoughts
The process of deploying a django app via docker sure is somewhat complicated. In the end I am still glad to have done it as I think it a) will make deployment more reliable & easier to maintain b) encouraged me to make some design decisions that improved the app itself.
Reach out if you have questions or think this blog post could be improved!

Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB