Add blog post docker ansible django traefik
This commit is contained in:
parent
9feb9df73b
commit
b47ff31d43
157
content/post/deploying-django-with-docker-and-ansible.md
Normal file
157
content/post/deploying-django-with-docker-and-ansible.md
Normal file
@ -0,0 +1,157 @@
|
|||||||
|
---
|
||||||
|
title: "Deploying a django app with docker, ansible and traefik"
|
||||||
|
date: 2023-07-24T22:10:10+02:00
|
||||||
|
draft: false
|
||||||
|
image: "uploads/docker-ansible-django-traefik/django_docker_ansible_traefik.png"
|
||||||
|
categrories: ['English']
|
||||||
|
tags: ['MASH', 'django', 'ilmo', 'ansible', 'traefik', 'docker']
|
||||||
|
---
|
||||||
|
|
||||||
|
This blog post will try to outline the process of deploying [ILMO](https://github.com/moan0s/ILMO2) (a [Django](https://www.djangoproject.com/) app) by building a [docker](https://www.docker.com/) image, using [ansible](https://www.ansible.com/) to install&configure it on our server and use [Traefik](https://traefik.io/) as webserver that is readily configured and obtains certificates for us.
|
||||||
|
|
||||||
|
I will go through the steps one by one and link more extensive documentation.
|
||||||
|
|
||||||
|
# Building the docker image
|
||||||
|
|
||||||
|
Building the docker image is pretty straightforward as it closely resembles the steps of [manual deployment](https://ilmo2.readthedocs.io/en/latest/dev/deployment.html#manual-deployment). The docker file is probably terribly inefficient as it is to large and should be build in stages. Consider this a working example, not a best practice. Also feel free to give me pointers on how to improve it. Specifics I want to point out are:
|
||||||
|
|
||||||
|
* static files are collected when building the image
|
||||||
|
* `pip install -e .` is used to install the python package. Without `-e` the apps static files will not be collected correctly. I haven't figured out why.
|
||||||
|
* the CMD `ilmo` is executed when starting the container and maps to the script in `docker/ilmo.bash` (see below).
|
||||||
|
|
||||||
|
```Dockerfile
|
||||||
|
FROM python:3-slim
|
||||||
|
MAINTAINER Julian-Samuel Gebühr
|
||||||
|
|
||||||
|
ENV DOCKER_BUILD=true
|
||||||
|
|
||||||
|
RUN apt update
|
||||||
|
RUN apt install gettext -y
|
||||||
|
ENV VIRTUAL_ENV=/var/ilmo/venv
|
||||||
|
RUN python -m venv $VIRTUAL_ENV
|
||||||
|
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
|
||||||
|
COPY src/requirements.txt requirements.txt
|
||||||
|
RUN pip install -r requirements.txt
|
||||||
|
WORKDIR /var/ilmo
|
||||||
|
COPY . .
|
||||||
|
RUN pip install -e . # Without the -e the library static folder will not be copied by collectstatic!
|
||||||
|
RUN mkdir /ilmo
|
||||||
|
RUN mkdir /ilmo/static
|
||||||
|
RUN ilmo-manage collectstatic --noinput
|
||||||
|
RUN ilmo-manage compilemessages --ignore venv
|
||||||
|
|
||||||
|
COPY docker/ilmo.bash $VIRTUAL_ENV/bin/ilmo
|
||||||
|
|
||||||
|
EXPOSE 8345
|
||||||
|
CMD ["ilmo"]
|
||||||
|
```
|
||||||
|
|
||||||
|
The standard command of the container is a small bash script located at `docker/ilmo.bash` that
|
||||||
|
|
||||||
|
* activates the virtual environment
|
||||||
|
* sets a number of workers based on the available CPU cores
|
||||||
|
* applies migrations to the database
|
||||||
|
* executes [gunicorn](https://gunicorn.org/) as [WSGI](https://de.wikipedia.org/wiki/Web_Server_Gateway_Interface) HTTP Server on port 8345
|
||||||
|
|
||||||
|
```bash
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
set -eux
|
||||||
|
|
||||||
|
cd /var/ilmo/src
|
||||||
|
export DATA_DIR=/var/ilmo/
|
||||||
|
source /var/ilmo/venv/bin/activate
|
||||||
|
|
||||||
|
AUTOMIGRATE=${AUTOMIGRATE:-yes}
|
||||||
|
NUM_WORKERS_DEFAULT=$((2 * $(nproc --all)))
|
||||||
|
export NUM_WORKERS=${NUM_WORKERS:-$NUM_WORKERS_DEFAULT}
|
||||||
|
|
||||||
|
if [ "$AUTOMIGRATE" != "skip" ]; then
|
||||||
|
ilmo-manage migrate --noinput
|
||||||
|
fi
|
||||||
|
|
||||||
|
exec gunicorn ilmo.wsgi \
|
||||||
|
--name ilmo \
|
||||||
|
--workers $NUM_WORKERS \
|
||||||
|
--max-requests 1200 \
|
||||||
|
--max-requests-jitter 50 \
|
||||||
|
--log-level=info \
|
||||||
|
--bind 0.0.0.0:8345
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
# Using WhiteNoise to serve static files
|
||||||
|
|
||||||
|
Django apps usually put their static files in the directory you define in `STATIC_ROOT` after running `python manage.py collectstatic` and expect a webserver like nginx to serve theses files. Now as [discussed before](/post/static-sites-with-mash/) traefik does not easily serve static files. Luckily there is a solution for that: [WhiteNoise](https://whitenoise.readthedocs.io). It allows a django app to serve it's own static files [pretty efficiently](https://whitenoise.readthedocs.io/en/latest/#isn-t-serving-static-files-from-python-horribly-inefficient) while it also takes care of best-practices for you, for instance:
|
||||||
|
|
||||||
|
* Serving compressed content (gzip and Brotli formats, handling Accept-Encoding and Vary headers correctly)
|
||||||
|
* Setting far-future cache headers on content which won’t change (useful if working with CDNs).
|
||||||
|
|
||||||
|
To get it to work we have to:
|
||||||
|
|
||||||
|
* add WhiteNoise to the dependencies (see my [pyproject.toml](https://github.com/moan0s/ILMO2/blob/main/pyproject.toml))
|
||||||
|
* add the WhiteNoise middleware directly after the SecurityMiddleware
|
||||||
|
```python
|
||||||
|
MIDDLEWARE = [
|
||||||
|
# ...
|
||||||
|
"django.middleware.security.SecurityMiddleware",
|
||||||
|
"whitenoise.middleware.WhiteNoiseMiddleware",
|
||||||
|
# ...
|
||||||
|
]
|
||||||
|
```
|
||||||
|
* define the [storage backend](https://docs.djangoproject.com/en/4.2/ref/settings/#storages) (this is new for django >4.2, for previous version use [`STATICFILES_STORAGE`](https://docs.djangoproject.com/en/4.2/ref/settings/#staticfiles-storage)). This is not strictly necessary but improves performance.
|
||||||
|
```python
|
||||||
|
STORAGES = {
|
||||||
|
"staticfiles": {
|
||||||
|
"BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
When testing if the new configuration works you should test with `DEBUG=False`. Otherwise django will serve static files by itself (which is not safe for production). If you encounter problems check the [Whitenoise Documentation](https://whitenoise.readthedocs.io/en/latest/django.html).
|
||||||
|
|
||||||
|
# Traefik as webserver
|
||||||
|
|
||||||
|
[Traefik](https://traefik.io/) is a HTTP(S) reverse proxy and load balancer. It is focused on containers and supports dynamic configuration. This means we can spin up a docker container with the `--label /path/to/label_file` flag and traefik will use the configuration in the label file to register a new service and router, obtain SSL certificates and start routing traffic to your application.
|
||||||
|
|
||||||
|
For ILMO our traefik configuration adds some sensible response headers, defines an entrypoint (`web-secure` stands for HTTPS via port 443), add a SSL certificate resolver (`default` is here LetsEncrypt) and tells traefik where to send traefik to `traefik.docker.network=traefik` and `traefik.http.services.mash-ilmo.loadbalancer.server.port=8345`. It assumes traefik and the application are both in the docker network called `traefik`.
|
||||||
|
|
||||||
|
Everything together looks like this:
|
||||||
|
|
||||||
|
```cfg
|
||||||
|
traefik.docker.network=traefik
|
||||||
|
|
||||||
|
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.X-XSS-Protection=1; mode=block
|
||||||
|
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.X-Frame-Options=SAMEORIGIN
|
||||||
|
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.X-Content-Type-Options=nosniff
|
||||||
|
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.Content-Security-Policy=frame-ancestors 'self'
|
||||||
|
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.Permission-Policy=interest-cohort=()
|
||||||
|
traefik.http.middlewares.mash-ilmo-add-response-headers.headers.customresponseheaders.Strict-Transport-Security=max-age=31536000; includeSubDomains
|
||||||
|
|
||||||
|
traefik.enable=true
|
||||||
|
traefik.http.routers.mash-ilmo.rule=Host("ilmo.example.com")
|
||||||
|
traefik.http.routers.mash-ilmo.middlewares=mash-ilmo-add-response-headers
|
||||||
|
traefik.http.routers.mash-ilmo.service=mash-ilmo
|
||||||
|
traefik.http.routers.mash-ilmo.entrypoints=web-secure
|
||||||
|
traefik.http.routers.mash-ilmo.tls=true
|
||||||
|
traefik.http.routers.mash-ilmo.tls.certResolver=default
|
||||||
|
traefik.http.services.mash-ilmo.loadbalancer.server.port=8345
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
# Ansible to deploy
|
||||||
|
|
||||||
|
The ansible role will set up everything we did so far on the server. I will not discuss the inner workings of the role in detail as the role is mostly derived from the generic role layout we use in [MASH](https://github.com/mother-of-all-self-hosting) for a large variety of services.
|
||||||
|
|
||||||
|
The role features: Install, uninstall and creating the first user. It does so by installing a config and data path, configuring the traefik labels and configuration file, pulling the docker image and finally setting up a systemd service to start the container.
|
||||||
|
|
||||||
|
Used together with the [MASH playbook](https://github.com/mother-of-all-self-hosting/mash-playbook) it will also set up a database user and database and install traefik.
|
||||||
|
|
||||||
|
The full role can be found at [ansible-role-ilmo](https://github.com/moan0s/ansible-role-ilmo)
|
||||||
|
|
||||||
|
|
||||||
|
# Final thoughts
|
||||||
|
|
||||||
|
The process of deploying a django app via docker sure is somewhat complicated. In the end I am still glad to have done it as I think it a) will make deployment more reliable & easier to maintain b) encouraged me to make some design decisions that improved the app itself.
|
||||||
|
|
||||||
|
Reach out if you have questions or think this blog post could be improved!
|
Binary file not shown.
After Width: | Height: | Size: 56 KiB |
Loading…
Reference in New Issue
Block a user