moanos/Notfellchen
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Add initial documentation

Browse Source
This commit is contained in:
moanos [he/him]
2024-10-03 09:03:28 +02:00
parent cf061e02d2
commit 8798f38f2a
29 changed files with 1193 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

210
docs/dev/backup.rst Normal file
View File

@@ -0,0 +1,210 @@
Backup & Restore
****************
If you do no heavy modification of the code you should be fine with backing up :file:`/etc/notfellchen/` and the database.
Assuming you used a PostgreSQL database the following solution might help you with backups and restores.
Backup
++++++
The following code is a modification of `this script <https://wiki.postgresql.org/wiki/Automated_Backup_on_Linux>`_
licensed under the :ref:`postgresql_license`.
You will first need to create a backup configuration at :file:`/var/notfellchen/pg_backup.config`.
.. code-block::
##############################
## POSTGRESQL BACKUP CONFIG ##
##############################
# Optional system user to run backups as. If the user the script is running as doesn't match this
# the script terminates. Leave blank to skip check.
BACKUP_USER=notfellchen
# Optional hostname to adhere to pg_hba policies. Will default to "localhost" if none specified.
HOSTNAME=localhost
# Optional username to connect to database as. Will default to "postgres" if none specified.
USERNAME=notfellchen
# This dir will be created if it doesn't exist. This must be writable by the user the script is
# running as.
BACKUP_DIR=/var/notfellchen/backups/postgresql
# Enter database to backup
DATABSE=notfellchen
#### SETTINGS FOR ROTATED BACKUPS ####
# Which day to take the weekly backup from (1-7 = Monday-Sunday)
DAY_OF_WEEK_TO_KEEP=7
# Number of days to keep daily backups
DAYS_TO_KEEP=7
# How many weeks to keep weekly backups
WEEKS_TO_KEEP=5
######################################
And then add the script that will do the actual backup at :file:`/var/notfellchen/backup_rotate.sh`
.. code-block:: bash
#!/bin/bash
###########################
####### LOAD CONFIG #######
###########################
while [ $# -gt 0 ]; do
case $1 in
-c)
CONFIG_FILE_PATH="$2"
shift 2
;;
*)
${ECHO} "Unknown Option \"$1\"" 1>&2
exit 2
;;
esac
done
if [ -z $CONFIG_FILE_PATH ] ; then
SCRIPTPATH=$(cd ${0%/*} && pwd -P)
CONFIG_FILE_PATH="${SCRIPTPATH}/pg_backup.config"
fi
if [ ! -r ${CONFIG_FILE_PATH} ] ; then
echo "Could not load config file from ${CONFIG_FILE_PATH}" 1>&2
exit 1
fi
source "${CONFIG_FILE_PATH}"
###########################
#### PRE-BACKUP CHECKS ####
###########################
# Make sure we're running as the required backup user
if [ "$BACKUP_USER" != "" -a "$(id -un)" != "$BACKUP_USER" ] ; then
echo "This script must be run as $BACKUP_USER. Exiting." 1>&2
exit 1
fi
###########################
### INITIALISE DEFAULTS ###
###########################
if [ ! $HOSTNAME ]; then
HOSTNAME="localhost"
fi;
if [ ! $USERNAME ]; then
USERNAME="postgres"
fi;
###########################
#### START THE BACKUPS ####
###########################
function perform_backups()
{
SUFFIX=$1
FINAL_BACKUP_DIR=$BACKUP_DIR"`date +\%Y-\%m-\%d`$SUFFIX/"
echo "Making backup directory in $FINAL_BACKUP_DIR"
if ! mkdir -p $FINAL_BACKUP_DIR; then
echo "Cannot create backup directory in $FINAL_BACKUP_DIR. Go and fix it!" 1>&2
exit 1;
fi;
#######################
### GLOBALS BACKUPS ###
#######################
echo -e "\n\nPerforming backup"
echo -e "--------------------------------------------\n"
echo "Backup"
set -o pipefail
if ! pg_dump $DATABASE | gzip > $FINAL_BACKUP_DIR"$DATABASE".sql.gz.in_progress; then
echo "[!!ERROR!!] Failed to produce globals backup" 1>&2
else
mv $FINAL_BACKUP_DIR"$DATABASE".sql.gz.in_progress $FINAL_BACKUP_DIR"$DATABSE".sql.gz
fi
set +o pipefail
echo -e "\nAll database backups complete!"
}
# MONTHLY BACKUPS
DAY_OF_MONTH=`date +%d`
if [ $DAY_OF_MONTH -eq 1 ];
then
# Delete all expired monthly directories
find $BACKUP_DIR -maxdepth 1 -name "*-monthly" -exec rm -rf '{}' ';'
perform_backups "-monthly"
exit 0;
fi
# WEEKLY BACKUPS
DAY_OF_WEEK=`date +%u` #1-7 (Monday-Sunday)
EXPIRED_DAYS=`expr $((($WEEKS_TO_KEEP * 7) + 1))`
if [ $DAY_OF_WEEK = $DAY_OF_WEEK_TO_KEEP ];
then
# Delete all expired weekly directories
find $BACKUP_DIR -maxdepth 1 -mtime +$EXPIRED_DAYS -name "*-weekly" -exec rm -rf '{}' ';'
perform_backups "-weekly"
exit 0;
fi
# DAILY BACKUPS
# Delete daily backups 7 days old or more
find $BACKUP_DIR -maxdepth 1 -mtime +$DAYS_TO_KEEP -name "*-daily" -exec rm -rf '{}' ';'
perform_backups "-daily"
You should make the script executable test it and automate the execution with :program:`crontab`
.. code-block:: bash
$ chmod +x backup_rotate.sh
$ ./backup_rotate.sh
$ crontab -e
# enter the following to backup every day at 3am
0 3 * * * /var/notfellchen/backup_rotate.sh
Restore
+++++++
If you for any reason want to restore a backup you can use the following:
.. code-block:: bash
$ sudo systemctl stop notfellchen
$ pg_dump notfellchen > notfellchen_YYYY_MM_DD-hh_mm.psql # Make a backup for later analysis
$ dropdb notfellchen
$ cd /path/to/backup
$ gzip -d notfellchen.sql.gz
$ sudo -u postgres createdb -O notfellchen notfellchen
$ psql notfellchen < notfellchen.sql
$ systemctl restart notfellchen

39
docs/dev/contributing.rst Normal file
View File

@@ -0,0 +1,39 @@
Contributing
------------
Report a bug
^^^^^^^^^^^^
To report a bug, file an issue on `Github
<https://codeberg.org/moanos/notfellchen/issues>`_
Try to include the following information:
- The information needed to reproduce the problem
- What you would expect to happen
- What did actually happen
- Error messages
You are also invited to include:
- Screenshots
- Which browser you are using
- The URL of the site
- How urgent it is
- Any additional information you consider useful
Get involved!
^^^^^^^^^^^^^
To contribute simply clone the directory, make your changes and file a
pull request.
If you want to know what can be done, have a look at the current `Github
<https://codeberg.org/moanos/notfellchen/issues>`_.
Get in touch!
^^^^^^^^^^^^^
If you have questions, want to contribute or want to message me regarding something else
you can find contact information at https://hyteck.de/about/ or directly write
an `E-Mail <mailto:info@notfellchen.org>`_

261
docs/dev/deployment.rst Normal file
View File

@@ -0,0 +1,261 @@
.. highlight:: none
**********
Deployment
**********
There are different ways to deploy ILMO. We support an ansible+docker based deployment and manual installation.
Ansible deployment
==================
ILMO can be deployed with the `ilmo-ansible-role <https://github.com/moan0s/ansible-role-ilmo>`_ that is based on the
official ILMO docker image. This role will only install ilmo itself. If you want a complete setup that includes a
database and a webserver with minimal configuration you can use the
`mash-playbook <https://github.com/mother-of-all-self-hosting/mash-playbook>`_ by following `it's documentation
on ILMO <https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/ilmo.md>`_.
Manual Deployment
=================
This guide describes the installation of a installation of ILMO from source. It is inspired by this great guide from
pretix_.
.. warning:: Even though this guide tries to make it as straightforward to run ILMO, it still requires some Linux experience to
get it right. If you're not feeling comfortable managing a Linux server, check out a managed service_.
This guide is tested on **Ubuntu20.04** but it should work very similar on other modern systemd based distributions.
Requirements
------------
Please set up the following systems beforehand, it will not be explained here in detail (but see these links for external
installation guides):
* A SMTP server to send out mails, e.g. `Postfix`_ on your machine or some third-party server you have credentials for
* A HTTP reverse proxy, e.g. `nginx`_ or Traefik to allow HTTPS connections
* A `PostgreSQL`_ database server
Also recommended is, that you use a firewall, although this is not a ILMO-specific recommendation. If you're new to
Linux and firewalls, it is recommended that you start with `ufw`_.
.. note:: Please, do not run ILMO without HTTPS encryption. You'll handle user data and thanks to `Let's Encrypt`_
SSL certificates can be obtained for free these days.
Unix user
---------
As we do not want to run ilmo as root, we first create a new unprivileged user::
# adduser ilmo --disabled-password --home /var/ilmo
In this guide, all code lines prepended with a ``#`` symbol are commands that you need to execute on your server as
``root`` user (e.g. using ``sudo``); all lines prepended with a ``$`` symbol should be run by the unprivileged user.
Database
--------
Having the database server installed, we still need a database and a database user. We can create these with any kind
of database managing tool or directly on our database's shell. Please make sure that UTF8 is used as encoding for the
best compatibility. You can check this with the following command::
# sudo -u postgres psql -c 'SHOW SERVER_ENCODING'
For PostgreSQL database creation, we would do::
# sudo -u postgres createuser ilmo
# sudo -u postgres createdb -O ilmo ilmo
# su ilmo
$ psql
> ALTER USER ilmo PASSWORD 'strong_password';
Package dependencies
--------------------
To build and run ilmo, you will need the following debian packages::
# apt-get install git build-essential python-dev python3-venv python3 python3-pip \
python3-dev
Config file
-----------
We now create a config directory and config file for ilmo::
# mkdir /etc/ilmo
# touch /etc/ilmo/ilmo.cfg
# chown -R ilmo:ilmo /etc/ilmo/
# chmod 0600 /etc/ilmo/ilmo.cfg
Fill the configuration file ``/etc/ilmo/ilmo.cfg`` with the following content (adjusted to your environment)::
[ilmo]
instance_name=My library
url=https://ilmo.example.com
[database]
backend=postgresql
name=ilmo
user=ilmo
[locations]
static=/var/ilmo/static
[mail]
; See config file documentation for more options
; from=ilmo@example.com
; host=127.0.0.1
; user=ilmo
; password=foobar
; port=587
[security]
; See https://securitytxt.org/ for reference
;Contact=
;Expires=
;Encryption=
;Preferred-Languages=
;Scope=
;Policy=
Install ilmo as package
------------------------
Now we will install ilmo itself. The following steps are to be executed as the ``ilmo`` user. Before we
actually install ilmo, we will create a virtual environment to isolate the python packages from your global
python installation::
$ python3 -m venv /var/ilmo/venv
$ source /var/ilmo/venv/bin/activate
(venv)$ pip3 install -U pip setuptools wheel
We now clone and install ilmo, its direct dependencies and gunicorn::
(venv)$ git clone https://github.com/moan0s/ILMO2
(venv)$ cd ILMO2/src/
(venv)$ pip3 install -r requirements.txt
(venv)$ pip3 install -e .
Note that you need Python 3.6 or newer. You can find out your Python version using ``python -V``.
Finally, we compile static files and create the database structure::
(venv)$ ./manage.py collectstatic
(venv)$ ./manage.py migrate
(venv)$ django-admin compilemessages --ignore venv
Start ilmo as a service
-------------------------
You should start ilmo using systemd to automatically start it after a reboot. Create a file
named ``/etc/systemd/system/ilmo-web.service`` with the following content::
[Unit]
Description=ilmo web service
After=network.target
[Service]
User=ilmo
Group=ilmo
Environment="VIRTUAL_ENV=/var/ilmo/venv"
Environment="PATH=/var/ilmo/venv/bin:/usr/local/bin:/usr/bin:/bin"
ExecStart=/var/ilmo/venv/bin/gunicorn ilmo.wsgi \
--name ilmo --workers 5 \
--max-requests 1200 --max-requests-jitter 50 \
--log-level=info --bind=127.0.0.1:8345
WorkingDirectory=/var/ilmo
Restart=on-failure
[Install]
WantedBy=multi-user.target
You can now run the following commands to enable and start the services::
# systemctl daemon-reload
# systemctl enable ilmo-web
# systemctl start ilmo-web
SSL
---
The following snippet is an example on how to configure a nginx proxy for ilmo::
server {
listen 80;
listen [::]:80;
if ($scheme = http) {
return 301 https://$server_name$request_uri;
}
#
listen 443 ssl;
listen [::]:443 ssl;
ssl_certificate /etc/letsencrypt/live/ilmo.example.com/cert.pem;
ssl_certificate_key /etc/letsencrypt/live/ilmo.example.com/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
# Set header
add_header X-Clacks-Overhead "GNU Terry Pratchett";
add_header Permissions-Policy interest-cohort=(); #Anti FLoC
add_header Referrer-Policy same-origin;
add_header X-Content-Type-Options nosniff;
server_name ilmo.example.com;
location / {
proxy_pass http://localhost:8345;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Host $http_host;
}
location /static/ {
alias /var/ilmo/static/;
access_log off;
expires 365d;
add_header Cache-Control "public";
}
}
We recommend reading about setting `strong encryption settings`_ for your web server.
Next steps
----------
Yay, you are done! You should now be able to reach ilmo at https://ilmo.example.com/
Updates
-------
.. warning:: While we try hard not to break things, **please perform a backup before every upgrade**.
To upgrade to a new ilmo release, pull the latest code changes and run the following commands::
$ source /var/ilmo/venv/bin/activate
(venv)$ git pull
(venv)$ pg_dump ilmo > ilmo.psql
(venv)$ python manage.py migrate
(venv)$ django-admin compilemessages --ignore venv
# systemctl restart ilmo-web
.. _Postfix: https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-postfix-as-a-send-only-smtp-server-on-ubuntu-16-04
.. _nginx: https://botleg.com/stories/https-with-lets-encrypt-and-nginx/
.. _Let's Encrypt: https://letsencrypt.org/
.. _MySQL: https://dev.mysql.com/doc/refman/5.7/en/linux-installation-apt-repo.html
.. _PostgreSQL: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-postgresql-on-ubuntu-20-04
.. _redis: https://blog.programster.org/debian-8-install-redis-server/
.. _ufw: https://en.wikipedia.org/wiki/Uncomplicated_Firewall
.. _strong encryption settings: https://mozilla.github.io/server-side-tls/ssl-config-generator/
.. _service: hyteck.de/services
.. _pretix: https://docs.pretix.eu/en/latest/admin/installation/manual_smallscale.html

12
docs/dev/index.rst Normal file
View File

@@ -0,0 +1,12 @@
********************************************
Installation, customization and contributing
********************************************
.. toctree::
:maxdepth: 2
:caption: Contents:
deployment.rst
contributing.rst
release.rst
backup.rst

14
docs/dev/postgresql_license.rst Normal file
View File

@@ -0,0 +1,14 @@
.. _postgresql_license:
PostgreSQL License
******************
.. code-block::
PostgreSQL Database Management System (formerly known as Postgres, then as Postgres95)
Portions Copyright (c) 1996-2008, The PostgreSQL Global Development Group
Portions Copyright (c) 1994, The Regents of the University of California
Permission to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a written agreement is hereby granted, provided that the above copyright notice and this paragraph and the following two paragraphs appear in all copies.
IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.

41
docs/dev/release.rst Normal file
View File

@@ -0,0 +1,41 @@
Release
-------------
What qualifies as release?
^^^^^^^^^^^^^^^^^^^^^^^^^^
A new release should be announced when a significant number functions, bugfixes or other improvements to the software
is made. Usually this indicates a minor release.
Major releases are yet to be determined.
What should be done before a release?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Tested basic functions
######################
Run :command:`pytest`
Test upgrade on a copy of a production database
###############################################
.. WARNING::
You have to prevent e-mails from being sent, otherwise users could receive duplicate e-mails!
* Ensure correct migration if necessary
* Views correct?
Release
^^^^^^^
After testing everything you are good to go. Open the file :file:`src/setup.py` with a text editor
you can adjust the version number:
Do a final commit on this change, and tag the commit as release with appropriate version number.
.. code::
git tag -a v1.0.0 -m "Releasing version v1.0.0"
git push origin v1.0.0
Make sure the tag is visible on Codeberg and celebrate 🥳

18
docs/dev/translation.rst Normal file
View File

@@ -0,0 +1,18 @@
Translation
===========
Translate HTML-files
____________________
First you have to add the text "{% load i18n %}" in every html file at the top.
Write the string in your html file between these two tags: {% translate "String" %}
Translate python-files
______________________
The underscore markes the string for translation. e.g. _("String")
Workflow
_________
- Generate the messages with the command: "django-admin makemessages -l de --ignore venv" de stands in this example for german
- Translate the strings in the file src/local/de/LC_MESSAGES/django.po
- Convert the strings for django with the command: "django-admin compilemessages --ignore venv"