Add part one on monitoring with MASH

content/post/monitoring-iot-devices-with-MASH.md

@@ -0,0 +1,362 @@
+---
+title: "Moitoring IoT devices with MASH - Part 1"
+date: 2023-07-06T18:18:10+02:00
+draft: false
+image: "/uploads/monitoring-iot-devices-with-MASH/monitoring.png"
+categrories: ['English']
+tags: ['MASH', 'IoT', 'Monitoring', 'english', 'ansible']
+---
+
+This article is about monitoring some IoT devices (e.g. a CO2 sensor) with a combination of [Mosquitto](https://mosquitto.org/) (a MQTT broker), [Telegraf](https://www.influxdata.com/time-series-platform/telegraf/) (a metric collector), [InfluxDB](https://www.influxdata.com/) (a time-series database) and [Grafana](grafana.com/) (for displaying everything nicely).
+All mentioned services should run on a server that can be reached from the monitoring device(s) and your PC where you want to check the data. We will use MASH (see below) to deploy the services.
+
+In the end this will enable you to get to something like this:
+
+![A screenshot of a grafana dashboard. It shows one panel with temperature and humindity values over an hour, one panel showing CO2 data and one showing energy consumption](/uploads/monitoring-iot-devices-with-MASH/grafana_screenshot1.png)
+
+While writing, I decided to split this into two parts. Part 1 will set up our server infrastructure, Part 2 will show inegrate some real sensors.
+
+Let's get started with Part 1 and answer:
+
+# What is MASH?
+
+MASH is short for "Mother of all self-hosting". It's a collection of [ansible](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/ansible.md) roles, tied together by a playbook that can help deploy and maintain a large number of services. It was inspired by the [matrix-docker-ansible-deploy](https://github.com/spantaleev/matrix-docker-ansible-deploy/) follows the same philosophy and is maintained partly by the same people.
+
+MASH includes services like
+
+* Nextcloud
+* Authentik
+* Gitea
+* Peertube
+* Funkwhale
+
+and more than 50 others at time of writing this blogpost ([List of all supported services](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/supported-services.md)).
+
+The motivation for this playbook is to have all services run in docker, be as configurable as possible, while maintaining easy upgrades and reproducible deployments.
+The documentation IMHO is very good so if people are willing to use ansible this might be a lot easier than writing roles themself.
+
+# Architecture
+
+In the end we will have a IoT device that sends data to a MQTT topic on the MQTT broker Mosquitto. Telegraf will listen to this topic and write this into InfluxDB. When you want to have a look at the data, you can access Grafan which will querey InfluxDB.
+
+# Prerequisits
+
+To have everything it needs to use MASH you should ensure the following things are done and working
+
+* [Prerequisits](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/prerequisites.md)
+* [Getting the playbook](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/getting-the-playbook.md)
+
+
+**Configure DNS**
+
+In thi playbook we will set up some services wich will be reachable from via web interface (Grafana and InfluxDB) while others will not (Telegraf, Mosquitto). We will set a domain name for the server itself, one for Grafana and one for InfluxDB. This allows you to move services more easily. Feel free to adjust the setup to your needs (but don't forget to change tha `_hostname` variables accordingly later).
+
+
+| Service | Domain | Type | Target |
+| --- | --- | --- | --- |
+| Server general | s0.example.com | A | `<IPv4-IP>` |
+| Server general | s0.example.com | AAAA | `<IPv6-IP>` |
+| Grafana | grafana.example.com | CNAME | s0.example.com |
+| InfluxDB | influxdb.example.com | CNAME | s0.example.com |
+
+# Set up the services
+
+Setting up the services will require three steps. The first will set up the general server with mosquitto and influxdb. Then we configure mosquitto and influxdb and use this to set up telegraf and grafana.
+
+## Setting up the basis, Mosquitto and InfluxDB
+
+Setting up will be based on one configuration file of the playbook. `<your-domain>` is the one you set up as Server General (`s0.example.com`).
+
+Execute the following these steps inside the playbook directory:
+
+1. create a directory to hold your configuration (`mkdir -p inventory/host_vars/<your-domain>`)
+
+2. copy the sample configuration file (`cp examples/vars.yml inventory/host_vars/<your-domain>/vars.yml`)
+
+3. copy the sample inventory hosts file (`cp examples/hosts inventory/hosts`)
+
+4. edit the inventory hosts file (`inventory/hosts`) to your liking
+
+Now you are ready to modify the main configuration file at`inventory/host_vars/<your-domain>/vars.yml`.
+
+Add the following to your playbook and replace the default values (`IN_CAPS`)
+
+```yaml
+############
+## Basics ##
+############
+
+
+# Put a strong secret below, generated with `pwgen -s 64 1` or in another way
+# Various other secrets will be derived from this secret automatically.
+mash_playbook_generic_secret_key: ''
+
+mash_playbook_docker_installation_enabled: true
+devture_docker_sdk_for_python_installation_enabled: true
+
+# To ensure the server's clock is synchronized (using systemd-timesyncd/ntpd),
+# we enable the timesync service.
+
+devture_timesync_installation_enabled: true
+
+
+#############
+## traefik ##
+#############
+
+# Traefik will be our revers proxy that makes grafana and influxdb accessible from the outside. It will automatically obtain SSL certificates for us
+
+mash_playbook_reverse_proxy_type: playbook-managed-traefik
+# The E-Mail address that traefik will use to obtain certificates with
+devture_traefik_config_certificatesResolvers_acme_email: certs@example.com
+
+##############
+## influxdb ##
+##############
+influxdb_enabled: true
+influxdb_hostname: influxdb.example.com
+
+influxdb_init: true
+influxdb_init_username: "USERNAME"
+influxdb_init_password: "SECURE-PASSWORD"
+influxdb_init_org: "YOUR-ORG"
+influxdb_init_bucket: "monitoring"
+
+
+#################
+## Mosquitto   ##
+#################
+
+mosquitto_enabled: true
+```
+
+**Installing**
+
+Before installing and each time you update the playbook in the future, you will need to update the Ansible roles in this playbook by running `just roles`. `just roles` is a shortcut to download the latest Ansible roles. If you don't have `just`, you can also manually run the `roles` commands seen in the `justfile` (this gets tedious fast).
+
+To install you should can `just install-all`. That's it.
+
+Congratulations! You installed your first services and can now visit influxdb.example.com and login with the credentials you set above!
+
+
+## Configuring Mosquitto and InfluxDB
+
+**Mosquitto**
+
+To configure mosquitto you only need to set up users. I would recommend to set up seperate users for telegraf and your IoT devices as you might want to restrict the IoT users permissions (not covered here).
+
+Setting up mosquitto users can be done `just run-tags mosquitto-add-user --extra-vars=username=<username> --extra-vars=password=<password>`. For the setting to take effect, you must restart the container. To do that you can use `just start-group mosquitto`.
+
+**InfluxDB**
+
+Log in on influxdb.example.com with the credentials you configured in your `vars.yml`.
+
+We will now create a configuration that will automatically be read by telegraf.
+
+Got to `Load Data -> Telegraf` and press `Create configuration`. Choose the bucket monitoring and search for the MQTT consumer as data source.
+
+Add the following replace the `[[inputs.mqtt_consumer]]` with the following configuration (and make sure that you replace the example values).
+
+```toml
+[[inputs.mqtt_consumer]]
+  servers = ["tcp://s0.example.com:1883"]
+
+  ## Topics that will be subscribed to.
+  topics = [
+    "sensors/#",
+  ]
+  data_format = "value"
+  data_type = "float"
+  username = "USERNAME_SET_WHEN_CONFIGURING_MOSQUITTO"
+  password = "PASSWORD_SET_WHEN_CONFIGURING_MOSQUITTO"
+```
+
+Now got to `Setup Instructions` and copy the `INFLUX_TOKEN`. It will only show once! Also copy the config URL. You will need both in the following step.
+
+
+## Setting up Telegraf and Grafana
+
+You now have everything for the last step: Setting up Telegraf and grafana. We did not do this in the first step as wee needed the access tokes/configuration link which we have now. Therefore you should now add two new sections to your `vars.yml`.
+
+```yaml
+##############
+## telegraf ##
+##############
+telegraf_enabled: true
+telegraf_influx_token: TOKEN-YOU-GOT-FROM-INFLUX
+telegraf_config_link: https://influxdb.example.com/api/v2/telegrafs/0b6d11ca6bb1b000
+
+#############
+## grafana ##
+#############
+
+grafana_enabled: true
+grafana_hostname: grafana.example.com
+grafana_default_admin_user: USERNAME
+grafana_default_admin_password: 'SECURE-PASSWORD'
+
+grafana_provisioning_datasources:
+  - name: InfluxDBs3
+    type: influxdb
+    access: proxy
+    url: "https://{{ influxdb_hostname }}"
+    jsonData:
+      version: Flux
+      organization: YOUR-ORG
+      defaultBucket: monitoring
+    secureJsonData:
+      token: "TOKEN-YOU-GOT-FROM-INFLUXDB"
+
+```
+
+After that, once again do `just install-all`. You should now have a working setup of all services. Now
+
+# Let's put some data in and display it
+
+To send some data that you can display you can use the following python script. Save it as `cli.py`
+
+```python
+import argparse
+import paho.mqtt.client as mqtt
+import random
+import numpy as np
+import time
+
+
+def gen_time():
+    return time.strftime("%H:%M:%S", time.localtime())
+
+
+def gen_temp():
+    """
+    amplitude: amplitude of temperature changes
+    mean: mean of the simulated temperature
+    offset: offset of the signal in seconds
+    period: periond time in seconds
+    """
+    amplitude = 5
+    mean = 20
+    offset = 0
+    period = 900
+    temp = np.sin((time.time() + offset) % (period) * 2 * np.pi) * amplitude + mean
+    return temp
+
+
+def fake(client, data_type="temperature", topic="sensors/temperature"):
+    if data_type == "temperature":
+        data_generator = gen_temp
+    elif data_type == "time":
+        data_generator = gen_time
+
+    while True:
+        print("Publishing")
+        if data_type == "temperature":
+            client.publish(topic, gen_temp())
+        elif data_type == "time":
+            client.publish(topic, gen_time())
+        time.sleep(2)
+
+
+def on_message(client, userdata, msg):
+    print(msg.topic + " " + str(msg.payload))
+
+
+def on_connect(client, userdata, flags, rc):
+    print("Connected with result code " + str(rc))
+    if rc == 5:
+        raise ConnectionError("MQTT server refused connection")
+
+
+def connect_broker(server, port, username="", password=""):
+    client_id = f'python-mqtt-{random.randint(0, 1000)}'
+    client = mqtt.Client(client_id)
+    client.on_connect = on_connect
+    client.username_pw_set(username=username,
+                           password=password)
+    client.connect(server, port, 60)
+
+    return client
+
+
+def cli():
+    parser = argparse.ArgumentParser(description='Do basic MQTT operations')
+    parser.add_argument('action', choices=['sub', 'pub', 'fake'], help="")
+    parser.add_argument('-t', '--topic', help="The MQTT topic")
+    parser.add_argument('--type', help="The type of data to fake")
+    parser.add_argument('-b', '--broker', help="Hostname of the MQTT broker")
+    parser.add_argument('--port', type=int, default=1883, help="The MQTT brokers port (default: 1883)")
+    parser.add_argument('-u', '--user', help="User for the MQTT broker")
+    parser.add_argument('-p', '--password', help="Password of the MQTT broker")
+    parser.add_argument('-d', '--payload', help="The payload to send with a set command")
+
+    args = parser.parse_args()
+
+    client = connect_broker(args.broker, args.port, args.user, args.password)
+    if args.action == "sub":
+        client.subscribe(args.topic)
+        client.on_message = on_message
+        client.loop_forever()
+    elif args.action == "pub":
+        r = client.publish(args.topic, payload=args.payload)
+        print(r)
+    elif args.action == "fake":
+        fake(client, args.topic)
+
+
+if __name__ == "__main__":
+    cli()
+
+```
+
+Start the script with
+
+```sh
+python cli.py fake -t sensors/temperature -b s0.example.com -u USERNAME -p SECURE_PASSWORD`
+```
+
+This will fake a temperature sensor and publish the results to the MQTT topic sensors/temperature. To make sure the MQTT broker works correctly you can subscribe with
+
+```bash
+python cli.py sub -t sensors/temperature -b stats.hyteck.de -u qzt -p fisch-salz-hof
+```
+
+Now let's display this data in Grafana. Add a new dashboard and then use `Add->Visualization`.
+
+![Screenshot of the grafana interface with numbers indicating where to click for the following sentence](/uploads/monitoring-iot-devices-with-MASH/grafana_new_query.jpeg)
+In the panel select InfluxDB as datasource (1+2) put in the folowing query (3).
+
+```flux
+from(bucket: "monitoring")
+  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
+  |> filter(fn: (r) => r["_measurement"] == "mqtt_consumer")
+  |> filter(fn: (r) => r["_field"] == "value")
+  |> filter(fn: (r) => r["topic"] == "sensors/temperature")
+  |> group(columns: ["topic"])
+  |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)
+  |> yield(name: "mean")
+```
+
+add a nice title (4) and apply to see tha data (5). You should now have a nice display of your fake sensor!
+
+# Some last words
+
+Please let me know when you follow this guide - wheter sucessful or not! I would love to improve it.
+
+I know this seems like a lot of effort to get a few numbers to display a few numbers nicely. On the other hands, once set up this setup can doa lot of stuff, monitoring, to alerting. Upgrades should be taken care of by MASH, check out how to do that [here](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/maintenance-upgrading-services.md). Also setting up more services is a breeze. either check the list of existing services, or add a new role to the playbook. If you don't know how, I am happy to help!
+
+So make sure to follow my [RSS feed](https://hyteck.de/index.xml) or any of my socials to get notified when Part 2 comes around. 
+
+# Troubleshooting
+
+When something doesn't work feel free to
+
+* **Leave a comment**
+
+* **Consult the docs**
+
+  + [Mosquitto](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/mosquitto.md)
+  + [Telegraf](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/telegraf.md)
+  + [InfluxDB](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/influxdb.md)
+  + [Grafana](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/grafana.md)
+
+* **[Message me](https://hyteck.de/about/#contact)**