Telegraf, InfluxDB, Grafana with docker-compose

Posted bynicolasPosted inAllgemein, SoftwareTags:, , , , , , ,

Docker is a great and easy way to run software in containers. Docker compose is a nice extension which allows to run whole systems composed of multiple containers in one command. This post shows you how you can set up a so-called “TIG” stack consisting of the open-source tools

  • Telegraf (for data collection),
  • InfluxDB (for storing the data in a time series database)
  • Grafana (for creating nice and beautiful dashboards for visualizing the data)

Step 0: Install Docker

If you haven’t already, setup Docker on your machine. It is available for many operating systems, including Mac, Windows and Linux. You can even install it on a Raspberry Pi. If you want, you can follow my guide for Linux, or just use the official Docker documentation.

Step 1: What we need

The containers for Telegraf, InfluxDB and Grafana all have their own needs regarding configuration file and volume mounts.

Telegraf needs a configuration telling it what data to collect from where, and where to write this data.

InfluxDB also needs a configuration file and a directory (mounted into the Docker container as a volume mount) where it can store its database files.

Grafana also needs some place where it can store its dashboards.

Step 2: Writing the influxdb.conf file

An initial InfluxDB configuration can be generated using this command. This will start a temporary influxdb container using the 2.7.1 version based on Alpine linux (which is the recommended way of setting up slim and lightweight containers in the cloud).

docker run --rm influxdb:2.7.1-alpine influxd print-config > influx-conf.yml

Step 3: Writing the telegraf.conf file

It is possible to generate a sample configuration file using the “telegraf config” command. However, I think that the generated file is too long, so I suggest you check the official documentation and only put the parts into the file that you really need. But, if you need the command, here it is:

docker run --rm telegraf:1.28.2-alpine telegraf config > telegraf.conf

For this tutorial, we assume that we want to collect some JSON data which comes in from an MQTT broker. This will be a Telegraf input plugin called “MQTT Consumer”. This data should be written to InfluxDB later on, so we also need an output plugin. You can see all available configuration options in the official Telegraf documentation.

Detailed configuration options for the MQTT input plugin can also be found in the official GitHub repo of Telegraf.

The configuration file is written in the TOML language.

[[inputs.mqtt_consumer]]
  servers = ["tcp://10.20.30.40:1883"]
  # Topics that will be subscribed to
  topics = [
    "sensors/data/#",
  ]

  # ... more custom config that you need for this plugin

[[outputs.influxdb_v2]]
  urls = ["http://influxdb:8086"]
  token = "INSERT_CREATED_TOKEN_HERE"
  organization = "first-org"
  bucket = "bucket01"

The host name in the url parameter of the output plugin needs to match the name of the InfluxDB service as we will define it in the docker-compose file in the next step.

Step 5: Write the docker-compose.yaml

We want to configure the containers as follows:

InfluxDB: Port 8086 should be exposed, and we need volumes for data (/var/lib/influxdb2) and config (/etc/influxdb2/). Also, the configuration file (influx-conf.yml) is needed. Remember to put it into the local directory ./influxdb/config on your host system, which we will mount into the container.

Grafana: The default port is 3000. We also need the following volumes: /var/lib/grafana/,  /etc/grafana, and /var/log/grafana. The permissions are important, so that Grafana can write to these directories.

Telegraf: Telegraf only needs its configuration mounted. That’s it.

version: "3"
services:
  influxdb:
    image: influxdb:2.7.1-alpine
    container_name: influxdb
    environment:
      - DOCKER_INFLUXDB_INIT_MODE=setup
      - DOCKER_INFLUXDB_INIT_USERNAME=inituser
      - DOCKER_INFLUXDB_INIT_PASSWORD=initpass
      - DOCKER_INFLUXDB_INIT_ORG=first-org
      - DOCKER_INFLUXDB_INIT_BUCKET=bucket01
    volumes:
      # Make sure you create these local directories
      - ./influxdb/data:/var/lib/influxdb2
      - ./influxdb/config:/etc/influxdb2
    ports:
      - 8086:8086
    restart: unless-stopped
    networks:
      - tig_network
    healthcheck:
      test: "curl -f http://localhost:8086/ping"
      interval: 5s
      timeout: 10s
      retries: 5

  telegraf:
    image: telegraf:1.28.2-alpine
    container_name: telegraf
    user: "1000:1000"
    volumes:
      # Make sure you create this local directory
      - ./telegraf/telegraf.conf:/etc/telegraf/telegraf.conf
    restart: unless-stopped
    networks:
      - tig_network
    depends_on:
      # Waits until influxdb container is up
      influxdb:
        condition: service_healthy

  grafana:
    image: grafana/grafana-oss:9.5.12
    container_name: grafana
    ports:
      - 3000:3000
    volumes:
      # Make sure you create these local directories
      - ./grafana/data:/var/lib/grafana
      - ./grafana/config:/etc/grafana
      - ./grafana/log:/var/log/grafana
    restart: unless-stopped
    networks:
      - tig_network

networks:
  tig_network:
    driver: bridge

More details for Grafana docker configuration can be found here.

Step 6: Create the directories and put the config files into place

Create the necessary directories:

mkdir -p influxdb/data \
  influxdb/config \
  telegraf \
  grafana/data \
  grafana/config \
  grafana/log

# Move telegraf.conf into the telegraf dir.
mv telegraf.conf telegraf/

# Move influx-conf.yml to influxdb dir
mv influx-conf.yml influxdb/config/config.yml

# Grafana needs a default empty config file to work
touch grafana/config/grafana.ini

Step 7: Run it without Telegraf once (to configure an API token)

An API token is needed so that Telegraf can connect to InfluxDB. For that, comment out the telegraf service in the docker compose file. Then run 

docker compose up

Now you can access the InfluxDB frontend at http://localhost:8086/ and login with the inituser and initpass provided in the environment variable configuration of the InfluxDB service.

On the left, go to “Load Data” –> “API Tokens”. Click on “Generate API token” –> “Custom API token”. Enter a name for the token, and then choose the “write” permission for the bucket “bucket01”. Click on “Generate”.

Copy the token to the clipboard, and put it into the telegraf.conf file.

Run it!

Now uncomment the telegraf service in the docker-compose.yaml.

To run your full setup, execute this command again:

docker compose up

Now all applications should be up successfully. Done!

Leave a Reply