Grafana+Traefik+Docker+IPv6 Update

Thursday, September 26, 2024 • server monitoring docker

This is a short follow-up post on Prometheus+Grafana+Traefik+Watchtower with IPv6.

With Debian 12 Bookworm, docker compose is now newer than 1.27, in fact, it is 1.29.2. So, we should be able to properly use IPv6 support. It is still not straightforward. While we now can simply say in docker-compose.yml enable_ipv6: true for the network, the docker daemon itself needs to be configured:

Configuring address pools

In order to automatically assign (IPv6) addresses for networks/containers, docker needs to know, which addresses to use. For IPv6, there are no default address pools configured. Therefore, when docker-compose tries to create the network, you’ll get the following error message:

ERROR: could not find an available, non-overlapping IPv6 address pool among the defaults to assign to the network

You’ll need to configure IPv6 pools manually, see Dynamic IPv6 subnet allocation.

1. Create (or modify) /etc/docker/daemon.json $ sudo nano /etc/docker/daemon.json

{
  "default-address-pools": [
    { "base": "172.17.0.0/16", "size": 16 },
    { "base": "172.18.0.0/16", "size": 16 },
    { "base": "172.19.0.0/16", "size": 16 },
    { "base": "172.20.0.0/14", "size": 16 },
    { "base": "172.24.0.0/14", "size": 16 },
    { "base": "172.28.0.0/14", "size": 16 },
    { "base": "192.168.0.0/16", "size": 20 },
    { "base": "fd00:0:0:1::/64", "size": 64 },
    { "base": "fd00:0:0:2::/64", "size": 64 }
  ]
}

This defines the default IPv4 pools and adds two pools for IPv6. This allows for configuring two networks. You might need to add more entries if you need more.

It is using the ULA (Unique local address) space, which is not routed in the internet.

1. Restart docker: $ sudo systemctl restart docker.service

In the docker-compose.yml, you can now define the network as follows:

networks:
  traefik:
    enable_ipv6: true

Volumes

In the old post, I used docker managed volumes (e.g. grafana-data or traefik-data). For making it easier to backup the data, I use now bind mounts for that, but that creates different problems: The permission issue.

It turns out, that the grafana image runs as user “472”, so you need to make sure to set the owner of the data files correctly:

$ mkdir -p data/grafana
$ # copy data from backup into data/grafana
$ chown -R 472 data/grafana

And in docker-compose.yml, you can now use this:

    volumes:
      - ./data/grafana:/var/lib/grafana

Similar for traefik - however, traefik runs as “root” (uid = 0), so it won’t have permission problems. But it refuses to function, if the file acme.json has too broad permissions, so:

$ mkdir -p data/traefik
$ # copy data from backup into data/traefik
$ chmod 600 data/traefik/acme.json

And change the volume config in docker-compose.yaml:

    volumes:
      - ./data/traefik:/letsencrypt/
      - /var/run/docker.sock:/var/run/docker.sock

Also, you can update traefik from v2.9 to v2.10 without any problems.

Verification

After a docker-compose up -d, the network should look like this (e.g.):

$ docker network inspect grafana_traefik | grep Subnet
                    "Subnet": "172.18.0.0/16",
                    "Subnet": "fd00:0:0:1::/64",