Docker Swarm Mode and Distributed Traefik proxy with HTTPS

Intro

So, you have a Docker Swarm mode cluster set up as described in DockerSwarm.rocks.

  • Handle connections.
  • Expose specific services and applications based on their domain names.
  • Handle multiple domains (if you need to). Similar to “virtual hosts”.
  • Handle HTTPS.
  • Acquire (generate) HTTPS certificates automatically (including renewals) with Let’s Encrypt.
  • Add HTTP Basic Auth for any service that you need to protect and doesn’t have its own security, etc.
  • Get all its configurations automatically from Docker labels set in your stacks (you don’t need to update configuration files).

Background

Docker Swarm mode with a main Traefik load balancer/proxy is the base cluster architecture that I’m using with my current team for most of the applications and projects.

Overview

This guide will show you how to set up Traefik as a load balancer/proxy and Consul to store configurations and HTTPS certificates.

Redundancy

As Traefik and Consul are both distributed, it doesn’t matter if one of your nodes/machines goes down, the distributed Traefik will be able to handle it from another of the nodes running, preserving HTTPS certificates, etc.

User Interface

The guide includes how to expose the internal Traefik web UI through the same Traefik load balancer, using a secure HTTPS certificate and HTTP Basic Auth.

How it works

The idea is to have a main load balancer/proxy that covers all the Docker Swarm cluster and handles HTTPS certificates and requests for each domain.

About Consul

Consul is a distributed configuration key/value store, it will:

  • Store the Traefik configurations in a distributed manner.
  • Make sure configurations are synchronized among Consul services across the cluster.
  • Store HTTPS certificates for Traefik.

Set up environment variables

  • Connect via SSH to a manager node in your cluster (you might have only one node) that will have the Traefik service.
  • Create a network that will be shared with Traefik and the containers that should be accessible from the outside, with:
docker network create --driver=overlay traefik-public
  • Create an environment variable with your email, to be used for the generation of Let’s Encrypt certificates:
export EMAIL=admin@example.com
  • Create an environment variable with the domain you want to use for the Traefik UI (user interface) and the Consul UI of the host, e.g.:
export DOMAIN=sys.example.com
  • Create an environment variable with a username (you will use it for the HTTP Basic Auth for Traefik and Consul UIs), for example:
export USERNAME=admin
  • Create an environment variable with the password, e.g.:
export PASSWORD=changethis
  • Use openssl to generate the "hashed" version of the password and store it in an environment variable:
export HASHED_PASSWORD=$(openssl passwd -apr1 $PASSWORD)
  • You can check the contents with:
echo $HASHED_PASSWORD
$apr1$89eqM5Ro$CxaFELthUKV21DpI3UTQO.
  • Create an environment variable with the number of replicas for the Consul service (if you don’t set it, by default it will be 3):
export CONSUL_REPLICAS=3
export CONSUL_REPLICAS=0
  • Create an environment variable with the number of replicas for the Traefik service (if you don’t set it, by default it will be 3):
export TRAEFIK_REPLICAS=3
export TRAEFIK_REPLICAS=$(docker node ls -q | wc -l)
export TRAEFIK_REPLICAS=1

Create the Docker Compose file

  • Download the file traefik.yml:
curl -L dockerswarm.rocks/traefik.yml -o traefik.yml
  • …or create it manually, for example, using nano:
nano traefik.yml
  • And copy the contents inside:
version: '3.3'services:
consul-leader:
image: consul
command: agent -server -client=0.0.0.0 -bootstrap -ui
volumes:
- consul-data-leader:/consul/data
environment:
- CONSUL_BIND_INTERFACE=eth0
- 'CONSUL_LOCAL_CONFIG={"leave_on_terminate": true}'
networks:
- default
- traefik-public
deploy:
labels:
- traefik.frontend.rule=Host:consul.${DOMAIN}
- traefik.enable=true
- traefik.port=8500
- traefik.tags=${TRAEFIK_PUBLIC_TAG:-traefik-public}
- traefik.docker.network=traefik-public
# Traefik service that listens to HTTP
- traefik.redirectorservice.frontend.entryPoints=http
- traefik.redirectorservice.frontend.redirect.entryPoint=https
# Traefik service that listens to HTTPS
- traefik.webservice.frontend.entryPoints=https
- traefik.frontend.auth.basic.users=${USERNAME}:${HASHED_PASSWORD}
consul-replica:
image: consul
command: agent -server -client=0.0.0.0 -retry-join="consul-leader"
volumes:
- consul-data-replica:/consul/data
environment:
- CONSUL_BIND_INTERFACE=eth0
- 'CONSUL_LOCAL_CONFIG={"leave_on_terminate": true}'
networks:
- default
- traefik-public
deploy:
replicas: ${CONSUL_REPLICAS:-3}
placement:
preferences:
- spread: node.id
traefik:
image: traefik:v1.7
ports:
- 80:80
- 443:443
deploy:
replicas: ${TRAEFIK_REPLICAS:-3}
placement:
constraints:
- node.role == manager
preferences:
- spread: node.id
labels:
- traefik.frontend.rule=Host:traefik.${DOMAIN}
- traefik.enable=true
- traefik.port=8080
- traefik.tags=traefik-public
- traefik.docker.network=traefik-public
# Traefik service that listens to HTTP
- traefik.redirectorservice.frontend.entryPoints=http
- traefik.redirectorservice.frontend.redirect.entryPoint=https
# Traefik service that listens to HTTPS
- traefik.webservice.frontend.entryPoints=https
- traefik.frontend.auth.basic.users=${USERNAME}:${HASHED_PASSWORD}
volumes:
- /var/run/docker.sock:/var/run/docker.sock
command: >
--docker
--docker.swarmmode
--docker.watch
--docker.exposedbydefault=false
--constraints=tag==traefik-public
--entrypoints='Name:http Address::80'
--entrypoints='Name:https Address::443 TLS'
--consul
--consul.endpoint="consul-leader:8500"
--acme
--acme.email=${EMAIL}
--acme.storage="traefik/acme/account"
--acme.entryPoint=https
--acme.httpChallenge.entryPoint=http
--acme.onhostrule=true
--acme.acmelogging=true
--logLevel=INFO
--accessLog
--api
networks:
- default
- traefik-public
depends_on:
- consul-leader

volumes:
consul-data-leader:
consul-data-replica:
networks:
traefik-public:
external: true

Deploy it

Deploy the stack with:

docker stack deploy -c traefik.yml traefik-consul

Check it

  • Check if the stack was deployed with:
docker stack ps traefik-consul
ID             NAME                              IMAGE           NODE                DESIRED STATE   CURRENT STATE          ERROR   PORT
xvyasdfh56hg traefik-consul_traefik.1 traefik:v1.7 dog.example.com Running Running 1 minute ago
j3ahasdfe0mr traefik-consul_consul-replica.1 consul:latest cat.example.com Running Running 1 minute ago
bfdasdfasr92 traefik-consul_consul-leader.1 consul:latest cat.example.com Running Running 1 minute ago
ofvasdfqtsi6 traefik-consul_traefik.2 traefik:v1.7 snake.example.com Running Running 1 minute ago
tybasdfqdutt traefik-consul_consul-replica.2 consul:latest dog.example.com Running Running 1 minute ago
3ejasdfq2l3g traefik-consul_traefik.3 traefik:v1.7 cat.example.com Running Running 1 minute ago
w0oasdfqsv33 traefik-consul_consul-replica.3 consul:latest snake.example.com Running Running 1 minute ago
  • You can check the Traefik logs with:
docker service logs traefik-consul_traefik

Check the user interfaces

After some seconds/minutes, Traefik will acquire the HTTPS certificates for the web user interfaces.

Updating

Let’s say you add a couple of new nodes to your cluster, and you want to increment the number of Consul replicas or Traefik replicas.

What’s next

The next thing would be to deploy a stack (a complete web application, with backend, frontend, database, etc) using this Docker Swarm mode cluster.

Technical Details

If you want to see the technical details of what each part of the Docker Compose traefik.ymlfile do, check the chapter "Traefik Proxy with HTTPS - Technical Details".

About me

You can follow me, contact me, ask questions, see what I do, or use my open source code:

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sebastián Ramírez

Sebastián Ramírez

1.8K Followers

Creator of FastAPI and Typer. Dev at Exposion AI. APIs, Deep Learning/Machine Learning, full-stack distributed systems, SQL/NoSQL, Python, Docker, JS, TS, etc.