Docker¶
What is Docker¶
Step 1 - setting up the servers¶
Nowadays the servers are usually preinstalled or an installation process can be kicked off via web interface. For the F4A use case we chose Ubuntu 16.04 LTS (Long term support).
First we should ensure that the system is up-to-date and secure. This is done by kicking off the advanced packaging tool (apt). Within this process we can directly install the docker server component. All steps are done by issuing the following command:
apt-get update && apt-get upgrade -y && apt install -y docker.io
As docker is still being developed, certain functionality still changes. This tutorial has been created using the following docker version (you can find our yours by executing `docker version`
):
root@nxp100:~# docker version
Client:
Version: 1.13.1
API version: 1.26
Go version: go1.6.2
Git commit: 092cba3
Built: Thu Nov 2 20:40:23 2017
OS/Arch: linux/amd64
Server:
Version: 1.13.1
API version: 1.26 (minimum version 1.12)
Go version: go1.6.2
Git commit: 092cba3
Built: Thu Nov 2 20:40:23 2017
OS/Arch: linux/amd64
Experimental: false
root@nxp100:~#
Step 2 - initiate a swarm¶
Setting up the docker swarm. A swarm is a group of computers:
docker swarm init --advertise-addr 89.144.27.100
getting the feedback:
Swarm initialized: current node (r3t3pu7rd74njml1afsf2uoev) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join \
--token <some secret token displayed here> \
89.144.27.100:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
If you don’t remember the token etc. - just run:
docker swarm join-token worker
Step 3 - preparing the domain¶
register any domain you like. Just make sure that the domain names are pointing to all the server IPs you have. With that load balancing / failover is possible:
f4a.me. 86400 IN SOA nsa3.schlundtech.de. mail.tillwitt.de. 2017112808 43200 7200 1209600 86400
f4a.me. 86400 IN NS nsa3.schlundtech.de.
f4a.me. 86400 IN NS nsb3.schlundtech.de.
f4a.me. 86400 IN NS nsc3.schlundtech.de.
f4a.me. 86400 IN NS nsd3.schlundtech.de.
f4a.me. 600 IN MX 10 mail.f4a.me.
*.f4a.me. 600 IN A 89.144.24.15
*.f4a.me. 600 IN A 89.144.27.100
*.f4a.me. 600 IN A 89.144.27.101
*.f4a.me. 600 IN A 89.144.27.102
*.f4a.me. 600 IN A 89.144.27.103
nxp100.f4a.me. 600 IN A 89.144.27.100
nxp101.f4a.me. 600 IN A 89.144.27.101
nxp102.f4a.me. 600 IN A 89.144.27.102
nxp103.f4a.me. 600 IN A 89.144.27.103
nxp104.f4a.me. 600 IN A 89.144.24.15
Docker commands¶
docker run vs docker compose¶
Advantages of docker run are that the command is easy to issue, just a copy & paste to the servers command line. Downside is, that the commands get quite long and adding line breaks introduces another possible fault. If you want to correct a running service you need to remove it first and then reissue it.
Advantages of using a docker-compose.yml is that they are usually easy to edit. Disadvantage is that you have to create them on the server first then issue the command to start them - so one additional step. But the biggest advantage is that they can be re-executed on existing services which will lead to a service update.
Examples¶
starting a generic web application with docker run
docker service create \
--name demo \
--label "traefik.port=80" \
--network traefik-net \
kitematic/hello-world-nginx
Thats all - and the service is running.
To create the same via docker-compose.yml
version: "3"
services:
nginx:
image: kitematic/hello-world-nginx
networks:
- traefik-net
deploy:
labels:
- traefik.port=80
- "traefik.frontend.rule=Host:demo.f4a.me"
networks:
traefik-net:
external: true
Then you need to issue the following command
docker stack deploy --compose-file docker-compose.yml demo
Conclusion¶
To quickly test a service - docker run is nice. But to maintain a production environment docker-compose files are strongly recommended.
Docker registry¶
Running your own registry
docker service create \
--name backoffice \
--network traefik-net \
--label "traefik.port=5000" \
--label 'traefik.frontend.auth.basic=flex4apps:$apr1$G9e4rgPu$jbn2AAk2F.OeGnRVFnIR/1' \
--mount type=bind,src=/swarm/volumes/registry,dst=/var/lib/registry \
registry:2
Pushing to private registry¶
The local image needs to be tagged and then pushed
docker tag phabricator_image registry.f4a.me/phabricator
docker push registry.f4a.me/phabricator
Run that image
docker service create \
--name demo \
--label "traefik.port=80" \
-e "GITURL=https://secret@gogs.tillwitt.de/NXP/homomorphic-encryption-demo.git" \
flex4apps:GQfgCEsjkHC7LRf3Q9PkW4L6onDLtu@backoffice.f4a.me/homomorphic_img
Query the registry¶
Get the overview of all images:
https://registry.f4a.me/v2/_catalog
Get all tags of an image:
https://registry.f4a.me/v2/henc/tags/list
Private repository viewer¶
In the mean time there are good solutions to provide a secure, private and selfhosted docker registry like.
Applications¶
What are applications¶
On the created system applications can be launched. Templates for applications are called images. Running applications are called containers.
Generic demonstrator¶
Simple demo¶
The following application is already working with the current setup.
To launch an easy demonstrator, lets instantiate a webserver and make it available at demo.f4a.net:
docker service create \
--name demo \
--label "traefik.port=80" \
--network traefik-net \
kitematic/hello-world-nginx
Generating a new user with password run:
htpasswd -nbm flex4apps password
or go to: http://www.htaccesstools.com/htpasswd-generator/
The output will be something like:
flex4apps:$apr1$XqnUcSgR$39wlPxxyyxPxXZjFb34wo.
Example for traefik label usage below. If single quotes are in the password they would need to be escaped.
To do that close the quoting before it, insert the escaped single quote, and re-open the quoting: `'first part'\''second part'`
How to start the demo service:
docker service create \
--name demopw \
--label "traefik.port=80" \
--label 'traefik.frontend.auth.basic=myName:$apr1$a7R637Ua$TvXp8/lgky5MDLGLacI1e1' \
--network traefik-net \
kitematic/hello-world-nginx
Docker compose¶
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --compose-file docker-compose.yml demo
# REMOVE STACK
# docker stack rm demo
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3"
services:
nginx:
image: kitematic/hello-world-nginx
networks:
- traefik-net
environment:
- test=noContent
deploy:
labels:
- traefik.port=80
# - "traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW"
# - "traefik.frontend.auth.basic=ich:$$2y$$05$$jTZv0re2cXmiGrzRxW./8Ofse.6g/AEChvbMGdqYKIMqsr8xW/c"
# - "traefik.frontend.auth.basic=user:$$2y$$05$$IRrTxLpG7ICzroI8Pb5P4.p2rMXGqyeeZM857BJxTFzP5q9W4RYuS"
- "traefik.frontend.rule=Host:demo.f4a.me"
networks:
traefik-net:
external: true
grafana¶
Grafana is an open source software for time series analytics
create the service like this:
docker service create \
--name=grafana \
--network traefik-net \
--label "traefik.port=3000" \
--mount type=bind,src=/swarm/volumes/grafana,dst=/var/lib/grafana \
-e "GF_SECURITY_ADMIN_PASSWORD=someSecretPassword" \
grafana/grafana
Below you find the docker-compose for grafana
# Flex4Apps documentation
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --compose-file docker-compose.yml grafana
# REMOVE STACK
# docker stack rm grafana
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3.1"
services:
server:
image: grafana/grafana
volumes:
- /swarm/volumes/grafana:/var/lib/grafana
networks:
- traefik-net
environment:
- "GF_SECURITY_ADMIN_PASSWORD=someSecretPassword"
deploy:
labels:
- traefik.port=3000
- "traefik.frontend.rule=Host:grafana.f4a.me"
replicas: 1
# update_config:
# parallelism: 1
# delay: 30s
# placement:
# constraints: [node.role == manager]
networks:
traefik-net:
external: true
Prometheus¶
Prometheus is an open-source software application used for event monitoring and alerting. It records real-time metrics in a time series database (allowing for high dimensionality) built using a HTTP pull model, with flexible queries and real-time alerting. The project is written in Go and licensed under the Apache 2 License, with source code available on GitHub, and is a graduated project of the Cloud Native Computing Foundation, along with Kubernetes and Envoy.
Below you find the reference code from the repository
# Flex4Apps documentation
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --with-registry-auth --compose-file docker-compose.yml prom
# REMOVE STACK
# docker stack rm prom
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3.1"
services:
server:
image: prom/prometheus
volumes:
- /swarm/volumes/prometheus/data:/prometheus
- ./prometheus.yml:/etc/prometheus/prometheus.yml
networks:
- traefik-net
deploy:
labels:
- traefik.port=9090
- "traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW"
- "traefik.frontend.rule=Host:prometheus.f4a.me"
replicas: 1
update_config:
parallelism: 1
delay: 30s
placement:
constraints: [node.role == manager]
nodeexporter:
image: prom/node-exporter:latest
volumes:
- /proc:/host/proc:ro \
- /sys:/host/sys:ro \
- /:/rootfs:ro \
networks:
- traefik-net
ports:
- 9100:9100
deploy:
mode: global
labels:
- com.group="prom-monitoring"
command:
- '--path.procfs=/host/proc'
- '--path.sysfs=/host/sys'
- --collector.filesystem.ignored-mount-points
- "^/(sys|proc|dev|host|etc|rootfs/var/lib/docker/containers|rootfs/var/lib/docker/overlay2|rootfs/run/docker/netns|rootfs/var/lib/docker/aufs)($$|/)"
networks:
traefik-net:
external: true
portainer¶
start portainer as a service we first need to create a data directory:
mkdir -p /docker/portainer
To start the container itself:
docker service create \
--name "portainer" \
--constraint 'node.role == manager' \
--network "traefik-net" --replicas "1" \
--mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
--mount type=bind,src=/docker/portainer,dst=/data \
--label "traefik.frontend.rule=Host:portainer.f4a.me" \
--label "traefik.backend=tool-portainer" \
--label "traefik.port=9000" \
--label "traefik.docker.network=traefik-net" \
--reserve-memory "20M" --limit-memory "40M" \
--restart-condition "any" --restart-max-attempts "55" \
--update-delay "5s" --update-parallelism "1" \
portainer/portainer \
-H unix:///var/run/docker.sock
Below you find the reference code from the repository for portainer.
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --compose-file docker-compose.yml portainer
# REMOVE STACK
# docker stack rm mariadb
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3"
services:
server:
image: portainer/portainer
networks:
traefik-net:
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /data/local/portainer:/data
environment:
- MYSQL_ROOT_PASSWORD=someSecretPassword
deploy:
labels:
- traefik.port=9000
- "traefik.frontend.rule=Host:portainer.f4a.me"
placement:
constraints: [node.role == manager]
networks:
default:
traefik-net:
external: true
Phabricator¶
The database user needs to be able to create databases
change your password:
sed -i 's/<some secret>/yourPassword/g' Dockerfile
build the image with:
docker build -t phabricator_image .
tag and push it to the registry:
docker tag phabricator_image registry.f4a.me/phabricator docker push registry.f4a.me/phabricator
deploy it from the registry by executing the docker-compose:
docker-compose up
Below you find the reference code from the repository for phabricator. This is not completely working at the end of the project
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --with-registry-auth --compose-file docker-compose.yml phabricator
# REMOVE STACK
# docker stack rm phabricator
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3.1"
services:
web:
image: registry.f4a.me/phabricator
networks:
- traefik-net
# ports:
# - "80:80"
deploy:
labels:
- traefik.port=80
- "traefik.frontend.rule=Host:phabricator.f4a.me"
# db:
# image: mysql:5.7
# volumes:
# - ./data:/var/lib/mysql
# restart: always
# environment:
# MYSQL_ROOT_PASSWORD: root
# MYSQL_DATABASE: phabricator
# MYSQL_USER: phabricator
# MYSQL_PASSWORD: phabricator
networks:
traefik-net:
external: true
MariaDB¶
MariaDB is free and open source relational database system. It was created as a :fork: from MySQL after Oracle started releasing new functionality not as open source anymore and due to the high support cost of MySQL.
As usual make sure that the path for data volume exists:
mkdir -p /swarm/volumes/mariadb
The initiate the docker service:
docker service create \
--name mariadb \
--publish 3306:3306 \
--network traefik-net \
--mount type=bind,src=/swarm/volumes/mariadb,dst=/var/lib/mysq \
--label "traefik.port=3306" \
-e MYSQL_ROOT_PASSWORD=someSecretPassword \
mariadb:latest
Below you find the reference code from the repository for mariadb
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --compose-file docker-compose.yml mariadb
# REMOVE STACK
# docker stack rm mariadb
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3"
services:
server:
image: mariadb:latest
networks:
traefik-net:
volumes:
- /data/shared/mariadb=/var/lib/mysq
environment:
- MYSQL_ROOT_PASSWORD=someSecretPassword
deploy:
labels:
- traefik.port=3306
- "traefik.frontend.rule=Host:mariadb.f4a.me"
phpmyadmin:
image: nazarpc/phpmyadmin
networks:
traefik-net:
environment:
- MYSQL_HOST=mariadb_server:3306
deploy:
labels:
- traefik.port=80
- "traefik.frontend.rule=Host:phpmyadmin.f4a.me"
networks:
default:
traefik-net:
external: true
PhpMyAdmin¶
phpMyAdmin is a free and open source administration tool for MySQL and MariaDB. As a portable web application written primarily in PHP, it has become one of the most popular MySQL administration tools, especially for web hosting services.
The following command will start up PhpMyAdmin:
docker service create \
--name phpmyadmin \
--label "traefik.port=80" \
--network traefik-net \
-e ALLOW_ARBITRARY=1 \
nazarpc/phpmyadmin
Below you find the reference code from the repository for phpmyadmin.
gogs¶
Gogs is a painless self-hosted Git service.
Pull image from Docker Hub.
very strange installation. First need to use –publish 3000:3000 and connect direct for install. Then remove instance and also remove published port. This is certainly something I need to review.
create the data volume for gogs:
mkdir -p /swarm/volumes/gogs
start the service:
docker service create \
--name gogs \
--mount type=bind,src=/swarm/volumes/gogs,dst=/data \
--label "traefik.port=3000" \
--network traefik-net \
gogs/gogs
We now would suggest to rather look into Gitea
Gitea is a community managed lightweight code hosting solution written in Go. It published under the MIT license.
Below you find the docker-compose for gogs
Below you find the docker-compose for gitea
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --compose-file docker-compose.yml gitea
# REMOVE STACK
# docker stack rm gitea
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3"
services:
server:
image: gitea/gitea
networks:
- traefik-net
volumes:
- /swarm/volumes/gitea:/data
deploy:
labels:
- traefik.port=3000
- "traefik.frontend.rule=Host:gitea.f4a.me"
replicas: 1
update_config:
parallelism: 1
delay: 30s
networks:
traefik-net:
external: true
Drone¶
A continuous integration server which is open source, and tightly integrates with open source git platforms like gogs or services like github.
A good installation procedure is available here at http://docs.drone.io/install-for-gogs/. The corresponding commands for F4A are below:
docker run \
--name drone \
--label "traefik.port=8000" \
--publish 8000:8000 \
--publish 9000:9000 \
-e DRONE_OPEN=true \
-e DRONE_HOST=drone.f4a.me \
-e DRONE_GOGS=true \
-e DRONE_GOGS_URL=https://gogs.tillwitt.de \
-e DRONE_SECRET=<some secret> \
drone/drone:0.8
mkdir -p /swarm/volumes/drone
docker service create \
--name drone \
--label "traefik.port=8000" \
--label "traefik.docker.network=traefik-net" \
--network traefik-net \
--mount type=bind,src=/swarm/volumes/drone,dst=/var/lib/drone/ \
--publish 8000:8000 \
--publish 9000:9000 \
-e DRONE_OPEN=true \
-e DRONE_HOST=drone.f4a.me \
-e DRONE_GOGS=true \
-e DRONE_GOGS_URL=https://gogs.tillwitt.de \
-e DRONE_SECRET=<some secret> \
-e DRONE_ADMIN=witt \
drone/drone:0.8
docker service create \
--name drone_agent \
--mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
--network traefik-net \
-e DRONE_SERVER=drone:9000 \
-e DRONE_SECRET=<some secret> \
drone/agent:0.8
Once setup, with in this case gogs, you can log into the web interface. After a short sync all repositories should be visible. Activate drone.io for the corresponding repository.
To tell drone.io what to execute you need to add a .drone.yml
to your repository. Examples are below.
example:
image: dockerfile/nginx
script:
- echo hello world
publish:
docker:
registry: registry.f4a.me
email: witt@f4a.me
repo: registry.f4a.me/flex4apps/flex4apps/homomorphic-encryption
file: homomorphic-encryption/Dockerfile
context: homomorphic-encryption
tag: latest
secrets: [ docker_username, docker_password ]
elasticsearch and kibana¶
elasticsearch is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. Elasticsearch is developed in Java.
Inspired by https://sematext.com/blog/docker-elasticsearch-swarm/. Issue the following command:
docker service create \
--name esc24 \
--label "traefik.port=9200" \
--label 'traefik.frontend.auth.basic=flex4apps:$apr1$G9e4rgPu$jbn2AAk2F.OeGnRVFnIR/1' \
--network traefik-net \
--replicas 3 \
--endpoint-mode dnsrr \
--update-parallelism 1 \
--update-delay 60s \
--mount type=volume,source=esc24,target=/data \
elasticsearch:2.4 \
elasticsearch \
-Des.discovery.zen.ping.multicast.enabled=false \
-Des.discovery.zen.ping.unicast.hosts=esc24 \
-Des.gateway.expected_nodes=3 \
-Des.discovery.zen.minimum_master_nodes=2 \
-Des.gateway.recover_after_nodes=2 \
-Des.network.bind=_eth0:ipv4_
Inspired by https://github.com/elastic/elasticsearch-docker/issues/91 and https://idle.run/elasticsearch-cluster
The host systems have to be prepared to run elasticsearch in a docker:
echo vm.max_map_count=262144 >> /etc/sysctl.conf && sysctl --system && sysctl vm.max_map_count
The issue the following command to start three instances of elasticsearch:
docker service create \
--replicas 3 \
--name esc56 \
--label "traefik.port=9200" \
--label 'traefik.frontend.auth.basic=flex4apps:$apr1$G9e4rgPu$jbn2AAk2F.OeGnRVFnIR/1' \
--mount type=volume,source=esc56,target=/data \
--network traefik-net \
elasticsearch:5.6.4 bash -c 'ip addr && IP=$(ip addr | awk -F"[ /]*" "/inet .*\/24/{print \$3}") && \
echo publish_host=$IP && \
exec /docker-entrypoint.sh -Enetwork.bind_host=0.0.0.0 -Enetwork.publish_host=$IP -Ediscovery.zen.minimum_master_nodes=2 -Ediscovery.zen.ping.unicast.hosts=tasks.esc56'
Below you find the reference code from the repository for the ELK stack version 2.4
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --compose-file docker-compose.yml drone
# REMOVE STACK
# docker stack rm drone
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3"
services:
server:
image: drone/drone:0.8
volumes:
- /swarm/volumes/drone:/var/lib/drone/
environment:
- DRONE_OPEN=true
- DRONE_HOST=drone.f4a.me
- DRONE_GOGS=true
- DRONE_GOGS_URL=https://gogs.tillwitt.de
- DRONE_SECRET=<some secret password>
- DRONE_ADMIN=witt
deploy:
labels:
- traefik.port=8000
- "traefik.frontend.rule=Host:drone.f4a.me"
agent:
image: drone/drone:0.8
command: agent
depends_on: [ server ]
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- DRONE_SERVER=ws://server:8000/ws/broker
- DRONE_SECRET=<some secret password>
networks:
traefik-net:
external: true
Below you find the reference code from the repository for the ELK stack version 5.6
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --compose-file docker-compose.yml elk56
# REMOVE STACK
# docker stack rm drone
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3.0"
services:
elasticsearch:
environment:
ES_JAVA_OPTS: -Xms2g -Xmx2g
image: elasticsearch:5.6
ulimits:
memlock: -1
nofile:
hard: 65536
soft: 65536
nproc: 65538
volumes:
- /usr/share/elasticsearch/data
networks:
- traefik-net
command: >
bash -c 'ip addr && IP=$$(ip addr | awk -F"[ /]*" "/inet .*\/24/{print \$$3}") && \
echo publish_host=$$IP && \
exec /docker-entrypoint.sh -Enetwork.bind_host=0.0.0.0 -Enetwork.publish_host=$$IP -Ediscovery.zen.minimum_master_nodes=3 -Ediscovery.zen.ping.unicast.hosts=elasticsearch'
deploy:
mode: global
labels:
- traefik.port=9200
- "traefik.frontend.rule=Host:esc56.f4a.me"
- "traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW"
- "traefik.frontend.auth.basic="
- traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.
kibana:
image: kibana:5.6
environment:
ELASTICSEARCH_URL: http://elasticsearch:9200
networks:
- traefik-net
deploy:
labels:
- traefik.port=5601
- "traefik.frontend.rule=Host:kb56.f4a.me"
- traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.
networks:
traefik-net:
external: true
Below you find the reference code from the repository for the ELK stack version 6.2
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --compose-file docker-compose.yml elk
# REMOVE STACK
# docker stack rm drone
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3.0"
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.2.2
environment:
- cluster.name=docker-cluster
- bootstrap.memory_lock=false
- http.cors.enabled=true
- http.cors.allow-origin=*
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
# - "discovery.zen.ping.unicast.hosts=elasticsearch"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- /data/local/elasticsearch:/usr/share/elasticsearch/data
networks:
- traefik-net
# command: >
# bash -c 'yum install -y iproute && ip addr && IP=$$(ip addr | awk -F"[ /]*" "/inet .*\/24/{print \$$3}") && \
# echo publish_host=$$IP && \
# exec bin/elasticsearch -Enetwork.bind_host=0.0.0.0 -Enetwork.publish_host=$$IP -Ediscovery.zen.minimum_master_nodes=3 -Ediscovery.zen.ping.unicast.hosts=elasticsearch'
deploy:
labels:
- traefik.port=9200
- "traefik.frontend.rule=Host:elastic.f4a.me"
- "traefik.frontend.headers.customResponseHeaders='Access-Control-Allow-Origin: *'"
- traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.,scai.ndv:$$apr1$$FDUP9rVB$$3.oEyn1LEXDCUhcQ5LQza.,christian.pfeiffer:$$apr1$$7efxqZjC$$RKpeAwtFKXjQEA4MzS/K61,f4a:$$2y$$05$$d01dYZVn8MZnIyu.EsSuB.cPDVBCmSSQOaEpt2P2EZTbAtG594pyG
placement:
constraints: [node.role == manager]
kibana:
image: docker.elastic.co/kibana/kibana:6.2.2
environment:
SERVER_NAME: elasticsearch
ELASTICSEARCH_URL: http://elasticsearch:9200
networks:
- traefik-net
deploy:
labels:
- traefik.port=5601
- "traefik.frontend.rule=Host:kibana.f4a.me"
- traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.,scai.ndv:$$apr1$$FDUP9rVB$$3.oEyn1LEXDCUhcQ5LQza.,christian.pfeiffer:$$apr1$$7efxqZjC$$RKpeAwtFKXjQEA4MzS/K61,f4a:$$2y$$05$$d01dYZVn8MZnIyu.EsSuB.cPDVBCmSSQOaEpt2P2EZTbAtG594pyG
placement:
constraints: [node.role == manager]
connector:
image: ubuntu:17.10
networks:
- traefik-net
tty: true
stdin_open: true
ports:
- 26:22
deploy:
labels:
- traefik.port=80
- "traefik.frontend.rule=Host:connector.f4a.me"
placement:
constraints: [node.role == manager]
networks:
traefik-net:
external: true
kibana¶
Kibana is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on an Elasticsearch cluster. Users can create bar, line and scatter plots, or pie charts and maps on top of large volumes of data.
Issue the following command:
docker service create \
--name kb56 \
--label "traefik.port=5601" \
--label 'traefik.frontend.auth.basic=flex4apps:$apr1$G9e4rgPu$jbn2AAk2F.OeGnRVFnIR/1' \
--network traefik-net \
-e "ELASTICSEARCH_URL=http://esc56:9200" \
kibana:5.6
Below you find the reference code from the repository for the ELK stack version 2.4
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --compose-file docker-compose.yml drone
# REMOVE STACK
# docker stack rm drone
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3"
services:
server:
image: drone/drone:0.8
volumes:
- /swarm/volumes/drone:/var/lib/drone/
environment:
- DRONE_OPEN=true
- DRONE_HOST=drone.f4a.me
- DRONE_GOGS=true
- DRONE_GOGS_URL=https://gogs.tillwitt.de
- DRONE_SECRET=<some secret password>
- DRONE_ADMIN=witt
deploy:
labels:
- traefik.port=8000
- "traefik.frontend.rule=Host:drone.f4a.me"
agent:
image: drone/drone:0.8
command: agent
depends_on: [ server ]
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- DRONE_SERVER=ws://server:8000/ws/broker
- DRONE_SECRET=<some secret password>
networks:
traefik-net:
external: true
Below you find the reference code from the repository for the ELK stack version 5.6
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --compose-file docker-compose.yml elk56
# REMOVE STACK
# docker stack rm drone
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3.0"
services:
elasticsearch:
environment:
ES_JAVA_OPTS: -Xms2g -Xmx2g
image: elasticsearch:5.6
ulimits:
memlock: -1
nofile:
hard: 65536
soft: 65536
nproc: 65538
volumes:
- /usr/share/elasticsearch/data
networks:
- traefik-net
command: >
bash -c 'ip addr && IP=$$(ip addr | awk -F"[ /]*" "/inet .*\/24/{print \$$3}") && \
echo publish_host=$$IP && \
exec /docker-entrypoint.sh -Enetwork.bind_host=0.0.0.0 -Enetwork.publish_host=$$IP -Ediscovery.zen.minimum_master_nodes=3 -Ediscovery.zen.ping.unicast.hosts=elasticsearch'
deploy:
mode: global
labels:
- traefik.port=9200
- "traefik.frontend.rule=Host:esc56.f4a.me"
- "traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW"
- "traefik.frontend.auth.basic="
- traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.
kibana:
image: kibana:5.6
environment:
ELASTICSEARCH_URL: http://elasticsearch:9200
networks:
- traefik-net
deploy:
labels:
- traefik.port=5601
- "traefik.frontend.rule=Host:kb56.f4a.me"
- traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.
networks:
traefik-net:
external: true
Below you find the reference code from the repository for the ELK stack version 6.2
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --compose-file docker-compose.yml elk
# REMOVE STACK
# docker stack rm drone
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3.0"
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.2.2
environment:
- cluster.name=docker-cluster
- bootstrap.memory_lock=false
- http.cors.enabled=true
- http.cors.allow-origin=*
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
# - "discovery.zen.ping.unicast.hosts=elasticsearch"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- /data/local/elasticsearch:/usr/share/elasticsearch/data
networks:
- traefik-net
# command: >
# bash -c 'yum install -y iproute && ip addr && IP=$$(ip addr | awk -F"[ /]*" "/inet .*\/24/{print \$$3}") && \
# echo publish_host=$$IP && \
# exec bin/elasticsearch -Enetwork.bind_host=0.0.0.0 -Enetwork.publish_host=$$IP -Ediscovery.zen.minimum_master_nodes=3 -Ediscovery.zen.ping.unicast.hosts=elasticsearch'
deploy:
labels:
- traefik.port=9200
- "traefik.frontend.rule=Host:elastic.f4a.me"
- "traefik.frontend.headers.customResponseHeaders='Access-Control-Allow-Origin: *'"
- traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.,scai.ndv:$$apr1$$FDUP9rVB$$3.oEyn1LEXDCUhcQ5LQza.,christian.pfeiffer:$$apr1$$7efxqZjC$$RKpeAwtFKXjQEA4MzS/K61,f4a:$$2y$$05$$d01dYZVn8MZnIyu.EsSuB.cPDVBCmSSQOaEpt2P2EZTbAtG594pyG
placement:
constraints: [node.role == manager]
kibana:
image: docker.elastic.co/kibana/kibana:6.2.2
environment:
SERVER_NAME: elasticsearch
ELASTICSEARCH_URL: http://elasticsearch:9200
networks:
- traefik-net
deploy:
labels:
- traefik.port=5601
- "traefik.frontend.rule=Host:kibana.f4a.me"
- traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW,sebastian.schmitz:$$apr1$$cCDTHFrI$$m9QaIkyU8d86N4/r/jljJ/,flex4apps:$$2y$$05$$JvvVJToTDOfTzaNA/oIflecuy5l3pct94/3PfCB/g/xq4qZ49baU.,scai.ndv:$$apr1$$FDUP9rVB$$3.oEyn1LEXDCUhcQ5LQza.,christian.pfeiffer:$$apr1$$7efxqZjC$$RKpeAwtFKXjQEA4MzS/K61,f4a:$$2y$$05$$d01dYZVn8MZnIyu.EsSuB.cPDVBCmSSQOaEpt2P2EZTbAtG594pyG
placement:
constraints: [node.role == manager]
connector:
image: ubuntu:17.10
networks:
- traefik-net
tty: true
stdin_open: true
ports:
- 26:22
deploy:
labels:
- traefik.port=80
- "traefik.frontend.rule=Host:connector.f4a.me"
placement:
constraints: [node.role == manager]
networks:
traefik-net:
external: true
Registry¶
the Registry is a stateless, highly scalable server side application that stores and lets you distribute Docker images. The Registry is open-source, under the permissive Apache license.
Below you find the reference code from the repository
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --compose-file docker-compose.yml registry
# REMOVE STACK
# docker stack rm registry
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3"
services:
registry:
image: registry:latest
networks:
- traefik-net
environment:
- "REGISTRY_HTTP_SECRET=myOwnSecret"
volumes:
- "/data/local/registry:/var/lib/registry"
deploy:
labels:
- traefik.port=5000
- "traefik.frontend.rule=Host:registry.f4a.me"
- "traefik.frontend.auth.basic=witt:$$2y$$05$$kOFY7071ilbnpiJNDaIO9e1WeuhHnKtp9Adrevz4r8wJ3b3X1XuqW"
placement:
constraints: [node.role == manager]
networks:
traefik-net:
external: true
Sheperd¶
Docker swarm service for automatically updating your services whenever their base image is refreshed
Below you find the reference code from the repository
# PULL UPDATE & LAUNCH DOCKER
# git pull && docker stack deploy --compose-file docker-compose.yml shepherd
# REMOVE STACK
# docker stack rm shepherd
# ADD BASIC AUTH and ESCAPE FOR docker-compose usage
# htpasswd -bBn user password | sed 's/\$/\$\$/g' #escape for docker-compose usage
version: "3"
services:
shepherd:
image: mazzolino/shepherd
networks:
- traefik-net
environment:
- SLEEP_TIME="1m"
- BLACKLIST_SERVICES="traefik" #space seperated names
volumes:
- /var/run/docker.sock:/var/run/docker.sock
deploy:
placement:
constraints: [node.role == manager]
networks:
traefik-net:
external: true
Security¶
Let’s encrypt¶
Let’s Encrypt (German “Lasst uns verschlüsseln”) is a certification authority that went into operation at the end of 2015 and offers free X.509 certificates for Transport Layer Security (TLS). An automated process replaces the previously common complex manual procedures for the creation, validation, signing, setup and renewal of certificates for encrypted websites.
There are certain rate limits which apply and you might run into when experimenting with the swarm.
https://letsencrypt.org/docs/rate-limits/
To search for issued certificates you can look go to:
Docker swarm networks¶
Overlay networking for Docker Engine swarm mode comes secure out of the box. The swarm nodes exchange overlay network information using a gossip protocol. By default the nodes encrypt and authenticate information they exchange via gossip using the AES algorithm in GCM mode. Manager nodes in the swarm rotate the key used to encrypt gossip data every 12 hours. [*]
[*] | https://docs.docker.com/v17.09/engine/userguide/networking/overlay-security-model/ |
Docker secrets¶
in terms of Docker Swarm services, a secret is a blob of data, such as a password, SSH private key, SSL certificate, or another piece of data that should not be transmitted over a network or stored unencrypted in a Dockerfile or in your application’s source code. In Docker 1.13 and higher, you can use Docker secrets to centrally manage this data and securely transmit it to only those containers that need access to it. Secrets are encrypted during transit and at rest in a Docker swarm. A given secret is only accessible to those services which have been granted explicit access to it, and only while those service tasks are running.
You can use secrets to manage any sensitive data which a container needs at runtime but you don’t want to store in the image or in source control, such as:
Usernames and passwords SSH keys Other important data such as the name of a database or internal server Generic strings or binary content (up to 500 kb in size) [†]
[†] | https://docs.docker.com/engine/swarm/secrets/ |
Docker notary¶
Notary is a tool for publishing and managing trusted collections of content. Publishers can digitally sign collections and consumers can verify integrity and origin of content. This ability is built on a straightforward key management and signing interface to create signed collections and configure trusted publishers.
With Notary anyone can provide trust over arbitrary collections of data. Using The Update Framework (TUF) as the underlying security framework, Notary takes care of the operations necessary to create, manage, and distribute the metadata necessary to ensure the integrity and freshness of your content. [‡]
[‡] | https://docs.docker.com/notary/getting_started/ |
As the virtualisation platform Docker becomes more and more widely adopted, it becomes increasingly important to secure images and containers throughout their entire lifecycle. Docker Notary is a component introduced with Docker Engine 1.8 that aims at securing the “last mile” of this lifecycle (image distribution from the registry to the Docker client), by verifying the image publisher as well as image integrity. The verification is based on digital signatures created with multiple keys. Since an in-depth security analysis of key compromise scenarios in the context of Notary seems to be missing in scientific literature, an extensive attack model is developed in this work. Based on the model, it is argued that particularly devastating attacks can be prevented by storing some of the signing keys in a hardware vault, a Secure Element. It is described how an interface can be integrated into the Notary codebase that makes this possible. It is also shown that Notary’s signing process needs to be extended to prevent an attacker from exploiting one particular Notary component, the Notary server, to create arbitrary signatures on his behalf.[*]_
[§] | Docker Notary’s management of signing keys - An analysis of key compromise and integrability of a Secure Element (Simon Wessling, 2018) |