This is was done for course Palvelinten hallinta ICT4TN022-5 in Haaga-Helia University of Applied Sciences

Different container solutions are gaining quite a lot of popularity and interest in tech industry. Following example shows a simple way to get you started with Docker/Kubernetes deployment with SaltStack. These can also be found from my GitHub!

Pre-requisites

Docker and Kubernets are still relatively new softwares so unfortunately most distributions don’t offer official packages for installation of the following files or the offered files are old.

Docker CE

Docker CE is Docker’s community edition which is provided free from their website. I’m using Debian myself, but you can find installation guide for your distribution from docs.docker.com.

Before installation I make sure that I don’t have any older versions of Docker installed:

$ sudo apt-get remove docker docker-engine docker.io

After that is done I need to set up the repository for Docker. I start by updateing apt package index:

$ sudo apt-get update

After that I install few packages that are needed for using the new repository over HTTPS:

$ sudo apt-get install \
     apt-transport-https \
     ca-certificates \
     curl \
     gnupg2 \
     software-properties-common

Then I need to add Docker’s GPG key:

$ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -

It is good thing to verify that the fingerprint is the following:

9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88

That can be done with apt-key and searching for the last 8 characters:

$ sudo apt-key fingerprint 0EBFCD88

The output should be something like this:

pub   rsa4096 2017-02-22 [SCEA]
      9DC8 5822 9FC7 DD38 854A  E2D8 8D81 803C 0EBF CD88
uid           [ unknown] Docker Release (CE deb) <[email protected]>
sub   rsa4096 2017-02-22 [S]

Now I can add the Docker repository to /etc/apt/sources.list.d:

$ echo "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list

lsb_release -cs here returns the name of your Debian distribution, e.g. stretch.

Now when the new repository is added I need to update our package index with:

$ sudo apt-get update

After that I can install the latest version of Docker CE with:

$ sudo apt-get install -y docker-ce

Minikube

Minikube is a tool that makes running Kubernetes locally easy.

Minikube uses VM for running single node Kubernetes cluster and is mainly used for testing and developing. You can choose from many different VM-drivers for Minikube to work, but I personally prefer VirtualBox, so I’ll install that. Since I’m using currently Debian Stretch I need to add new repository for VirtualBox so I can install it via native package manager. The other way to install it is enabling stretch-backport and install it from earlier Debian release, e.g. Jessie.

Install VirtualBox on Debian Stretch

First start by making new repository list to /etc/apt/sources.list.d:

$ echo "deb http://download.virtualbox.org/virtualbox/debian stretch contrib" | sudo tee /etc/apt/sources.list.d/virtualbox.list

Then add Oracle VirtualBox public key by:

$ wget https://www.virtualbox.org/download/oracle_vbox_2016.asc
$ sudo apt-key add oracle_vbox_2016.asc

After that update apt and install the latest VirtualBox:

$ sudo apt-get update
$ sudo apt-get install -y virtualbox-5.1

Install Minikube

Installing can be done by running:

$ curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/

Again after that test the installation:

$ minikube version
minikube version: v0.26.1

$ minikube start

Kubectl

kubectl is Kubernetes’ command-line tool for managing Kubernetes applications.

Update 10.05.2018: Rewrote installation

kubectl can be installed from apt after you’ve added official Kubernetes repository to /etc/apt/sources.list.d:

First make sure apt can install packages over HTTPS:

$ sudo apt-get update
$ sudo apt-get install -y apt-transport-https

After that, add Kubernetes’ public key to your keyring:

$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

Then add Kubernetes repository to /etc/apt/sources.list.d:

$ echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

And finally update apt package index and install kubectl:

$ sudo apt-get update
$ sudo apt-get install -y kubectl

Then test kubectl installation by running (requires running Kubernetes node):

$ kubectl version

Docker Compose

Docker Compose is a tool for defining and running multi-container Docker applications. Docker Compose uses simple YAML-file for configuring the container services. From that configuration file you can create and start all your containers with one command.

Docker Compose is currently installed from Compose’s repository release page on GitHub.

You can download the binary straight from that page with curl:

$ sudo curl -L https://github.com/docker/compose/releases/download/1.21.0/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose

Note: Check the latest release from the release page, since that link above might be outdated someday

After that I need to give that binary executable rights:

$ sudo chmod +x /usr/local/bin/docker-compose

And finally test the installation:

$ docker-compose --version
docker-compose version 1.21.1, build 5a3f1a3

Spreading the setup with salt-master

Docker

SaltStack state for Docker CE:

$ sudo mkdir -p /srv/salt/docker
$ sudo emacs /srv/salt/docker/init.sls
# Docker Community Edition

{% set gpg = {
    'Debian': 'https://download.docker.com/linux/debian/gpg',
    'Ubuntu': 'https://download.docker.com/linux/ubuntu/gpg',
}.get(grains.os) %}

{% set repo = {
    'Debian': '"deb [arch=amd64] https://download.docker.com/linux/debian stretch stable"',
    'Ubuntu': '"deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable"',
}.get(grains.os) %}

Allow apt to use repository over HTTPS:
  pkg.installed:
    - pkgs:
        - apt-transport-https
        - ca-certificates
        - curl
        - software-properties-common
        {% if grains['os'] == 'Debian' %}
        - gnupg2
        {% endif %}

curl -fsSL {{ gpg }} | apt-key add -:
  cmd.run

/etc/apt/sources.list.d/docker.list:
  file.managed:
    - source: salt://docker/docker.list
    - template: jinja
    - context:
      repo: {{ repo }}

apt-get update:
  cmd.run

docker-ce:
  pkg.installed

Update 10.05.2018: Typo xenial->stretch

Then I make following docker.list for Docker’s repository:

$ sudo emacs /srv/salt/docker/docker.list
{{ repo }}

I test this state on my server which is running Ubuntu 16.04. Output of salt linux-palvelimet state.apply docker --state-output terse:

linux-palvelimet:
  Name: Allow apt to use repository over HTTPS - Function: pkg.installed - Result: Clean Started: - 15:13:53.495303 Duration: 107.086 ms
  Name: curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - - Function: cmd.run - Result: Changed Started: - 15:13:53.604860 Duration: 390.799 ms
  Name: /etc/apt/sources.list.d/docker.list - Function: file.managed - Result: Changed Started: - 15:13:54.002529 Duration: 53.664 ms
  Name: apt-get update - Function: cmd.run - Result: Changed Started: - 15:13:54.056466 Duration: 2533.854 ms
  Name: docker-ce - Function: pkg.installed - Result: Clean Started: - 15:13:56.590644 Duration: 25.779 ms

Summary for linux-palvelimet
------------
Succeeded: 5 (changed=4)
Failed:    0
------------
Total states run:     7
Total run time:  12.465 s

Seems to be working!

Update 10.05.2018: Rewrote apt repository addition

Docker Compose

I also add Docker Compose installation to that same state since quite often these two go hand in hand:

.
.
.
# Docker Compose

curl -L https://github.com/docker/compose/releases/download/1.21.0/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose:
  cmd.run

chmod +x /usr/local/bin/docker-compose:
  cmd.run

Then I apply the state again with the same command salt linux-palvelimet state.apply docker --state-output terse:

linux-palvelimet:
  Name: Allow apt to use repository over HTTPS - Function: pkg.installed - Result: Clean Started: - 15:16:23.434163 Duration: 64.451 ms
  Name: curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - - Function: cmd.run - Result: Changed Started: - 15:16:23.500822 Duration: 263.287 ms
  Name: /etc/apt/sources.list.d/docker.list - Function: file.managed - Result: Clean Started: - 15:16:23.768899 Duration: 44.67 ms
  Name: apt-get update - Function: cmd.run - Result: Changed Started: - 15:16:23.813923 Duration: 2241.98 ms
  Name: docker-ce - Function: pkg.installed - Result: Clean Started: - 15:16:26.056249 Duration: 23.251 ms
  Name: curl -L https://github.com/docker/compose/releases/download/1.21.0/docker-compose-Linux-x86_64 -o /usr/local/bin/docker-compose - Function: cmd.run - Result: Changed Started: - 15:16:26.079777 Duration: 79367.42 ms
  Name: chmod +x /usr/local/bin/docker-compose - Function: cmd.run - Result: Changed Started: - 15:17:45.447595 Duration: 14.193 ms

Summary for linux-palvelimet
------------
Succeeded: 7 (changed=4)
Failed:    0
------------
Total states run:     7
Total run time:  82.019 s

Note 08.05.2018: The state should be rewritten so that it doesn’t add unnecessary commented lines to /etc/apt/sources.list with add-apt-repository

Kubernetes

$ sudo mkdir -p /srv/salt/kubernetes
$ sudo emacs /srv/salt/kubernetes/init.sls
{% set repo = {
    'Debian': '"deb http://apt.kubernetes.io/ kubernetes-xenial main"',
    'Ubuntu': '"deb http://apt.kubernetes.io/ kubernetes-xenial main"',
}.get(grains.os) %}

/etc/apt/sources.list.d/kubernetes.list:
  file.managed:
    - source: salt://kubernetes/kubernetes.list
    - template: jinja
    - context:
      repo: {{ repo }}

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
  cmd.run

/etc/apt/sources.list.d/kubernetes.list:
  file.managed:
    - source: salt://kubernetes/kubernetes.list
    - template: jinja
    - context:
      repo: {{ repo }}

{% if grains['os'] == 'Debian' %}
/etc/apt/sources.list.d/virtualbox.list:
  file.managed:
    - source: salt://kubernetes/virtualbox.list

wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | apt-key add -:
  cmd.run
{% endif %}

apt-get update:
  cmd.run

Allow apt to use repository over HTTPS:
  pkg.installed:
    - pkgs:
        - apt-transport-https
        - ca-certificates
        - curl
        - software-properties-common
        {% if grains['os'] == 'Debian' %}
        - gnupg2
        {% endif %}

{% if grains['os'] == 'Debian' %}
virtualbox-5.1:
  pkg.installed
{% else %}
virtualbox:
  pkg.installed
{% endif %}

# Minikube (Local Kubernetes)

curl -L https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 -o /usr/local/bin/minikube:
  cmd.run

chmod +x /usr/local/bin/minikube:
  cmd.run

# Kubectl

kubectl:
  pkg.installed

Note 10.05.2018: Debian 9 (Stretch) doesn’t have VirtualBox packages in its repositories so it is needed to add new repository for VirtualBox installation.

Update 15.05.2018: Rearrange

Then I make following kubernetes.list and virtualbox.list:

$ sudo emacs /srv/salt/kubernetes/kubernetes.list
{{ repo }}
$ sudo cp /srv/salt/kubernetes/kubernetes.list /srv/salt/kubernetes/virtualbox.list

Output of salt linux-palvelimet state.apply kubernetes --state-output terse:

linux-palvelimet:
  Name: Allow apt to use repository over HTTPS - Function: pkg.installed - Result: Clean Started: - 15:39:55.175335 Duration: 62.456 ms
  Name: virtualbox - Function: pkg.installed - Result: Clean Started: - 15:39:55.238065 Duration: 18.758 ms
  Name: curl -L https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 -o /usr/local/bin/minikube - Function: cmd.run - Result: Changed Started: - 15:39:55.259027 Duration: 685.311 ms
  Name: chmod +x /usr/local/bin/minikube - Function: cmd.run - Result: Changed Started: - 15:39:55.944679 Duration: 11.258 ms
  Name: curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - - Function: cmd.run - Result: Changed Started: - 15:39:55.956254 Duration: 303.107 ms
  Name: /etc/apt/sources.list.d/kubernetes.list - Function: file.managed - Result: Changed Started: - 15:39:56.264703 Duration: 44.172 ms
  Name: apt-get update - Function: cmd.run - Result: Changed Started: - 15:39:56.309156 Duration: 2392.147 ms
  Name: kubectl - Function: pkg.installed - Result: Changed Started: - 15:39:58.701833 Duration: 8011.351 ms

Summary for linux-palvelimet
------------
Succeeded: 8 (changed=6)
Failed:    0
------------
Total states run:     8
Total run time:  11.529 s

Complete Setup Test

I decided to run the test inside Docker. I start by making a directory for my Docker container.

$ mkdir testing
$ cd testing

After that I make a simple Dockerfile that install few packages that are necessary for running this test.

$ emacs Dockerfile
FROM debian:stretch
LABEL maintainer "Topi Kettunen <[email protected]>"

RUN apt-get update && apt-get install -y \
    curl \
    nano \
    salt-minion \
    wget \
    --no-install-recommends \
    && rm -rf /var/lib/apt/lists/*

ENTRYPOINT [ "/bin/bash" ]

Here I start building the container from Docker’s Debian Stretch image. The Dockerfile install few necessary files when the image is built for the first time. I also remove the unnecessary file from /var/lib/apt/lists so that the image size would stay small. Lastly I indicate that when the container is start the container opens bash.

After that I build my Docker image:

$ docker build -t salt-test .

Notice the dot at the end of the command

I name my image salt-test.

After Docker has build the image I can see it on Docker image listing by running command:

$ docker images
REPOSITORY              TAG                 IMAGE ID            CREATED              SIZE
salt-test               latest              9d98d7583ac7        About a minute ago   198MB

Then I can run the container with:

$ docker run -it --rm salt-test
[email protected]:/# 

Here -it means interactive terminal which is needed if we want to use bash inside the container. After that --rm means that the container is deleted after it exits successfully.

When I’m inside the container I configure it to be my minion:

$ echo -e "master: ip-addr\nid: testing" > /etc/salt/minion

After that I restart the salt-minion.service:

$ service salt-minion restart

Note: Docker container images don’t have systemd

After that I go to my master and see if I can connect to that minion:

(master)$ sudo salt-key
Accepted Keys:
debian
linux-palvelimet
Denied Keys:
Unaccepted Keys:
testing
Rejected Keys:
(master)$ sudo salt-key -A
The following keys are going to be accepted:
Unaccepted Keys:
testing
Proceed? [n/Y] y
Key for minion docker accepted.
(master)$ sudo salt docker cmd.run 'whoami'
testing:
	root

After that I apply the states:

(master)$ sudo testing state.apply docker --state-output terse
testing:
  Name: Allow apt to use repository over HTTPS - Function: pkg.installed - Result: Changed Started: - 20:59:13.133181 Duration: 49842.08 ms
  Name: curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add - - Function: cmd.run - Result: Changed Started: - 21:00:02.976555 Duration: 2556.314 ms
  Name: /etc/apt/sources.list.d/docker.list - Function: file.managed - Result: Changed Started: - 21:00:05.534617 Duration: 600.102 ms
  Name: apt-get update - Function: cmd.run - Result: Changed Started: - 21:00:06.134816 Duration: 2005.493 ms
  Name: docker-ce - Function: pkg.installed - Result: Changed Started: - 21:00:08.142944 Duration: 56501.729 ms
  Name: curl -L https://github.com/docker/compose/releases/download/1.21.0/docker-compose-Linux-x86_64 -o /usr/local/bin/docker-compose - Function: cmd.run - Result: Changed Started: - 21:01:04.645991 Duration: 16295.185 ms
  Name: chmod +x /usr/local/bin/docker-compose - Function: cmd.run - Result: Changed Started: - 21:01:20.941372 Duration: 428.844 ms

Summary for testing
------------
Succeeded: 7 (changed=7)
Failed:    0
------------
Total states run:     7
Total run time: 128.230 s
(master)$ sudo testing state.apply kubernetes --state-output terse
testing:
  Name: /etc/apt/sources.list.d/kubernetes.list - Function: file.managed - Result: Changed Started: - 21:11:37.924057 Duration: 553.182 ms
  Name: /etc/apt/sources.list.d/virtualbox.list - Function: file.managed - Result: Changed Started: - 21:11:38.477330 Duration: 246.378 ms
  Name: wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | apt-key add - - Function: cmd.run - Result: Changed Started: - 21:11:38.724145 Duration: 2620.676 ms
  Name: apt-get update - Function: cmd.run - Result: Changed Started: - 21:11:41.345009 Duration: 2330.661 ms
  Name: Allow apt to use repository over HTTPS - Function: pkg.installed - Result: Clean Started: - 21:11:44.783453 Duration: 1244.129 ms
  Name: virtualbox-5.1 - Function: pkg.installed - Result: Changed Started: - 21:11:46.027720 Duration: 105520.02 ms
  Name: curl -L https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 -o /usr/local/bin/minikube - Function: cmd.run - Result: Changed Started: - 21:13:31.549700 Duration: 20557.019 ms
  Name: chmod +x /usr/local/bin/minikube - Function: cmd.run - Result: Changed Started: - 21:13:52.106884 Duration: 424.875 ms
  Name: curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - - Function: cmd.run - Result: Changed Started: - 21:13:52.531933 Duration: 912.819 ms
  Name: kubectl - Function: pkg.installed - Result: Changed Started: - 21:13:53.447393 Duration: 11841.639 ms

Summary for testing
-------------
Succeeded: 10 (changed=9)
Failed:     0
-------------
Total states run:     10
Total run time:  146.251 s

After that I check if the programs work:

$ docker version
Client:
 Version:      18.03.1-ce
 API version:  1.37
 Go version:   go1.9.5
 Git commit:   9ee9f40
 Built:        Thu Apr 26 07:17:14 2018
 OS/Arch:      linux/amd64
 Experimental: false
 Orchestrator: swarm
$ docker-compose version
docker-compose version 1.21.0, build 5920eb0
docker-py version: 3.2.1
CPython version: 3.6.5
OpenSSL version: OpenSSL 1.0.1t  3 May 2016
$ minikube version
minikube version: v0.26.1

Note: minikube start doesn’t work with the current configurations when ran inside VM or Docker container

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.2", GitCommit:"81753b10df112992bf51bbc2c2f85208aad78335", GitTreeState:"clean", BuildDate:"2018-04-27T09:22:21Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

User Story - For Who?

Due to the growing popularity and interest towards containers, these SaltStack states offer quick and easy installation of necessary tools and softwares to get you started with Docker/Kubernetes deployment.

TODO

  • Currently works only for Debian Stretch and Ubuntu Xenial (16.04). Support for more distributions would be useful.

  • Possibly Kubernetes multi-node cluster with kubeadm and DIND (Docker-in-Docker). (Strong maybe)

Resources