/admin

Minikube With Docker For Mac

I'm excited to say that Docker for Mac is now shipping with support for both Docker Swarm and Kubernetes built-in. Let's take a look at what this means, a brief history of developer tooling and then cover my first impressions as I kick the tyres.

Minikube Setup: Docker for Mac / Sierra. GitHub Gist: instantly share code, notes, and snippets.

Why does this matter to developers?

Docker CE (nee. Docker) is a piece of software which has brought the benefits of containers to the people (democratisation) and made them extremely easy to use and valuable (commoditisation). This has been a journey and we didn't start day one with Kubernetes support in Docker for Mac, so let's look at the context:

Containers (which normally means Linux containers) were not available on Windows or Mac when Docker began its story as a spin-out from DotCloud.

Docker tooling - a brief history

Here's a a brief history of developer tools and how they brought Linux containers to Mac and Windows users:

Virtual machines - version 1

Before any of this tooling someone on a Mac or Linux computer who wanted to access Linux container would have needed to have installed a Virtual Machine host such as VirtualBox or VMWare Workstation/Player. They would need to install a Linux host manually and then set up shared folders. Sometimes people also use vagrant from Hashicorp to provide a consistent CLI between VM host software.

Use when: none of the following options work for you. This option is not recommended.

Docker Machine

Docker Machine was the next step in the tooling evolution. Docker Machine automated the setup of a virtual machine on a local or remote environment and made use of standard ISO images (boot2docker) plus a writeable disk area. This also meant upgrading from one version to another was easy.

Once the VM was running SSL certificates were generated and then the Docker client accessed the remote or local VM over TCP/IP. It could support multiple-concurrent Docker versions or hosts at the same time for clustering.

Pros:

  • multiple Docker back-ends even on the same computer
  • works on Linux
  • uses simple distribution boot2docker
  • pluggable architecture - with plugins for major cloud providers / distros

Cons:

With
  • CLI-driven
  • Less 'native' integration on Windows / Mac

Use when: you're on Windows 7 or Windows 10 Home or need a cluster of machines on your local computer. Or you're creating/managing a remote cloud cluster.

Docker for Mac/Windows

The problem with Docker Machine was that it involved too many manual steps (docker-machine env etc) and configuration sometimes needed to be regenerated for TLS. Docker for Mac/Windows or DfM was pitched as a 'native' integration which meant it came with a UI and a menu-bar that was hugely popular. The initial release of DfM was through a limited beta and there was a big demand on Twitter for it.

Pros:

  • 1-click install
  • Command-line is automatically configured
  • Configuration through UI for proxies / registries etc
  • Can be started/stopped with a single click

Cons:

  • DfM has had performance issues with shared volumes
  • High CPU usage is reported by hyperkit resulting in low battery life
  • Only available on Windows 10 Pro or Enterprise

Use when: it's available to you and you need Docker Swarm or Kubernetes support for local development.

Minikube

Minikube has a very similar user-experience to docker-machine and also relies on boot2docker. Its primary purpose is to create a single-node Kubernetes cluster which also includes a Docker host that can be used for development.

Example output of starting up minikube on my Mac:

Pros:

  • Easy access to Kubernetes for local development
  • It works

Cons:

  • Kubernetes uses a significant amount of battery at idle
  • Docker integration feels similar to docker-machine
  • Docker version lags behind significantly (i.e. no multi-stage builds until recently)
  • Some features hard to access or not officially supported such as RBAC (role-based authentication control)
  • Should use minikube start/stop

Use when: you need a local Kubernetes environment but don't mind if the Docker version is older.

To summarise the tooling there are pros and cons to each option. Let's move onto my first impressions of Kubernetes on Docker for Mac.

First impressions

Here are my first impressions through getting hold of the update, kicking the tyres with an application and seeing what the Docker 'stack' integration is like.

Getting it

You currently need to be on the 'edge' track of Docker CE in 17.12 or greater to get Kubernetes support. The support is then downloaded via a UI option and this can take several minutes.

Contexts and namespaces

If you've ever installed minikube then you'll have to switch out of that context into the DfM context otherwise kubectl will hang.

If you find that too verbose to type in then the Kubernetes community has a tool called kubectx that make that shorter.

One of the other key differences between Docker Swarm and Kubernetes is the support for namespaces. By default parts of the Kubernetes ecosystem run as containers in a hidden namespace called system.

To view all containers running type in kubectl get all --all-namespaces:

Artcut

You'll see that a lot of services are running by default. This is in effect the same as what is running for Docker Swarm, but it's hidden from you and baked into a few fixed binaries rather than being split out to this degree.

Let's check which version got shipped?

Looks like we get 1.8.2 which is not the latest but does include all the most important features.

Docker's integration

Docker is attempting to make Kubernetes easy to use for Docker Swarm users by creating an integration between Docker stacks (as seen in Swarm) and Kubernetes native deployments/services.

Kubernetes is by nature extensible and Docker are using Custom Resource Definitions (CRDs) which add a 'stack' concept. It means you can do this:

Let's try a Docker compose file I have and see what happens:

Here's the CRD:

And it appears to have created some Pods:

Prometheus has an issue that I can debug by typing in: kubectl logs pod/prom-76b4f584f7-qckc9.

Take away - if you're still using docker-compose for development or production - you can probably move straight to Kubernetes and enjoy the benefits of clustering.

Native workflow

So for a native workflow we'd expect a few things:

  • kubectl apply with YAML works
  • helm works
  • RBAC is enabled

Let's try to deploy OpenFaaS - Serverless Functions Made Simple for Docker and Kubernetes.

If you run into an error about namespaces then type in kubectl apply -f ./namespaces.yml in the faas-netes folder and try again.

This is a good test because OpenFaaS will display a UI at localhost:31112 and also uses both RBAC and two namespaces (openfaas / openfaas-fn).

We see no errors:

The services are created too. See them by typing in:

Now open the UI and deploy a function using the store:

Then select Figlet - figlet is a Linux binary that can generate ASCII text-logos.

You'll see the Function/Pod created here:

Kubernetes uses more primitive objects to make up a 'service' than Docker Swarm. Find out more about Kubernetes fundamentals below.

Now invoke the function and see the result:

It works very well and was really easy to deploy. This will help the community maintain and build the Kubernetes integration for OpenFaaS.

I said we'd need helm to work too, which is a package manager for distributing software like OpenFaaS, Minio or similar. The equivalent doesn't exist for Docker Swarm.

I saw this Tweet before writing the post so it looks like helm also works out the box:

Just setup #kubernetes on the @Docker for Mac beta and did a #helm install.. this is so so great!!

— Adnan Abdulhussein (@prydonius) December 15, 2017

Wrapping up

One of the key advantages of having this new support in Docker for Mac is not the stacks integration - it's the simplicity, speed and ease of use over existing tools. I believe a portion of existing Docker Swarm users will make use of the stacks integration and it could be a useful upgrade path for them if they decide to move to Kubernetes.

Is Docker Swarm dead? I don't have any inside information as a member of the Docker Captains group, but what I've been told generally is that while there is a customer-base for Docker Swarm it will continue to be supported. For example the original Swarm that ran as individual containers is still supported by Docker's UCP product.

If you read the developer reactions on Hacker News
you'll see Swarm still has its fans and if Kubernetes wants to convert them then my impression is that installation/maintenance for on-prem installations needs to improve.

So as for my first impressions - it's great and everything I was expecting to work does. I'm really impressed with what Docker has done with Docker for Mac and the internal components that make it up like LinuxKit.

Follow and share

Follow me and share this post on Twitter to help me get to 10k followers:

First impressions: Docker for Mac with Kubernetes and a brief history of developer tooling - https://t.co/XVlTaW8tgz@docker@kubernetesiopic.twitter.com/YrftDCEwEa

— Alex Ellis (@alexellisuk) January 7, 2018

See also:

I highly recommend reading the differences between Docker Swarm and Kubernetes - even if you're a Kubernetes veteran, this blog post may be useful for your colleagues and friends:

  • Blog - What you need to know: Docker Swarm and Kubernetes

  • Read the docs on the Docker and Kubernetes integration

  • Star: Award-winning OpenFaaS - serverless functions you can run yourself on Kubernetes or Docker Swarm

'Docker Captains are experts and leaders in their communities who demonstrate a commitment to sharing their Docker knowledge with others.'

Alex Ellis is a Docker Captain and the author of the OpenFaaS project

This is a step-by-step guide to installing and running Kubernetes on your Mac so that you can develop applications locally.

You will be guided through running and accessing a Kubernetes cluster on your local machine using the following tools:

  • Homebrew
  • Docker for Mac
  • Minikube
  • virtualbox
  • kubectl

Installation Guide

The only pre-requisite for this guide is that you have Homebrew installed. Homebrew is a package manager for the Mac. You’ll also need Homebrew Cask, which you can install after Homebrew by running brew tap caskroom/cask in your Terminal.

  1. Install Docker for Mac. Docker is used to create, manage, and run our containers. It lets us construct containers that will run in Kubernetes Pods.

  2. Install VirtualBox for Mac using Homebrew. Run brew cask install virtualbox in your Terminal. VirtualBox lets you run virtual machines on your Mac (like running Windows inside macOS, except for a Kubernetes cluster.)

    Skip to step three if everything has worked to this point.

    In my case, I already had the non-Homebrew VirtualBox app installed which caused issues when trying to start minikube.

    If you already have VirtualBox installed, start the installation as before with brew cask install virtualbox. You will get a warning that confirms this saying Warning: Cask 'virtualbox' is already installed.. Once this is confirmed, you can reinstall VirtualBox with Homebrew by running brew cask reinstall virtualbox.

    If you happen to have VirtualBox already running when you do this, you could see an error saying Failed to unload org.virtualbox.kext.VBoxDrv - (libkern/kext) kext is in use or retained (cannot unload).

    This is because the kernel extensions that VirtualBox uses were in use when the uninstall occurred. If you scroll up in the output of that command, beneath Warning! Found the following active VirtualBox processes: you’ll see a list of the processes that need to be killed.

    Kill each of these in turn by running kill first_column_number (first_column_number is the process identifier for that process).

    Now re-run brew cask reinstall virtualbox and it should succeed.

  3. Install kubectl for Mac. This is the command-line interface that lets you interact with Kuberentes. Run brew install kubectl in your Terminal.

  4. Install Minikube via the Installation > OSX instructions from the latest release. At the time of writing, this meant running the following command in Terminal…

    Minikube will run a Kubernetes cluster with a single node.

  5. Everything should work! Start your Minikube cluster with minikube start. Then run kubectl api-versions. If you see a list of versions, everything’s working! minikube start might take a few minutes.

    6. At this point, I got an error saying Error starting host: Error getting state for host: machine does not exist. because I had previously tried to run Minikube. You can fix this by running open ~/.minikube/ to open Minikube’s data files, and then deleting and deleting the machines directory. Then run minikube start again.

Come Together

You’ve installed all these tools and everything looks like it’s working. A quick explanation of how the components relate is needed.

  • VirtualBox is a generic tool for running virtual machines. You can use it to run Ubuntu, Windows, etc. inside your macOS operating system host.
  • Minikube is a Kubernetes-specific package that runs a Kubernetes cluster on your machine. That cluster has a single node and has some unique features that make it more suitable for local development. Minikube tells VirtualBox to run. Minikube can use other virtualization tools—not just VirtualBox—however these require extra configuration.
  • kubectl is the command line application that lets you interact with your Minikube Kubernetes cluster. It sends request to the Kubernetes API server running on the cluser to manage your Kubernetes environment. kubectl is like any other application that runs on your Mac—it just makes HTTP requests to the Kubernetes API on the cluster.