Setting Up Networking

The next step is to set up networking on the master.

First, set up WiFi. This is going to be the link between your cluster and the outside world. Edit the /boot/device-init.yaml file. Update the WiFi SSID and password to match your environment. If you ever want to switch networks, this is the file you need to edit. Once you have edited this, reboot with sudo reboot and validate that your networking is working.

The next step in networking is to set up a static IP address for your cluster’s internal network. To do this, edit /etc/network/interfaces.d/eth0 to read:

allow-hotplug eth0
iface eth0 inet static
    address 10.0.0.1
    netmask 255.255.255.0
    broadcast 10.0.0.255
    gateway 10.0.0.1

This sets the main Ethernet interface to have the statically allocated address 10.0.0.1.

Reboot the machine to claim the 10.0.0.1 address.

Next, we’re going to install DHCP on this master so it will allocate addresses to the worker nodes. Run:

$ apt-get install isc-dhcp-server

Then configure the DHCP server as follows (/etc/dhcp/dhcpd.conf):

# Set a domain name, can basically be anything
option domain-name "cluster.home";

# Use Google DNS by default, you can substitute ISP-supplied values here
option domain-name-servers 8.8.8.8, 8.8.4.4;

# We'll use 10.0.0.X for our subnet
subnet 10.0.0.0 netmask 255.255.255.0 {
    range 10.0.0.1 10.0.0.10;
    option subnet-mask 255.255.255.0;
    option broadcast-address 10.0.0.255;
    option routers 10.0.0.1;
}
default-lease-time 600;
max-lease-time 7200;
authoritative;

You may also need to edit /etc/defaults/isc-dhcp-server to set the INTERFACES environment variable to eth0.

Restart the DHCP server with sudo systemctl restart isc-dhcp-server.

Now your machine should be handing out IP addresses. You can test this by hooking up a second machine to the switch via the Ethernet. This second machine should get the address 10.0.0.2 from the DHCP server.

Remember to edit the /boot/device-init.yaml file to rename this machine to node-1.

The final step in setting up networking is setting up network address translation (NAT) so that your nodes can reach the public internet (if you want them to be able to do so).

Edit /etc/sysctl.conf and set net.ipv4.ip_forward=1 to turn on IP forwarding.

Then edit /etc/rc.local (or the equivalent) and add iptables rules for forwarding from eth0 to wlan0 (and back):

$ iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
$ iptables -A FORWARD -i wlan0 -o eth0 -m state \
  --state RELATED,ESTABLISHED -j ACCEPT
$ iptables -A FORWARD -i eth0 -o wlan0 -j ACCEPT

At this point, basic networking setup should be complete. Plug in and power up the remaining two boards (you should see them assigned the addresses 10.0.0.3 and 10.0.0.4). Edit the /boot/device-init.yaml file on each machine to name them node-2 and node-3, respectively.

Validate this by first looking at /var/lib/dhcp/dhcpd.leases and then SSH to the nodes (remember again to change the default password first thing). Validate that the nodes can connect to the external internet.

...
10.0.0.1 kubernetes
10.0.0.2 node-1
10.0.0.3 node-2
10.0.0.4 node-3
...

Installing Kubernetes

At this point you should have all nodes up, with IP addresses and capable of accessing the internet. Now it’s time to install Kubernetes on all of the nodes.

Using SSH, run the following commands on all nodes to install the kubelet and kubeadm tools. You will need to be root for the following commands. Use sudo su to elevate to the root user.

First, add the encryption key for the packages:

# curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -

Then add the repository to your list of repositories:

# echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" \
  >> /etc/apt/sources.list.d/kubernetes.list

Finally, update and install the Kubernetes tools. This will also update all packages on your system for good measure:

# apt-get update
$ apt-get upgrade
$ apt-get install -y kubelet kubeadm kubectl kubernetes-cni

Setting Up the Cluster

On the master node (the one running DHCP and connected to the internet), run:

$ sudo kubeadm init --pod-network-cidr 10.244.0.0/16 \
        --apiserver-advertise-address 10.0.0.1 \
        --apiserver-cert-extra-sans kubernetes.cluster.home

Note that you are advertising your internal-facing IP address, not your external address.

Eventually, this will print out a command for joining nodes to your cluster. It will look something like:

$ kubeadm join --token=<token> 10.0.0.1

SSH onto each of the worker nodes in your cluster and run that command.

When all of that is done, you should be able to run and see your working cluster:

$ kubectl get nodes

$ curl https://rawgit.com/coreos/flannel/master/Documentation/kube-flannel.yml \
  > kube-flannel.yaml

$ curl https://rawgit.com/coreos/flannel/master/Documentation/kube-flannel.yml \
|  sed "s/amd64/arm/g" | sed "s/vxlan/host-gw/g" \
  > kube-flannel.yaml

$ kubectl apply -f kube-flannel.yaml

$ kubectl describe --namespace=kube-system configmaps/kube-flannel-cfg
$ kubectl describe --namespace=kube-system daemonsets/kube-flannel-ds

$ DASHSRC=https://raw.githubusercontent.com/kubernetes/dashboard/master
$ curl -sSL \
  $DASHSRC/src/deploy/recommended/kubernetes-dashboard-arm-head.yaml \
  | kubectl apply -f -