Core AWS Setup

The rest of this section assumes that you have access to an AWS account and some familiarity with the service. You can learn about pricing and create a new account at aws.amazon.com/free/. Amazon offers a free service tier, which may be enough for you to experiment with if you don’t already have a paid account. After you have your AWS account set up, you will need at least one administrative user, a key pair, a virtual private cloud (VPC), and a default security group in your environment. If you do not already have these set up, follow the directions in the Amazon documentation.

IAM Role Setup

AWS’s Identity and Access Management (IAM) roles are used to control what actions a user can take within your cloud environment. We need to make sure we can grant access to the right actions before moving on with the Elastic Container Service.

To work with the ECS, you need to create a role called ecsInstanceRole that has the AmazonEC2ContainerServiceforEC2Role managed role attached to it. The easiest way to do this is by logging into the AWS console and then navigating to Identity and Access Management.

1. In the left sidebar, click Roles.

2. Then, click the “Create role” button.

3. Under AWS Service, select Elastic Container Service.

4. Under “Select your use case,” select Elastic Container Service.

5. Click Next: Permissions.

6. Click Next: Review.

7. In Role Name, type: ecsInstanceRole.

8. Click “Create role.”

If you are interested in storing container configuration in an S3 object storage bucket, take a look at the Amazon ECS Container Agent Configuration documentation.

AWS CLI Setup

Amazon supplies command-line tools that make it easy to work with their API-driven infrastructure. You will need to install a very recent version of the AWS command-line interface (CLI) tools. Amazon has detailed documentation that covers installation of their tools, but the basic steps are as follows.

$ easy_install awscli

$ aws --version

aws-cli/1.14.50 Python/3.6.4 Darwin/17.3.0 botocore/1.9.3

$ aws configure

AWS Access Key ID [None]: EXAMPLEEXAMPLEEXAMPLE
AWS Secret Access Key [None]: ExaMPleKEy/7EXAMPL3/EXaMPLeEXAMPLEKEY
Default region name [None]: us-east-1
Default output format [None]: json

$ aws iam list-users

{
    "Users": [
        {
            "Path": "/",
            "UserName": "administrator",
            "UserId": "ExmaPL3ExmaPL3ExmaPL3Ex",
            "Arn": "arn:aws:iam::936262807352:user/myuser",
            "CreateDate": "2017-12-14T19:33:23Z"
        }
    ]
}

Container Instances

The first thing you need to do after installing the required tools is to create at least a single cluster that your Docker hosts will register with when they are brought online.

The first thing you need to do is create a cluster in the container service. You will then launch your tasks in the cluster once it’s up and running. For these examples, you should start by creating a cluster called fargate-testing:

$ aws ecs create-cluster --cluster-name fargate-testing

{
    "cluster": {
        "clusterArn": "arn:aws:ecs:us-east-1:1...2:cluster/fargate-testing",
        "clusterName": "fargate-testing",
        "status": "ACTIVE",
        "registeredContainerInstancesCount": 0,
        "runningTasksCount": 0,
        "pendingTasksCount": 0,
        "activeServicesCount": 0
    }
}

Before AWS Fargate was released, you were required to create AWS EC2 instances running docker and the ecs-agent and add them into your cluster. You can still use this approach if you want (EC2 launch type), but Fargate makes it much easier to run a dynamic cluster that can scale fluidly with your workload.

Tasks

Now that our container cluster is set up, we need to start putting it to work. To do this, we need to create at least one task definition. The Amazon Elastic Container Service defines the term task definition as a list of containers grouped together.

To create your first task definition, open up your favorite editor, copy in the following JSON, and then save it as webgame-task.json in your current directory, as shown here:

{
  "containerDefinitions": [
    {
      "name": "web-game",
      "image": "spkane/quantum-game",
      "cpu": 0,
      "portMappings": [
        {
          "containerPort": 8080,
          "hostPort": 8080,
          "protocol": "tcp"
        }
      ],
      "essential": true,
      "environment": [],
      "mountPoints": [],
      "volumesFrom": []
    }
  ],
  "family": "fargate-game",
  "networkMode": "awsvpc",
  "volumes": [],
  "placementConstraints": [],
  "requiresCompatibilities": [
    "FARGATE"
  ],
  "cpu": "256",
  "memory": "512"
}

git clone \
    https://github.com/bluewhalebook/\
docker-up-and-running-2nd-edition.git

In this task definition, we are saying that we want to create a task family called fargate-game running a single container called web-game that is based on the Quantum web game. This Docker image launches a browser-based puzzle game that uses real quantum mechanics.

In this task definition, we define some constraints on memory and CPU usage for the container, in addition to telling Amazon whether this container is essential to the task. The essential flag is useful when you have multiple containers defined in a task, and not all of them are required for the task to be successful. If essential is true and the container fails to start, then all the containers defined in the task will be killed and the task will be marked as failed. We can also use the task definition to define almost all of the typical variables and settings that would be included in a Dockerfile or on the docker run command line.

To upload this task definition to Amazon, you will need to run a command similar to what is shown here:

$ aws ecs register-task-definition --cli-input-json file://./webgame-task.json

{
    "taskDefinition": {
        "taskDefinitionArn": "arn:aws:ecs:...:task-definition/fargate-game:1",
        "containerDefinitions": [
            {
                "name": "web-game",
                "image": "spkane/quantum-game",
                "cpu": 0,
                "portMappings": [
                    {
                        "containerPort": 8080,
                        "hostPort": 8080,
                        "protocol": "tcp"
                    }
                ],
                "essential": true,
                "environment": [],
                "mountPoints": [],
                "volumesFrom": []
            }
        ],
        "family": "fargate-game",
        "networkMode": "awsvpc",
        "revision": 2,
        "volumes": [],
        "status": "ACTIVE",
        "requiresAttributes": [
            {
                "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
            },
            {
                "name": "ecs.capability.task-eni"
            }
        ],
        "placementConstraints": [],
        "compatibilities": [
            "EC2",
            "FARGATE"
        ],
        "requiresCompatibilities": [
            "FARGATE"
        ],
        "cpu": "256",
        "memory": "512"
    }
}

We can then list all of our task definitions by running the following:

$ aws ecs list-task-definitions

{
    "taskDefinitionArns": [
        "arn:aws:ecs:us-east-1:012345678912:task-definition/fargate-game:1",
    ]
}

Now you are ready to create your first task in your cluster. You do so by running a command like the one shown next. The count argument in the command allows you to define how many copies of this task you want deployed into your cluster. For this job, one is enough.

You will need to modify the following command to reference a valid subnet ID and security-group ID from your AWS VPC. You should be able to find these in the AWS console or by using the AWS CLI commands aws ec2 describe-subnets and aws ec2 describe-security-groups. You can also tell AWS to assign your tasks a public IP address by using a network configuration similar to this:

awsvpcConfiguration={subnets=[subnet-abcd1234],securityGroups=[sg-abcd1234],assignPublicIp=ENABLED}.

$ aws ecs create-service --cluster fargate-testing --service-name \
    fargate-game-service --task-definition fargate-game:1 --desired-count 1 \
    --launch-type "FARGATE" --network-configuration \
    "awsvpcConfiguration={subnets=[subnet-abcd1234],\
    securityGroups=[sg-abcd1234]}"

{
    "service": {
        "serviceArn": "arn:aws:ecs:...:service/fargate-game-service",
        "serviceName": "fargate-game-service",
        "clusterArn": "arn:aws:ecs:...:cluster/fargate-testing",
        "loadBalancers": [],
        "status": "ACTIVE",
        "desiredCount": 1,
        "runningCount": 0,
        "pendingCount": 0,
        "launchType": "FARGATE",
        "platformVersion": "LATEST",
        "taskDefinition": "arn:aws:ecs:...:task-definition/fargate-game:1",
        "deploymentConfiguration": {
            "maximumPercent": 200,
            "minimumHealthyPercent": 100
        },
        "deployments": [
            {
...
            }
        ],
        "roleArn": "arn:...role/ecs.amazonaws.com/AWSServiceRoleForECS",
        "events": [],
        "createdAt": 1520727776.555,
        "placementConstraints": [],
        "placementStrategy": [],
        "networkConfiguration": {
            "awsvpcConfiguration": {
                "subnets": [
                    "subnet-abcd1234"
                ],
                "securityGroups": [
                    "sg-abcd1234"
                ],
                "assignPublicIp": "DISABLED"
            }
        }
    }
}

"role/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS"

aws iam create-service-linked-role \
  --aws-service-name ecs.amazonaws.com

You can now list all of the services in your cluster with the following command:

$ aws ecs list-services --cluster fargate-testing

{
    "serviceArns": [
        "arn:aws:ecs:us-east-1:012345678912:service/fargate-game-service"
    ]
}

To retrieve all the details about your service, run:

$ aws ecs describe-services --cluster fargate-testing \
    --services fargate-game-service

{
    "services": [
        {
...
            "deployments": [
                {
                    "id": "ecs-svc/9223370516126999252",
                    "status": "PRIMARY",
                    "taskDefinition": "arn:...:task-definition/fargate-game:1",
                    "desiredCount": 1,
                    "pendingCount": 1,
                    "runningCount": 0,
                    "createdAt": 1520727776.555,
                    "updatedAt": 1520727776.555,
                    "launchType": "FARGATE",
                    "platformVersion": "1.0.0",
                    "networkConfiguration": {
                        "awsvpcConfiguration": {
                            "subnets": [
                                "subnet-abcd1234"
                            ],
                            "securityGroups": [
                                "sg-abcd1234"
                            ],
                            "assignPublicIp": "DISABLED"
                        }
                    }
                }
            ],
            "roleArn": "...role/ecs.amazonaws.com/AWSServiceRoleForECS",
            "events": [
                {
                    "id": "2781cc1c-bdae-46f3-a767-f53013cc3801",
                    "createdAt": 1520727990.202,
                    "message": "(...game-service) has reached a steady state."
                }
            ],
...
        }
    ],
    "failures": []
}

This output will tell you a lot about all the tasks in your service. In this case we have a single task, which we can see has “reached a steady state.”

If you want to see what individual tasks are running in your cluster, you can run the following:

$ aws ecs list-tasks --cluster fargate-testing

{
    "taskArns": [
        "arn:aws:ecs:...:task/2781cc1c-bdae-46f3-a767-f53013cc3801"
    ]
}

Since you only have a single task in your cluster at the moment, this list is very small. You will also notice that the task ID matches the ID for the task that is listed in “a steady state” in your service.

To get more details about the individual task, you can run the following command after substituting the task ID with the correct one from your cluster:

$ aws ecs describe-tasks --cluster fargate-testing \
  --task 2781cc1c-bdae-46f3-a767-f53013cc3801

{
    "tasks": [
        {
            "taskArn": "arn:aws:...:task/2781cc1c-bdae-46f3-a767-f53013cc3801",
            "clusterArn": "arn:aws:ecs:...:cluster/fargate-testing",
            "taskDefinitionArn": "arn:aws:...:task-definition/fargate-game:1",
            "overrides": {
                "containerOverrides": [
                    {
                        "name": "web-game"
                    }
                ]
            },
            "lastStatus": "RUNNING",
            "desiredStatus": "RUNNING",
            "cpu": "256",
            "memory": "512",
            "containers": [
                {
...
                    "name": "web-game",
                    "lastStatus": "RUNNING",
                    "networkBindings": [],
                    "networkInterfaces": [
                        {
                            "attachmentId": "a0d40aec-...-0c4086ed10d7",
                            "privateIpv4Address": "10.11.6.240"
                        }
                    ]
                }
            ],
            "startedBy": "ecs-svc/9223370516124771373",
            "version": 4,
            "connectivity": "CONNECTED",
...
            "group": "service:fargate-game-service",
            "launchType": "FARGATE",
            "platformVersion": "1.0.0",
            "attachments": [
                {
                    "id": "a0d40aec-3469-4eb0-8bd3-0c4086ed10d7",
                    "type": "ElasticNetworkInterface",
                    "status": "ATTACHED",
                    "details": [
                        {
                            "name": "subnetId",
                            "value": "subnet-abcd1234"
                        },
                        {
                            "name": "networkInterfaceId",
                            "value": "eni-abcd1234"
                        },
                        {
                            "name": "macAddress",
                            "value": "0f:c7:7d:ac:d1:ff"
                        },
                        {
                            "name": "privateIPv4Address",
                            "value": "10.4.0.100"
                        }
                    ]
                }
            ]
        }
    ],
    "failures": []
}

If you notice that the lastStatus key is displaying a value of PENDING, this most likely means that your service is still starting up. You can describe the task again to ensure that it has completed transitioning into a RUNNING state. After verifying that the lastStatus key is set to RUNNING, you should be able to test your container.

Testing the Task

You will need a modern web browser installed on your system to connect to the container and test the web game.

In the previous output, you’ll notice that the privateIPv4Address for the example task was listed as 10.4.0.100. Yours will be different, and you may also have a publicIPv4Address if you configured your service for that.

Ensure that you are connected to a network that can reach either the public or private IP address of your host, then launch your web browser and navigate to port 8080 on that IP address.

In the example, this URL would look like:

http://10.4.0.100:8080/

If everything is working as expected, you should be greeted by the splash screen for “The Quantum Game,” which will then be quickly followed by the puzzle board.

The official version of the game can be found at http://quantumgame.io.

Stopping the Task

Right, so we have a running task. Now let’s take a look at stopping it. To do that, you need to know the task ID. One way to obtain this is by relisting all the tasks running in your cluster.

$ aws ecs list-tasks --cluster fargate-testing

{
    "taskArns": [
        "arn:aws:ecs:...:task/2781cc1c-bdae-46f3-a767-f53013cc3801"
    ]
}

You can also obtain it from the service information:

$ aws ecs describe-services --cluster fargate-testing \
    --services fargate-game-service

{
...
                {
                    "id": "a63e7114-9592-417c-96cf-559b82096cbc",
                    "createdAt": 1520730006.052,
                    "message": "(service fargate-game-service) has started 1..."
                }
...
}

Finally, we can stop the task by running the following command with the correct task ID:

$ aws ecs stop-task --cluster fargate-testing \
    --task 2781cc1c-bdae-46f3-a767-f53013cc3801

{
...
        "lastStatus": "RUNNING",
        "desiredStatus": "STOPPED",
...
}

If you describe the task again using the same task ID, you should now see that the lastStatus key is set to STOPPED:

$ aws ecs describe-tasks --cluster fargate-testing \
    --task 2781cc1c-bdae-46f3-a767-f53013cc3801

{
...
            "lastStatus": "STOPPED",
            "desiredStatus": "STOPPED",
...
}

And finally, listing all the tasks in our cluster should return an empty set:

$ aws ecs list-tasks --cluster fargate-testing

{
    "taskArns": []
}

At this point, you can start creating more complicated tasks that tie multiple containers together and rely on the ECS and Fargate tooling to spin up hosts and deploy the tasks into your cluster as needed.

What Is Minikube?

Minikube is a whole distribution of Kubernetes for a single instance. It runs on a virtual machine on your own computer and allows you to use all the same tooling against the cluster that you would use in a production system. In scope, it’s a little bit like Docker Compose: it will let you stand up a whole stack locally. It goes one step further than Compose, though, in that it actually has all the production APIs. As a result, if you run Kubernetes in production, you can have an environment on your desktop that is reasonably close in fucntion, if not in scale, to what you are running in production.

Minikube is fairly unique in that all of the distribution is controlled from a single binary you download and run locally. It will autodetect which virtual machine (VM) manager you have locally and will set up and run a VM with all of the necessary tooling on it. That means getting started with it is pretty simple.

So let’s install it!

Installing Minikube

Most of the installation is the same across all platforms because once you have the tools installed, they will be your gateway to the VM running your Kubernetes installation. Just skip to the section that applies to your operating system. Once you have the tool up and running, you can follow the shared documentation.

We need two tools to use Minikube effectively: minikube and kubectl. For the purposes of our simple installation, we’re going to leverage the fact that both of these commands are static binaries with no outside dependencies, which makes them easy to install.

$ brew cask install minikube

==> Auto-updated Homebrew!
...
==> Updated Formulae
...
==> Satisfying dependencies
All Formula dependencies satisfied.
==> Downloading https://storage.googleapis.com/minikube/.../minikube-darwin-amd64
######################################################################## 100.0%
==> Verifying checksum for Cask minikube
==> Installing Cask minikube
==> Linking Binary 'minikube-darwin-amd64' to '/usr/local/bin/minikube'.

$ which minikube
/usr/local/bin/minikube

$ brew install kubernetes-cli
==> Downloading https://.../kubernetes-cli-1.9.3.high_sierra.bottle.tar.gz
Already downloaded: /.../kubernetes-cli-1.9.3.high_sierra.bottle.tar.gz
==> Pouring kubernetes-cli-1.9.3.high_sierra.bottle.tar.gz
==> Caveats
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

zsh completions have been installed to:
  /usr/local/share/zsh/site-functions
==> Summary
  /usr/local/Cellar/kubernetes-cli/1.9.3: 172 files, 65.4MB

$ which kubectl
/usr/local/bin/kubectl

# Download the file, save as 'minikube'
$ curl -Lo minikube \
    https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64

# Make it executable
$ chmod +x minikube

# Move it to /usr/local/bin
$ sudo mv minikube /usr/local/bin/

# Get the latest version number
$ KUBE_VERSION=$(curl -s \
    https://storage.googleapis.com/kubernetes-release/release/stable.txt)

# Fetch the executable
$ curl -LO \
    https://storage.googleapis.com/kubernetes-release/\
release/$(KUBE_VERSION)/bin/linux/amd64/kubectl

# Make it executable
$ chmod +x kubectl

# Move it to /usr/local/bin
$ sudo mv kubectl /usr/local/bin/

Running Kubernetes

Now that we have the minikube tool, we can use it to bootstrap our Kubernetes cluster. This is normally pretty straightforward. You usually don’t need to do any configuration beforehand. You can simply run:

$ minikube start
Starting local Kubernetes v1.9.0 cluster...
Starting VM...
Downloading Minikube ISO
 142.22 MB / 142.22 MB [============================================] 100.00% 0s
Getting VM IP address...
Moving files into cluster...
Downloading localkube binary
 162.41 MB / 162.41 MB [============================================] 100.00% 0s
 0 B / 65 B [----------------------------------------------------------]   0.00%
 65 B / 65 B [======================================================] 100.00% 0s
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
Kubectl is now configured to use the cluster.
Loading cached images from config file.

So what did we just do? Minikube packs a lot into that one command. We installed a virtual machine that has a properly configured version of Docker on it. It then runs all of the necessary components of Kubernetes inside Docker containers on the host.

We can look at those containers to see what we got:

$ minikube ssh
                         _             _
            _         _ ( )           ( )
  ___ ___  (_)  ___  (_)| |/')  _   _ | |_      __
/' _ ` _ `\| |/' _ `\| || , <  ( ) ( )| '_`\  /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )(  ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)

$

On your Kubernetes cluster you probably won’t be SSHing into the command line that often. But we want to see what’s installed and get a handle on the fact that when we run minikube, we’re controlling a virtual machine. Let’s take a look at what is running on the Docker instance on our Kubernetes cluster:

$ docker ps
CONTAINER ID   IMAGE                               COMMAND                  ...
6039cd53ec91   gcr.io/k8s.../storage-provisioner   "/storage-provisioner"   ...
a28e64d209f7   fed89e8b4248                        "/sidecar --v=2 --..."   ...
e84b6d75105b   459944ce8cc4                        "/dnsmasq-nanny -v..."   ...
539530cbe6e7   512cd7425a73                        "/kube-dns --domai..."   ...
e73d514c68bf   e94d2f21bc0c                        "/dashboard --inse..."   ...
21e4b12c144f   gcr.io/google_.../pause-amd64:3.0   "/pause"                 ...
696ac03d09f5   gcr.io/google_.../pause-amd64:3.0   "/pause"                 ...
47282c695e9e   gcr.io/google_.../pause-amd64:3.0   "/pause"                 ...
92b1a4d2cd0c   d166ffa9201a                        "/opt/kube-addons.sh"    ...
97bab4a81ea8   gcr.io/google_.../pause-amd64:3.0   "/pause"                 ...

We won’t dive too much into what each components is. But by now you should hopefully see how the mechanism works. Also, it’s pretty easy to upgrade the components since they are just containers, are versioned, and can be pulled from Google’s container repository.

$ minikube status
minikube: Running
cluster: Running
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.100

Kubernetes Dashboard

Now that we have Minikube up and running, we don’t just have the command-line tools to interact with, we actually have a whole UI installed that we can connect to. We reach it via the minikube dashboard command. Go ahead and run that—it should launch your web browser, pointed to the correct IP address and port of the Kubernetes dashboard! There is a lot of stuff on the dashboard and we’re not able to cover it all, but you should feel free to click around and explore. Many of the terms on the left will be familiar to you at this point, and some will be totally foreign. If you don’t have a computer in front of you, Figure 10-2 shows a screenshot of what an empty Minikube installation looks like from the dashboard.

Figure 10-2. Kubernetes dashboard

If you explore the Nodes link on the lefthand menu, you should see a single node in the cluster, named minikube. This is the virtual machine we started, and the dashboard, like the other components, is hosted in one of the containers we saw when we SSH’d into the host earlier. We’ll take another look at the dashboard when we’ve actually deployed something into our cluster.

While clicking around, you may notice that Kubernetes itself shows up as a component inside the system, just like your applications will.

Kubernetes Containers and Pods

Now that we have a Kubernetes cluster up and running and you’ve seen how easy that is to do locally, we need to pause to talk about a concept that Kubernetes adds on top of the container abstraction. Kubernetes came out of the experiences that Google had running their own massive platform. They encountered most of the situations you might see in a production platform and had to work out concepts to make it easier to understand and solve the kinds of problems you run into when managing a large installation. In doing so, they created a complex set of new abstractions. Kubernetes embraces many of these and thus has a whole vocabulary unto itself. We won’t try to get into all of these, but it’s important to understand the most central of these new abstractions—a concept that sits a layer above the container and is known as a pod.

In Kubernetes parlance, a pod is one or more containers sharing the same cgroups and namespaces. You can also isolate the containers themselves from each other inside the same pod using cgroups and namespaces. The idea is that you may have applications that need to be deployed together all the time and that the abstraction your scheduler needs to work with is the group of applications, not just a single container. All of the containers in the pod can talk to each other on localhost, which eliminates any need to discover each other. So why not just deploy a big container with all the applications inside it? The advantage of a pod over a supercontainer is that you can still resource-limit the individual application separately, and leverage the large library of public Docker containers to construct your application.

Additionally, Kubernetes administrators often leverage the pod abstraction to have a container run on pod startup to make sure things are configured properly for the others, to maintain a shared resource, or to announce the application to others, for example. This allows you to make finer-grained containers than you might if you have to group things into the same container. Another nice part of the pod abstraction is the ability to share mounted volumes.

Pods have a lifespan much like a Docker container. They are essentially ephemeral and can be moved between hosts according to the lifecycle of the application or the host it runs on. Containers in a pod even share the same IP address when facing the outside world, which means they look like a single entity from the network level. Just as you would run only one instance of an application per container, you generally run one instance of that container per pod. The easiest way to think about pods is that they are a group of Docker containers that work tegether as if they were one container, for most purposes. If you need only one container, then you still get a pod deployed by Kubernetes, but that pod contains only one container. The nice thing about this is that there really is only one abstraction as far as the Kubernetes scheduler is concerned: the pod. Containers are managed by some of the runtime pieces that construct the pod and also by the configuration that you use to define them.

One critical difference between a pod and a container is that you don’t construct pods in a build step. They are a runtime abstraction that lives only inside Kubernetes. So you build your Docker containers and send them to a registry, then define and deploy your pods using Kubernetes. In reality you don’t usually directly describe a pod, either; the tools generate it for you through the concept of a deployment. But the pod is the unit of execution and scheduling in a Kubernetes cluster. There is a lot more to it, but that’s the basic concept and it’s probably easiest to understand with a simple example. The pod abstraction is more complicated than thinking of your system in terms of individual containers, but it can be pretty powerful.

Let’s Deploy Something

When actually working with pods in Kubernetes, we usually manage them through the abstraction of a deployment. A deployment is just a pod definition with some health monitoring and replication. It contains the definition of the pod and a little metadata about it. So let’s look at a basic deployment and get it running.

The simplest thing we can deploy on Kubernetes is a pod that contains just one container. The Minikube project ships a sample application called echoserver that we can use to explore the basics of deployment on Kubernetes. We’ll call our deployment hello-minikube just like the Minikube documentation does.

We’ve used the minikube command, but to get things done on Kubernetes itself, we now need to leverage the kubectl command we installed earlier.

$ kubectl run hello-minikube --image=k8s.gcr.io/echoserver:1.4 --port=8080
deployment "hello-minikube" created

To see what that did for us, we can use the kubectl get command to list what’s now in our cluster. We’ll trim this down to the most interesting parts:

$ kubectl get all
NAME                    DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/hello-minikube   1         1         1            1           2m

NAME                         DESIRED   CURRENT   READY     AGE
rs/hello-minikube-c6c6764d   1         1         1         2m

NAME                               READY     STATUS    RESTARTS   AGE
po/hello-minikube-c6c6764d-cjcb4   1/1       Running   0          2m

With that one command, Kubernetes created a deployment, a replica set to manage scaling, and a pod. We ran this a couple minutes after running the first command so our pod is in the READY state. If yours isn’t, just wait and run the command a couple more times until you see it become READY. The bottom entry is a running service that represents Kubernetes itself. But where is our service? We can’t get to it yet. It’s essentially in the same state a Docker container would be if you didn’t tell it to expose any ports. So we need to tell Kubernetes to do that for us:

$ kubectl expose deployment hello-minikube --type=NodePort
service "hello-minikube" exposed

This has now created a service we can reach and interact with. A service is a wrapper for one or more deployments of an application and can tell us how to contact the application. In this case, we get a NodePort, which works a lot like EXPOSE would for a container on Docker itself, but this time for the whole deployment. Let’s get Kubernetes to tell us how to get to it:

$ kubectl get svc
NAME             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
hello-minikube   NodePort    10.101.150.217   <none>        8080:30616/TCP   3s
kubernetes       ClusterIP   10.96.0.1        <none>        443/TCP          4d

You might think you could now connect to http://10.101.150.217:8080 to get to our service. But those addresses are not reachable from your host system because of the virtual machine in which Minikube is running. So we need to get minikube to tell us where to find the service:

$ minikube service hello-minikube --url
http://192.168.99.100:30616

The nice thing about this command, like many of the other Kubernetes commands, is that it is scriptable and command-line-friendly. If we want to open it with curl on the command line, we can just include the minikube command call in our request:

$ curl $(minikube service hello-minikube --url)
CLIENT VALUES:
client_address=172.17.0.1
command=GET
real path=/
query=nil
request_version=1.1
request_uri=http://192.168.99.100:8080/

SERVER VALUES:
server_version=nginx: 1.10.0 - lua: 10001

HEADERS RECEIVED:
accept=*/*
host=192.168.99.100:30616
user-agent=curl/7.54.0
BODY:
-no body in request-

This uses nginx to grab some information from the request and play it back to us. Not the world’s most exciting application, admittedly, but you can see that we get back our own address, visible inside the Minikube virtual machine, and the request we passed with curl.

This is really the simplest use case. We didn’t really configure or define anything and relied on Kubernetes to do the right thing using its defaults. Next we’ll take a look at something more complicated. But first, let’s shut down our new service and deployment. It takes two commands to do that: one to remove the service and the other to delete it.

$ kubectl delete service hello-minikube
service "hello-minikube" deleted
$ kubectl delete deployment hello-minikube
deployment "hello-minikube" deleted
$ kubectl get all
NAME                               READY     STATUS        RESTARTS   AGE

NAME             TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
svc/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   4d

Deploying a Realistic Stack

Let’s now deploy something that looks more like a production stack. We’ll deploy an application that can fetch PDF documents from an S3 bucket, cache them on disk locally, and rasterize individual pages to PNG images on request, using the cached document. To run this application, we’ll want to write our cache files somewhere other than inside the container. We want to have them go somewhere a little more permanent and stable. And this time we want to make things repeatable, so that we’re not deploying our application through a series of CLI commands that we need to remember and hopefully get right each time. Kubernetes, much like Docker Compose, lets us define our stack in one or more YAML files that contain all of the definitions we care about in one place. This is what you want in a production environment and is similar to what you’ve seen for the other production tools.

The service we’ll now create will be called lazyraster (as in, “rasterize on demand”), and so each time you see that in the YAML definition, you’ll know we’re referring to our application. Our persistent volume will be called cache-data. Again, Kubernetes has a huge vocabulary that we can’t entirely address here, but to make it clear what we’re looking at we need to introduce two more concepts: PersistentVolume and PersistentVolumeClaim. A PersistentVolume is a physical resource that we provision inside the cluster. Kubernetes has support for many kinds of volumes, from local storage on a node to EBS volumes on AWS and similar on other cloud providers. It also supports NFS and other more modern network filesystems. A PersistentVolume stores data whose lifecycle is independent from our application or deployments. This lets us store data that persists between application deployments. For our cache, that’s what we’ll use. A PersistentVolumeClaim is a link between the physical resource of the PersistentVolume and the application that needs to consume it. We can set a policy on the claim that allows either a single read/write claim or many read claims. For our application we just want a single read/write claim to our cache-data PersistentVolume.

So let’s take a look at that big YAML file, which we’ll call lazyraster-service.yaml when it’s all pasted together. We’ll break it into sections so we can explain each part more easily. You can either pass each of these to kubectl separately or join them together into a single file, separated by ---, and pass them all at once, as we’ll do next. But it’s a lot easier to reason about the definitions separately.

git clone \
    https://github.com/bluewhalebook/\
docker-up-and-running-2nd-edition.git

Service Definition

apiVersion: v1
kind: Service
metadata:
  name: lazyraster
  labels:
    app: lazyraster
spec:
  type: NodePort
  ports:
    - port: 8000
      targetPort: 8000
      protocol: TCP
  selector:
    app: lazyraster

The first section defines our Service. The second and third sections, which we’ll see in a moment, respectively define our PersistentVolumeClaim and then our actual Deployment. We’ve told Kubernetes that our service will be called lazyraster and that it will be exposed on port 8000 which maps to the actual 8000 in our container. We’ve exposed that with the NodePort mechanism, which simply makes sure that our application is exposed on the same port on each host, much like the -p flag to Docker itself. This is helpful with minikube since we’ll run only one instance, and the NodePort type makes it easy for us to access it from our own computer just like we did earlier. As with many part of Kubernetes, there are a number of options other than NodePort, and you can probably find a mechanism that’s ideal for your production environment. NodePort is good for minikube, but it might work well for more statically configured load balancers as well.

So, back to our Service definition. The Service is going to be connected to the Deployment via the selector, which we apply in the spec section. Kubernetes widely uses labels as a way to reason about similar components and to tie them together. Labels are key/value pairs that are arbitrarily defined and which can then be queried to identify pieces of your system. Here the selector tells Kubernetes to look for Deployments with the label app: lazyraster. Notice that we also apply the same label to the Service itself. That’s helpful if we want to identify all the components together later, but it’s the selector section that actually ties the Deployment to our Service. Great, we have a Service, but it doesn’t do anything yet. We need more definitions to make Kuberenetes do what we want.

PersistentVolumeClaim Definition

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: cache-data-claim
  labels:
    app: lazyraster
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Mi

The next section defines our PersistentVolumeClaim and likewise the PersistentVolume that backs it. A PersistentVolumeClaim is a way to name a volume and claim that you have a token to access that particular volume in a particular way. Notice, though, that we didn’t actually define the PersistentVolume here. That’s because Kubernetes is doing that work for us using what it calls Dynamic Volume Provisioning. In our case the use is pretty simple: we want a read/write claim to a volume and we’ll let Kubernetes put that in a volume container for us. But you can imagine a scenario where an application is going be to deployed into a cloud provider and where dynamic provisioning would really come into its own. In that scenario, we really don’t want to have to make separate calls to have our volume created in the cloud for us. We want Kubernetes to handle that. That’s what Dynamic Volume Provisioning is all about. Here it will just create a container for us to hold our persistent data, and mount it into our pod when we stake our claim. We don’t do a lot in this section except name it, ask for 100 MB of data, and tell Kubernetes it’s a read/write mount-once-only volume.

Deployment Definition

apiVersion: apps/v1
kind: Deployment
metadata:
  name: lazyraster
  labels:
    app: lazyraster
spec:
  selector:
    matchLabels:
      app: lazyraster
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: lazyraster
    spec:
      containers:
      - image: relistan/lazyraster:demo
        name: lazyraster
        env:
        - name: RASTER_RING_TYPE
          value: memberlist
        - name: RASTER_BASE_DIR
          value: /data
        ports:
        - containerPort: 8000
          name: lazyraster
        volumeMounts:
        - name: cache-data
          mountPath: /data
      volumes:
      - name: cache-data
        persistentVolumeClaim:
          claimName: cache-data-claim

The Deployment actually creates the pods for us and uses the Docker container for our application. We define some metadata about the application, including its name and a label, just like we did for the other definitions. We also apply another selector here to find the other resources we’re tied to. In the strategy section, we say we want to have a RollingUpdate, which is a strategy that causes our pods to be cycled through one-by-one during deployment. We could also pick Recreate, which would simply destroy all existing pods and then create new ones afterward.

In the template section, we define how to actually stamp out copies of this deployment. The container definition includes the Docker image name, the ports to map, volumes to mount, and some environment variables that the lazyraster application needs. The very last part of the spec asks to have our PersistentVolumeClaim named cache-data-claim.

And that’s it for the application definition. Now let’s stand it up!

Deploying the Application

Before we continue, let’s see what’s in our Kubernetes cluster, using the kubectl command:

$ kubectl get all
NAME             TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
svc/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   14d

We have only one thing defined at the moment, a service called svc/kubernetes. A naming convention used widely in Kubernetes is to preface the type of object with the two- or three-letter abbreviation for it. Here, svc means it’s a Service. So let’s go ahead and get our service, deployment, and volume into the cluster!

$ kubectl create -f lazyraster-service.yaml
service "lazyraster" created
persistentvolumeclaim "cache-data-claim" created
deployment "lazyraster" created

That output looks like what we expected: we got a service, a persistent volume claim, and a deployment. So let’s see what’s in the cluster now:

$ kubectl get all
NAME                DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/lazyraster   1         1         1            1           33s

NAME                       DESIRED   CURRENT   READY     AGE
rs/lazyraster-6b8dd8cb66   1         1         1         33s

NAME                DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/lazyraster   1         1         1            1           33s

NAME                       DESIRED   CURRENT   READY     AGE
rs/lazyraster-6b8dd8cb66   1         1         1         33s

NAME                             READY     STATUS    RESTARTS   AGE
po/lazyraster-6b8dd8cb66-mfzsr   1/1       Running   0          33s

NAME             TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
svc/kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP          14d
svc/lazyraster   NodePort    10.100.32.231   <none>        8000:32185/TCP   33s

You can see that a bunch more happened behind the scenes. And also, where is our volume or persistent volume claim? We have to ask for that separately:

$ kubectl get pvc
NAME       STATUS  VOLUME        CAPACITY  ACCESS MODES  STORAGECLASS  AGE
cache-...  Bound   pvc-b0582...  100Mi     RWO           standard      2m

So what about that rs and po that appeared in the get all output? Those are a ReplicaSet and Pod, respectively. A ReplicaSet is a piece of Kubernetes that is responsible for making sure that our application is running the right number of instances all the time and that they are healthy. We don’t normally have to worry about what happens inside the ReplicaSet because the Deployment we created manages one of them for us. You can actually manage the ReplicaSet yourself if need be, but most of the time you won’t need to or want to.

We didn’t tell kubectl any specific number of instances, so we got one. And we can see that both the desired and current states match. We’ll take a look at that in a moment. But first, let’s connect to our application and see what we’ve got.

$ minikube service --url lazyraster
http://192.168.99.100:32185

You will probably get a different IP address and port back. That’s totally fine! This is very dynamic stuff. And that’s why we use the minikube command to manage it for us. So grab the address that came back, open your web browser, and paste it into the URL bar like this: http://<192.168.99.100:32185>/documents/docker-up-and-running-public/sample.pdf?page=1. You’ll need to substitute the IP and port into the URL to make it work for you.

You’ll need to be connected to the internet because the lazyraster application is going to go out to the internet, fetch a PDF from a public S3 bucket, and then render in the first page from the document as a PNG in a process called rasterization. If everything worked, you should see a copy of the front cover of this book! This particular PDF has two pages, so feel free to try changing the argument to ?page=2. If you do that, you may notice it renders much faster than the first page. That’s because the application is using our persistent volume to cache the data. You can also specify width=2048, or ask for a JPEG instead of a PNG with ?imageType=image/jpeg. You could rasterize the front page as a very large JPEG like this:

http://<192.168.99.100:32185>/documents/docker-up-and-running-public/sample.pdf?page=1&imageType=image/jpeg&width=2048

If you have a public S3 bucket with other PDFs in it, you can simply substitute the bucket name for docker-up-and-running-public in the URL to hit your own bucket instead. If you want to play with the application some more, check out its repo on GitHub.

Scaling Up

In real life you don’t just deploy applications, you operate them as well. One of the huge advantages of scheduled workloads is the ability to scale them up and down at will, within the resource constraints available to the system. In our case we only have the one Minikube node, but we can still scale up our service to better handle load and provide more reliability during deployments. Kubernetes, as you might imagine, allows scaling up and down quite easily. For our service we will need only one command to do it. Then we’ll take another look at the kubectl output and also at the Kubernetes dashboard we introduced earlier so we can prove that the service scaled.

In Kubernetes the thing we will scale is not the Service, it’s the Deployment. Here’s what that looks like:

$ kubectl scale --replicas=2 deploy/lazyraster
deployment "lazyraster" scaled

Great, that did something! But what did we get?

$ kubectl get deploy/lazyraster
NAME         DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
lazyraster   2         2         2            2           12d

We now have two instances of our application running. Let’s see what we got in the logs:

$ kubectl logs deploy/lazyraster
Found 2 pods, using pod/lazyraster-6b8dd8cb66-mfzsr
Trying to clear existing Lazyraster cached files (if any) in the background...
Launching Lazyraster service...
time="..." level=info msg="Settings -----------------------"
time="..." level=info msg="  * BaseDir: /data"
time="..." level=info msg="  * HttpPort: 8000"
time="..." level=info msg="  * AdvertiseHttpPort: 8000"
...
time="..." level=info msg=--------------------------------------------------
time="..." level=info msg="No New Relic license found, not starting an agent"
time="..." level=warning msg="Ring manager was nil in delegate!"
time="..." ... msg="No URL signing secret passed...running in insecure mode!"
time="..." level=info msg="Listening on tcp://:6379"
time="..." ... msg="Using anonymous credentials since no AWS env vars found"

We asked for logs for the deployment, but Kubernetes tells us there are two running so it picked one, the most recent in this case. We can see our new replica starting up. If we want to specify a particular instance to look at, we can ask for the logs for that pod with something like kubectl logs po/lazyraster-6b8dd8cb66-rqmfx, using the output from kubectl get po to find the pod in question.

We now have a couple of copies of our application running. What does that look like on the Kubernetes dashboard? Let’s navigate there with minikube dashboard. Once we’re there, we’ll click on the layraster Deployment and should see a screen that looks like Figure 10-3.

Figure 10-3. Lazyraster service dashboard

We encourage you to click around some more in the Kubernetes dashboard to see what else is presented. With the concepts you’ve picked up here, there should be a lot that is clearer now and you can probably figure out some more on your own. Likewise, kubectl has a lot of other options available as well, many of which you’ll need in a real production system. The cheat sheet we linked earlier is a real lifesaver here!

kubectl API

We haven’t shown you an API yet and, as we’ve discussed with Docker, it can be really useful to have a simple API to interact with for scripting and operational needs. You could write programs to talk to the individual components yourself. But for most cases, you can use kubectl as a nice proxy to Kubernetes, and it presents a clean API that is accessible with curl and JSON command-line tools. Here’s an example of what you can do:

$ kubectl proxy
Starting to serve on 127.0.0.1:8001

We’ve now got kubectl itself serving up a web API on the local system! You’ll need to read more about what’s possible, but let’s get it to show us the individual instances of the lazyraster application. We can do that by opening the following URL in a browser, or with curl: http://localhost:8001/api/v1/namespaces/default/endpoints/lazyraster.

There is a lot of output here, but the part we care about is the subsets section:

{
...
   "subsets" : [
      {
         "addresses" : [
            {
               "targetRef" : {
                  "resourceVersion" : "291386",
                  "kind" : "Pod",
                  "namespace" : "default",
                  "uid" : "b05b79e4-2c7b-11e8-915a-0800274dbc88",
                  "name" : "lazyraster-6b8dd8cb66-mfzsr"
               },
               "nodeName" : "minikube",
               "ip" : "172.17.0.2"
            },
            {
               "ip" : "172.17.0.5",
               "nodeName" : "minikube",
               "targetRef" : {
                  "resourceVersion" : "291702",
                  "kind" : "Pod",
                  "uid" : "de00121d-36aa-11e8-b37c-0800274dbc88",
                  "namespace" : "default",
                  "name" : "lazyraster-6b8dd8cb66-rqmfx"
               }
            }
         ],
         "ports" : [
            {
               "protocol" : "TCP",
               "port" : 8000
            }
         ]
      }
   ]
}

What’s interesting here is that we can see that both instances are running on the Minikube host, and that they have different IP addresses. If we were building a cloud-native application that needed to know where the other instances of the application were running, this would be a good way to do that. We could, in fact, run a copy of kubectl proxy in our pod and talk to it directly. We would normally use something like the hyperkube container to do that. If you are interested further, you can explore the documentation some more.