Internet Performance Delivered right to your inbox

Kubernetes: Running Containers at Scale

Kubernetes is one of the hottest topics in tech right now. But it didn’t happen overnight. In fact, my connection to it goes back many years.

In 2011, I started working at HP, in the HP Cloud group, our goal to build a public cloud using OpenStack. OpenStack provided the means to build public and private clouds running virtual machines with a vibrant community developing all the features one would want to have for a full-featured cloud.

After two years working on a number of projects including DNSaaS (OpenStack Designate), I had the privilege of working on the Advanced Technology Group team. Our goal was to find new technologies that were “two steps ahead” of the current projects that could be leveraged for building clouds.

We first saw that Docker was a project worth delving into. It made it much easier to run containers, which as I mentioned before, had been around for some time, and provided a much easier user experience for running containers:

  • Being able to easily run a container– this means start, stop, execute into, and deletion of a container.
  • Having a user interface to query information about what containers and their state
  • Having the ability to know numerous details about a given container
  • An API that other tools can take advantage of for managing containers such as Ansible
  • Having a means to build, manage, create, modify and index container images with a common file describing how the container is built (Dockerfile) or commit a running container to an image. This also includes running indexed repositories of container images
  • Docker runs as both a client and server

Running containers at scale: Kubernetes

In my most recent blog on Kubernetes, I set the stage by comparing the differences between containers and virtual machines. Today I’ll talk about how to run containers at scale.

With Docker available, the next step was: how can Docker be run on various systems? I had presented a talk at AnsibleFest in 2015 demonstrating the Ansible Docker module running containers on a HP Moonshot, which showed it was possible to use Ansible to run a lot of containers on a machine.

The first container scheduling project my group looked at was CoreOS, which consisted of a stripped-down Linux operating system meant to run containers using a rudimentary scheduler (Fleet) that would run containers (and other processes) backed by the key/value configuration database etcd. Fleet did this by using unit files through systemd scheduled across a given number of machines. The ATG group did some prototyping and internal demonstrations of how CoreOS, Fleet in particular, could be used to run an ELKStack logging setup with just a few commands, which worked quiet well and showed promise as a containerized approach to running what was traditionally something that required more configuration using tools like Chef, Puppet, or Ansible. Not only that, what was emerging was the concept of immutable infrastructure.

Around that time we started noticing a lot of talk about a project called Kubernetes. We were familiar with the Borg Paper that Google had made available detailing their system they use to run a huge cluster of containers — and all of their processes/applications– and how that system was what Kubernetes was inspired and modeled after.

The more we became familiar with Kubernetes and involved ourselves in the community either through speaking at conferences or discussions in the various Slack channels, it was becoming quite apparent that this project was growing at an incredibly fast pace and seemed to be the project and standard for containerized applications.

Also apparent was how the community around it was burgeoning by leaps and bounds and seemed to be organized in such a way that it was apparent that there had been a lot learned in terms of success and mistakes in other community-lead projects, not being tied up in committees and politics.

In the next blog, I’ll discuss immutable infrastructure and the differences between “pets” and “cattle”.

Share Now

Patrick Galbraith
Whois: Patrick Galbraith

Patrick Galbraith is a Principal Platform Engineer at Oracle Dyn Global Business Unit, a pioneer in managed DNS and a leader in cloud-based infrastructure that connects users with digital content and experiences across a global internet.