How to build production-ready hosting infrastructure with Kubernetes - Part 1: Cluster API

Are you interested in Kubernetes? Would you like to use it but got disheartened by the complexity and the maintenance effort? Or do you use it already and ran into one of the many problems that occur in practice? Then this article is for you.

Kubernetes is complex, expensive, and takes a lot of effort to maintain. Managed Kubernetes reduces complexity and some maintenance effort but is usually quite expensive. We will explain how the Cluster API together with its Hetzner provider integration might solve your problems.

Disclainer: This article is written by the maintainers of Cluster API Provider Hetzner. It is written in a subjective way and should inform readers about the advantages of Cluster API together with the Hetzner provider integration. Readers should not expect a detailed and neutral discussion about pros and cons of Cluster API.

Self-Hosted Kubernetes vs Managed Kubernetes and Their Shortcomings

Kubernetes is the widely adopted standard in managing containers. Containers, in turn, are the standard for deploying software. Companies that want to deploy software, either already use containers and Kubernetes, or strive for it. There are two options for getting a Kubernetes cluster: you can host it yourself or you can use a managed solution.

A self-hosted cluster means a lot of responsibility and work to set up and run, but also gives the most freedom to configure the cluster according to individual needs.

Managed Kubernetes is offered by all hyperscalers that claim to significantly reduce the maintenance effort for their customers. Unfortunately, looking at the details of their terms and conditions, they leave their customers alone with most of the cluster management. Nevertheless, many users of these products believe that Managed Kubernetes means that they do not have to do any management on their own.

Therefore, a skilled team of Kubernetes experts is needed in both cases to maintain the cluster environment. As Kubernetes experts are rare, this team won’t be very cheap. Managed Kubernetes reduces the workload of this team to some extend, but also does not have any answer when it comes to facilitating the cluster management that has to be done by their customers. This complicated management, e.g. updating nodes or removing nodes that are unresponsive, is a serious issue in production environments.

No matter whether companies choose a externally or self-managed solution, many choose the infrastructure of the hyperscalers, as they are the ones with the widest range of products. For big companies, this is not so much of a problem, as they get huge discounts for their large infrastructure. For small and medium-sized companies (SMEs), this is another story. Some of them start using products of the hyperscalers, see how expensive their hosting infrastructure gets and try to find another solution. They all ask the same question.

Wouldn’t it be nice to run Kubernetes clusters efficiently on European servers starting from 15 euros a month?

Production-ready, affordable and easy manageable Kubernetes with Cluster API and the Hetzner provider integration

Cluster API (CAPI) in combination with its Hetzner provider integration makes it possible to run affordable and production-ready clusters that are easy maintainable. CAPI is an open-source project to facilitate the set up (day 1) and management (day 2) of clusters. It is a perfect complement to the capabilities of Kubernetes to manage containers.

Cluster API manages infrastructure like Kubernetes manages containers.

We will have a closer look into its features and will give a hands-on example to show how it works.

Cluster API features

It is fully open-source and actively maintained by really smart people, many of them working in the biggest cloud-based companies in the world (Amazon, Microsoft, …). Furthermore, it is an official Kubernetes project.

Cluster API manages infastructure. All you need is to declare how the cluster should look like, including how many worker nodes and control planes. The Cluster API will take care of the rest and build up a cluster that fulfills your requirements. This is called day 1 operations.

Day 2 (i.e. maintaining the cluster) is also much easier with Cluster API. In Kubernetes clusters there are good processes to update software, but nothing comparable to easily update machines. As machine updates are necessary frequently, this is a big short-coming of Kubernetes. With Cluster API intelligent update strategies for machines are possible, e.g. rolling updates. There are also MachineHealthChecks ensuring that all machines are healthy. If that is not the case, they get removed and recreated.

To be able to work with different infastructure providers, so-called provider integrations are needed. They are responsible of managing the actual infrastructure and get or remove servers/load balancers/networks/etc.

There are a lot of them already and more will come. One of the latest provider integrations to join the official list is the purely community driven Cluster API Provider Hetzner. Cluster API unifies the management of different providers through these provider integrations and creates a foundation for a multi-cloud approach.

What is interesting about Cluster API Provider Hetzner?

The Hetzner provider integration has been asked for for a long time in the large community of Hetzner enthusiasts. The community loves Hetzner for the great value you get for much lower costs compared to other cloud providers, like AWS, GCP, Azure, etc. It is even more affordable than DigitalOcean that uses its low prices as a USP. The main disadvantage of Hetzner until now was the lack of good options to run Kubernetes clusters.

Cluster API Provider Hetzner (CAPH) solves this pain point. Since this project started, it has already gained quite some traction and a lot of people have tried it out and even use it in production. CAPH combines the affordability of Hetzner with CAPI, the latest Infrastructure as Software approach to run Kubernetes clusters. It helps you run GDPR-compliant clusters in a European (German) public cloud.

Advantages of Cluster API with Hetzner Provider Integration over Managed Kubernetes solutions

Cluster API is obviously interesting for everyone who wants to run self-managed Kubernetes clusters. On top of that, some people who are currently using the Managed Kubernetes products of AWS, Azure, GCP, or DigitalOcean might want to switch to running their own clusters with Cluster API.

Especially important is that Cluster API makes the management of clusters much easier and accessible for more people. It can perform intelligent update processes of machines and adds, provisions and removes machines automatically based on what you specify.

Cluster API lays the foundation for a multi cloud

There are more advantages though. Cluster API greatly facilitates multi clouds and has no vendor lock-in. You can always switch from one provider to the next or even use multiple at the same time. It is fully open-source with a great community and an ecosystem that becomes larger every day.

Running your own cluster means you have full control and that you can use your own configuration of the control planes. You can use latest Kubernetes versions (note that most of the Managed Kubernetes products are very slow in updating Kubernetes versions) and install your own node images. Custom node images might be important for experts wanting to choose their own container runtime or solutions like crun, runc, or Kata Containers.

Cluster API not only comes without management fee, but also greatly reduces the overall costs if used together with the Hetzner provider, as Hetzner is much more affordable than the other cloud providers.

Running Kubernetes clusters with Cluster API on Hetzner also has the advantage of being purely European, so that there is no issue with GDPR compliance. Many European companies even avoided public cloud altogether, as there was no fully GDPR-compliant and modern alternative to American public clouds.

Run a CAPI cluster in 15 minutes

To be able to understand a bit better how CAPI and the Hetzner provider work, there is a 15 minutes tutorial on Github. Here we will go through it and set up your own Kubernetes cluster!

Basic requirements

A local kind cluster (more advanced: an actual Kubernetes cluster)
clusterctl
kubectl
A Hetzner account

Before setting up the cluster, we some prelimiary work is necessary. First, create a project via the Hetzner Cloud UI and as well as an API token with read&write access. Then create an SSH key and upload it via Hetzner UI into the project.

Now we are ready to start the Hetzner Cloud (HCloud) cluster. We’ll only need six steps.

Create a kind cluster

kind create cluster

Initialize the CAPI cluster

clusterctl init --core cluster-api --bootstrap kubeadm --control-plane kubeadm --infrastructure hetzner

Export some environment variables

export HCLOUD_TOKEN="<YOUR-TOKEN>" \
export HCLOUD_SSH_KEY="<ssh-key-name>" \
export CLUSTER_NAME="my-cluster" \
export HCLOUD_REGION="fsn1" \
export CONTROL_PLANE_MACHINE_COUNT=3 \
export WORKER_MACHINE_COUNT=3 \
export KUBERNETES_VERSION=1.23.4 \
export HCLOUD_CONTROL_PLANE_MACHINE_TYPE=cx21 \
export HCLOUD_WORKER_MACHINE_TYPE=cx21

Create a secret with the API token

kubectl create secret generic hetzner --from-literal=hcloud=$HCLOUD_TOKEN && kubectl patch secret hetzner -p '{"metadata":{"labels":{"clusterctl.cluster.x-k8s.io/move":""}}}'

Generate the CAPI cluster with clusterctl

clusterctl generate cluster my-cluster --kubernetes-version v1.23.4 --control-plane-machine-count=3 --worker-machine-count=3  > my-cluster.yaml

Apply the cluster yaml file

kubectl apply -f my-cluster.yaml

That’s it! Now you can have a look in the HCloud UI and see how the servers appear out of nowhere one after the other! You can also have a look on the information the Cluster API/Kubernetes has about the cluster with the command kubectl get cluster.

If you want to try out even more, then you can scale the cluster to more worker machines:

kubectl scale --replicas=5 md/my-cluster-md-0"

Conclusion - affordable, maintainable and production-ready Kubernetes with Cluster API

Cluster API is a very interesting and new approach to manage cluster infrastructure. It makes Infrastructure as Software usable for everybody. Cluster API can be used on various providers. The Hetzner provider stands out because of the large community of developers and companies who value Hetzner’s affordability.

Naturally, there is a lot more going on than could be shown in this quick introduction. Running Kubernetes clusters is never easy and the Cluster API only facilitates some aspects of it. If you are interested in using Cluster API in production, especially with Hetzner, you can contact us via info@syself.com. We are looking forward to supporting you.