Creating a workload cluster with bare metal servers

Creating Workload Cluster

note

Secrets as of now are hardcoded given we are using a flavor which is essentially a template. If you want to use your own naming convention for secrets then you'll have to update the templates. Please make sure that you pay attention to the sshkey name.

For bare metal, use the Syself CCM until an upstream release includes PR fix robot name lookup after stale cache miss.

In the long run we (Syself) want to switch from our fork to the upstream CCM.

The CCM is the "Cloud Controller" which runs in the workload-cluster. The most important tasks of the CCM are:

  • Set ProviderID on Nodes. This is important, so that CAPI in the mgt-cluster knows which CAPI machine (in mgt-cluster) is which Node (in wl-cluster).
  • Creates LoadBalancers

The CCM calls the Hetzner APIs. To authenticate, it reads the credentials from a secret. This secret has to be in the workload cluster, when the CCM runs in the workload cluster. CAPH creates the secret and syncs the credentials specified in the management cluster to the workload cluster. In our default templates this secret is called hetzner . The upstream HCloud chart defaults to a secret called hcloud , so you need to override the secret references when installing the chart.

Important: CAPH and the CCM must both use the same ProviderID format for bare metal. Unfortunately (for historical reasons), there are two formats:

  • old: hcloud://bm-NNNN
  • new: hrobot://NNNN

The upstream HCloud CCM uses the new format.

If you use the new format, set the annotation capi.syself.com/use-hrobot-provider-id-for-baremetal to "true" on the HetznerCluster . Our default templates have this annotation set.

If CAPH and the CCM do not agree on the ProviderID format, then new nodes will not be able to join the cluster, because CAPI waits for the wrong ProviderID.

This only applies to new nodes. Once a node has a ProviderID, it will never change. Both CCMs support both formats when the ProviderID is already set.

This applies only to bare metal. HCloud nodes always use the format hcloud://NNNN .

Since we have already created secret in hetzner robot, hcloud and ssh-keys as secret in management cluster, we can create a workload cluster. Generate the manifest by using clusterctl generate :

shell
clusterctl generate cluster my-cluster --flavor hetzner-hcloud-control-planes > my-cluster.yaml

And apply it:

shell
kubectl apply -f my-cluster.yaml
console
$ kubectl apply -f my-cluster.yaml kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/my-cluster-md-0 created kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/my-cluster-md-1 created cluster.cluster.x-k8s.io/my-cluster created machinedeployment.cluster.x-k8s.io/my-cluster-md-0 created machinedeployment.cluster.x-k8s.io/my-cluster-md-1 created machinehealthcheck.cluster.x-k8s.io/my-cluster-control-plane-unhealthy-5m created machinehealthcheck.cluster.x-k8s.io/my-cluster-md-0-unhealthy-5m created machinehealthcheck.cluster.x-k8s.io/my-cluster-md-1-unhealthy-5m created kubeadmcontrolplane.controlplane.cluster.x-k8s.io/my-cluster-control-plane created hcloudmachinetemplate.infrastructure.cluster.x-k8s.io/my-cluster-control-plane created hcloudmachinetemplate.infrastructure.cluster.x-k8s.io/my-cluster-md-0 created hcloudremediationtemplate.infrastructure.cluster.x-k8s.io/control-plane-remediation-request created hcloudremediationtemplate.infrastructure.cluster.x-k8s.io/worker-remediation-request created hetznerbaremetalmachinetemplate.infrastructure.cluster.x-k8s.io/my-cluster-md-1 created hetznercluster.infrastructure.cluster.x-k8s.io/my-cluster created

Getting the kubeconfig of workload cluster

After a while, our first controlplane should be up and running. You can verify it using the output of kubectl get kcp followed by kubectl get machines

Once it's up and running, you can get the kubeconfig of the workload cluster using the following command:

shell
clusterctl get kubeconfig my-cluster > workload-kubeconfig chmod go-r workload-kubeconfig # required to avoid helm warning

Deploy Cluster Addons

note

This is important for the functioning of the cluster otherwise the cluster won't work.

Deploying the Hetzner Cloud Controller Manager

note

This requires a secret containing access credentials to both Hetzner Robot and HCloud.

For bare metal, use the Syself CCM until an upstream release includes PR fix robot name lookup after stale cache miss.

shell
helm repo add syself https://charts.syself.com helm repo update syself helm upgrade --install ccm syself/ccm-hetzner --version 2.0.6 \ --namespace kube-system \ --set-json 'extraEnvVars=[{"name":"HCLOUD_USE_HROBOT_PROVIDER_ID_FOR_BAREMETAL","value":"true"}]' \ --kubeconfig workload-kubeconfig

Be sure that the HetznerCluster has the annotation capi.syself.com/use-hrobot-provider-id-for-baremetal: "true" .

Installing CNI

For CNI, let's deploy cilium in the workload cluster that will facilitate the networking in the cluster.

console
$ helm install cilium cilium/cilium --kubeconfig workload-kubeconfig NAME: cilium LAST DEPLOYED: Thu Apr 4 21:11:13 2024 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: You have successfully installed Cilium with Hubble. For any further help, visit https://docs.cilium.io/en/v1.15/gettinghelp

Verifying the cluster

Now, the cluster should be up and you can verify it by running the following commands:

console
$ kubectl get clusters -A NAMESPACE NAME CLUSTERCLASS PHASE AGE VERSION default my-cluster Provisioned 10h $ kubectl get machines -A NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION default my-cluster-control-plane-6m6zf my-cluster my-cluster-control-plane-84hsn hcloud://45443706 Running 10h v1.35.4 default my-cluster-control-plane-m6frm my-cluster my-cluster-control-plane-hvl5d hcloud://45443651 Running 10h v1.35.4 default my-cluster-control-plane-qwsq6 my-cluster my-cluster-control-plane-ss9kc hcloud://45443746 Running 10h v1.35.4 default my-cluster-md-0-2xgj5-c5bhc my-cluster my-cluster-md-0-6xttr hcloud://45443694 Running 10h v1.35.4 default my-cluster-md-0-2xgj5-rbnbw my-cluster my-cluster-md-0-fdq9l hcloud://45443693 Running 10h v1.35.4 default my-cluster-md-0-2xgj5-tl2jr my-cluster my-cluster-md-0-59cgw hcloud://45443692 Running 10h v1.35.4 default my-cluster-md-1-cp2fd-7nld7 my-cluster bm-my-cluster-md-1-d7526 hcloud://bm-2317525 Running 9h v1.35.4 default my-cluster-md-1-cp2fd-n74sm my-cluster bm-my-cluster-md-1-l5dnr hcloud://bm-2105469 Running 10h v1.35.4

Please note that HCloud servers are prefixed with hcloud:// and bare-metal servers are prefixed with either hcloud://bm- or hrobot:// , depending on your ProviderID format configuration.