Secrets as of now are hardcoded given we are using a flavor which is essentially a template. If you want to use your own naming convention for secrets then you'll have to update the templates. Please make sure that you pay attention to the sshkey name.
In the long run we (Syself) want to switch from our fork to the upstream CCM.
The CCM is the "Cloud Controller" which runs in the workload-cluster. The most important tasks of the CCM are:
Set ProviderID on Nodes. This is important, so that CAPI in the mgt-cluster knows which CAPI machine (in mgt-cluster) is which Node (in wl-cluster).
Creates LoadBalancers
The CCM calls the Hetzner APIs. To authenticate, it reads the credentials from a secret. This secret has to be in the workload cluster, when the CCM runs in the workload cluster. CAPH creates the secret and syncs the credentials specified in the management cluster to the workload cluster. In our default templates this secret is called hetzner . The upstream HCloud chart defaults to a secret called hcloud , so you need to override the secret references when installing the chart.
Important: CAPH and the CCM must both use the same ProviderID format for bare metal. Unfortunately (for historical reasons), there are two formats:
old: hcloud://bm-NNNN
new: hrobot://NNNN
The upstream HCloud CCM uses the new format.
If you use the new format, set the annotation capi.syself.com/use-hrobot-provider-id-for-baremetal to "true" on the HetznerCluster . Our default templates have this annotation set.
If CAPH and the CCM do not agree on the ProviderID format, then new nodes will not be able to join the cluster, because CAPI waits for the wrong ProviderID.
This only applies to new nodes. Once a node has a ProviderID, it will never change. Both CCMs support both formats when the ProviderID is already set.
This applies only to bare metal. HCloud nodes always use the format hcloud://NNNN .
Since we have already created secret in hetzner robot, hcloud and ssh-keys as secret in management cluster, we can create a workload cluster. Generate the manifest by using clusterctl generate :
$ kubectl apply -f my-cluster.yaml
kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/my-cluster-md-0 created
kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/my-cluster-md-1 created
cluster.cluster.x-k8s.io/my-cluster created
machinedeployment.cluster.x-k8s.io/my-cluster-md-0 created
machinedeployment.cluster.x-k8s.io/my-cluster-md-1 created
machinehealthcheck.cluster.x-k8s.io/my-cluster-control-plane-unhealthy-5m created
machinehealthcheck.cluster.x-k8s.io/my-cluster-md-0-unhealthy-5m created
machinehealthcheck.cluster.x-k8s.io/my-cluster-md-1-unhealthy-5m created
kubeadmcontrolplane.controlplane.cluster.x-k8s.io/my-cluster-control-plane created
hcloudmachinetemplate.infrastructure.cluster.x-k8s.io/my-cluster-control-plane created
hcloudmachinetemplate.infrastructure.cluster.x-k8s.io/my-cluster-md-0 created
hcloudremediationtemplate.infrastructure.cluster.x-k8s.io/control-plane-remediation-request created
hcloudremediationtemplate.infrastructure.cluster.x-k8s.io/worker-remediation-request created
hetznerbaremetalmachinetemplate.infrastructure.cluster.x-k8s.io/my-cluster-md-1 created
hetznercluster.infrastructure.cluster.x-k8s.io/my-cluster created
For CNI, let's deploy cilium in the workload cluster that will facilitate the networking in the cluster.
console
$ helm install cilium cilium/cilium --kubeconfig workload-kubeconfig
NAME: cilium
LAST DEPLOYED: Thu Apr 4 21:11:13 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
You have successfully installed Cilium with Hubble.
For any further help, visit https://docs.cilium.io/en/v1.15/gettinghelp
Now, the cluster should be up and you can verify it by running the following commands:
console
$ kubectl get clusters -A
NAMESPACE NAME CLUSTERCLASS PHASE AGE VERSION
default my-cluster Provisioned 10h
$ kubectl get machines -A
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
default my-cluster-control-plane-6m6zf my-cluster my-cluster-control-plane-84hsn hcloud://45443706 Running 10h v1.35.4
default my-cluster-control-plane-m6frm my-cluster my-cluster-control-plane-hvl5d hcloud://45443651 Running 10h v1.35.4
default my-cluster-control-plane-qwsq6 my-cluster my-cluster-control-plane-ss9kc hcloud://45443746 Running 10h v1.35.4
default my-cluster-md-0-2xgj5-c5bhc my-cluster my-cluster-md-0-6xttr hcloud://45443694 Running 10h v1.35.4
default my-cluster-md-0-2xgj5-rbnbw my-cluster my-cluster-md-0-fdq9l hcloud://45443693 Running 10h v1.35.4
default my-cluster-md-0-2xgj5-tl2jr my-cluster my-cluster-md-0-59cgw hcloud://45443692 Running 10h v1.35.4
default my-cluster-md-1-cp2fd-7nld7 my-cluster bm-my-cluster-md-1-d7526 hcloud://bm-2317525 Running 9h v1.35.4
default my-cluster-md-1-cp2fd-n74sm my-cluster bm-my-cluster-md-1-l5dnr hcloud://bm-2105469 Running 10h v1.35.4
Please note that HCloud servers are prefixed with hcloud:// and bare-metal servers are prefixed with either hcloud://bm- or hrobot:// , depending on your ProviderID format configuration.