Using local storage in bare metal

Local storage keeps data on the node, so it is faster than network storage, and on bare-metal machines it is cheap too. Syself Autopilot handles the lifecycle: data persists through cluster updates and machine re-provisioning, which suits storage-intensive workloads such as databases. This one-time setup configures your cluster and machines to use local storage with TopoLVM.

1. Deploy cert-manager

You need to have cert-manager version v1.7.0 or higher installed on your cluster as a dependency of TopoLVM. Install it with the following command:

		$ helm repo add jetstack https://charts.jetstack.io
$ helm repo update
$ helm template --namespace=kube-system cert-manager jetstack/cert-manager --set installCRDs=true | kubectl apply -n kube-system -f -
	

You can follow the official guide⁠ on installing helm if you don't have it.

2. Deploy TopoLVM

We use the TopoLVM CSI driver for local storage on bare-metal. You can follow the steps below to deploy it to your workload cluster.

1. Add the Syself helm repository:

   		$ helm repo add syself https://charts.syself.com
   $ helm repo update
   	

2. Template the TopoLVM chart and apply it to the cluster:

   Warning

   Do not use helm install. This chart will be added as a base feature in later versions of Syself Autopilot, so stick to the installation steps shown here to guarantee compatibility. Use the command below to install it with helm template and kubectl apply to the kube-system namespace.

   		$ helm template --namespace=kube-system csi-local syself/topolvm | kubectl apply -n kube-system -f -
   	

   Now the storage space in your bare-metal server is exposed to your cluster via the local-nvme Storage Class:

   		$ kubectl get storageclasses
   NAME                  PROVISIONER         RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION
   local-hdd             topolvm.io          Retain          WaitForFirstConsumer   true
   local-nvme            topolvm.io          Retain          WaitForFirstConsumer   true
   local-ssd             topolvm.io          Retain          WaitForFirstConsumer   true
   standard (default)    csi.hetzner.cloud   Retain          WaitForFirstConsumer   true
   	

3. Configure your servers

In this step, you'll define the physical volumes and volume groups in your disks to be used by TopoLVM.

1. Access your server via ssh:

   		$ ssh -i path-to-your/ssh-key -p 100 root@<machine-ip>
   	

2. List your disks with lsblk:

   		$ lsblk
   NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
   nvme1n1     259:0    0 476.9G  0 disk
   nvme0n1     259:1    0 476.9G  0 disk
   |-nvme0n1p1 259:2    0   512M  0 part /boot/efi
   |-nvme0n1p2 259:3    0     1G  0 part /boot
   `-nvme0n1p3 259:4    0 475.4G  0 part /
   	

3. Identify if the disk(s) you want to use is an HDD, SATA SSD, or NVMe SSD.

   Warning

   Don't use your OS disk (nvme0n1 in the above output), as this can lead to data loss.

   Tip

   To identify the type of disk you have, you can look at the first column NAME and third column RM of the lsblk output.

   • An NVMe disk will have nvme at the beginning of its name, otherwise:
   • A SATA SSD disk will have the value 0 in the RM column.
   • A HDD disk will have the value 1 in the RM column.

4. Create a physical volume (point to every disk in your server where you want to store data) with pvcreate /dev/[disk-name]. For example:

   		$ pvcreate /dev/nvme1n1
   	

5. Map the disks to the appropriate volume group type with vgcreate vg-[type] /dev/[disk-name] /dev/[other-disk]. For example:

   		$ vgcreate vg-nvme /dev/nvme1n1
   	

   If you missed a disk and want to add it later, extend the volume group with vgextend vg-[type] /dev/[new-disk]. For example:

   		$ vgextend vg-nvme /dev/nvme2n1
   	

   Available volume group types

   - For NVMe disks you use: `vg-nvme` - For SATA SSD disks you use: `vg-ssd` - For HDD disks you use: `vg-hdd`

6. Repeat the previous steps for every disk you want to use.

7. After adding all disks to their respective volume groups, create a thin provisioned logical volume for each volume group. This step is done once per volume group, not per disk: Create a thin provisioned logical volume with lvcreate --thinpool pool-[type] --extents 100%FREE vg-[type].

		$ lvcreate --thinpool pool-nvme --extents 100%FREE vg-nvme
	

		$ lvcreate --thinpool pool-ssd --extents 100%FREE vg-ssd
	

		$ lvcreate --thinpool pool-hdd --extents 100%FREE vg-hdd
	

4. Use it!

You can use the newly created Storage Classes in the same way you would use any other.

If you want to test your new setup:

1. Create a pv-claim.yaml file with the following content:

   		apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: pv-claim
   spec:
     storageClassName: local-nvme
     accessModes:
       - ReadWriteOnce
     resources:
       requests:
         storage: 1Gi
   	

2. And apply it with kubectl apply -f pv-claim.yaml.

3. Create a pod.yaml file with the following content:

   		apiVersion: v1
   kind: Pod
   metadata:
     name: pv-pod
   spec:
     volumes:
       - name: pv-storage
         persistentVolumeClaim:
           claimName: pv-claim
     containers:
       - name: pv-container
         image: nginx
         ports:
           - containerPort: 80
             name: http-server
         volumeMounts:
           - mountPath: /usr/share/nginx/html
             name: pv-storage
   	

4. And apply it with kubectl apply -f pod.yaml.

   Now, all the data stored in the container under /usr/share/nginx/html will be in /mnt/data on your machine.

   Note

   In this example, we used the local-nvme class, but you can also use local-hdd and local-ssd too if your servers have disks of those types attached.

5. Create a test file in your pod:

   		$ kubectl exec -it pv-pod -- /bin/sh
   $ echo 'Hi from Kubernetes to bare metal!' > /usr/share/nginx/html/hi.txt
   	

6. Now delete the pod, so we are sure the storage is persistent, and not ephemeral: kubectl delete pod pv-pod

7. Apply the pod again with kubectl apply -f pod.yaml. Show the content of the hi.txt file:

   		$ cat /mnt/data/hi.txt
   Hi from Kubernetes to bare metal!