Using local storage in bare metal

Introduction#

Local storage offers faster access speeds than alternatives because data is stored directly on the node. And you can combine that with the benefits of bare-metal machines to make your storage surprisingly cost-effective.

Normally, this would mean a complex setup with higher maintenance costs. But with Syself Autopilot, we take that burden off your shoulders by simplifying every step. We enable you to persist data through cluster updates or even when you need to re-provision the machines, making it ideal for storage-intensive workloads such as databases.

This guide will walk you through the process of configuring your cluster and machines to use local storage with TopoLVM, a one-time process that allows you to efficiently use storage attached directly to your servers and rely on Autopilot for lifecycle automation.

1. Deploy cert-manager#

You need to have cert-manager version v1.7.0 or higher installed on your cluster as a dependency of TopoLVM. If you don't have it installed already, follow the steps in the cert-manager documentation⁠ and install it.

2. Deploy TopoLVM#

We use the TopoLVM CSI driver for local storage on bare-metal. You can follow the steps below to deploy it to your workload cluster.

1. Add the Syself helm repository:

   		$ helm repo add syself https://charts.syself.com
   $ helm repo update
   	

2. Template the TopoLVM chart and apply it to the cluster:

   		$ helm template --namespace=kube-system csi-local syself/topolvm | kubectl apply -n kube-system -f -
   	

   Now the storage space in your bare-metal server is exposed to your cluster via the local-nvme Storage Class:

   		$ kubectl get storageclasses
   NAME                  PROVISIONER         RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION
   local-hdd             topolvm.io          Retain          WaitForFirstConsumer   true
   local-nvme            topolvm.io          Retain          WaitForFirstConsumer   true
   local-ssd             topolvm.io          Retain          WaitForFirstConsumer   true
   standard (default)    csi.hetzner.cloud   Retain          WaitForFirstConsumer   true
   	

3. Configure your servers#

In this step, you'll define the physical volumes and volume groups in your disks to be used by TopoLVM.

1. Access your server via SSH:

   		$ ssh -i path-to-your/ssh-key -p 100 root@<machine-ip>
   	

   For detailed information on accessing your servers, visit this page⁠.

2. List your disks with lsblk:

   		$ lsblk
   NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
   nvme1n1     259:0    0 476.9G  0 disk
   nvme0n1     259:1    0 476.9G  0 disk
   |-nvme0n1p1 259:2    0   512M  0 part /boot/efi
   |-nvme0n1p2 259:3    0     1G  0 part /boot
   `-nvme0n1p3 259:4    0 475.4G  0 part /
   	

   Warning

   Don't use your OS disk (nvme0n1 in the above output), as this can lead to data loss.

   On your server, the OS disk might have a different name. You can identify the OS disk by the presence of the / and /boot mountpoints.

3. Identify if the disk(s) you want to use is an HDD, SATA SSD, or NVMe SSD.

   Tip

   To identify the type of disk you have, you can look at the first column NAME and third column RM of the lsblk output.

   • An NVMe disk will have nvme at the beginning of its name, otherwise:
   • A SATA SSD disk will have the value 0 in the RM column.
   • A HDD disk will have the value 1 in the RM column.

4. Create a physical volume (point to every disk in your server where you want to store data) with pvcreate /dev/[disk-name]. For example:

   		$ pvcreate /dev/nvme1n1
   	

5. Map the disks to the appropriate volume group type with vgcreate vg-[type] /dev/[disk-name] /dev/[other-disk]. For example:

   		$ vgcreate vg-nvme /dev/nvme1n1
   	

   If you missed a disk and want to add it later, extend the volume group with vgextend vg-[type] /dev/[new-disk]. For example:

   		$ vgextend vg-nvme /dev/nvme2n1
   	

   Available volume group types

   - For NVMe disks you use: `vg-nvme` - For SATA SSD disks you use: `vg-ssd` - For HDD disks you use: `vg-hdd`

6. Repeat the previous steps for every disk you want to use.

7. After adding all disks to their respective volume groups, create a thin provisioned logical volume for each volume group. This step is done once per volume group, not per disk: Create a thin provisioned logical volume with lvcreate --thinpool pool-[type] --extents 100%FREE vg-[type].

		$ lvcreate --thinpool pool-nvme --extents 100%FREE vg-nvme
	

		$ lvcreate --thinpool pool-ssd --extents 100%FREE vg-ssd
	

		$ lvcreate --thinpool pool-hdd --extents 100%FREE vg-hdd
	

4. Use it!#

You can use the newly created Storage Classes in the same way you would use any other.

Local storage can be up to 100 times faster than storage provided over the network. However, it provides no redundancy out of the box.

Some services handle replication and backups on their own, such as database operators. In these cases, local storage is a great option. For other services, it is advised to setup replication and disaster recovery.

If you want to test your new setup:

1. Create a pv-claim.yaml file with the following content:

   		apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: pv-claim
   spec:
     storageClassName: local-nvme
     accessModes:
       - ReadWriteOnce
     resources:
       requests:
         storage: 1Gi
   	

2. And apply it with kubectl apply -f pv-claim.yaml.

3. Create a pod.yaml file with the following content:

   		apiVersion: v1
   kind: Pod
   metadata:
     name: pv-pod
   spec:
     volumes:
       - name: pv-storage
         persistentVolumeClaim:
           claimName: pv-claim
     containers:
       - name: pv-container
         image: nginx
         ports:
           - containerPort: 80
             name: http-server
         volumeMounts:
           - mountPath: /usr/share/nginx/html
             name: pv-storage
   	

4. And apply it with kubectl apply -f pod.yaml.

   Now, all the data stored in the container under /usr/share/nginx/html will be in /mnt/data on your machine.

   Note

   In this example, we used the local-nvme class, but you can also use local-hdd and local-ssd too if your servers have disks of those types attached.

5. Create a test file in your pod:

   		$ kubectl exec -it pv-pod -- /bin/sh
   $ echo 'Hi from Kubernetes to bare metal!' > /usr/share/nginx/html/hi.txt
   	

6. Now delete the pod, so we are sure the storage is persistent, and not ephemeral: kubectl delete pod pv-pod

7. Apply the pod again with kubectl apply -f pod.yaml. Show the content of the hi.txt file:

   		$ cat /mnt/data/hi.txt
   Hi from Kubernetes to bare metal!