image-url-command

The hcloud spec.imageURLCommand field and the bare metal spec.installImage.imageURLCommand field can be used to execute a custom command to install the node image. This feature is also known as a "custom provisioner".

This provides you a flexible way to create nodes.

The script/binary will be copied into the rescue system and executed.

You need to enable two things:

• for hcloud: The HCloudMachine resource must set both spec.imageURL and spec.imageURLCommand (usually via a HCloudMachineTemplate)
• for baremetal: The HetznerBareMetalMachine must set spec.installImage.imageURLCommand, for example:

		spec:
  installImage:
    imageURLCommand: image-url-command-install-foo.sh
    image:
      url: oci://example.com/yourimage:v1
	

In bare metal custom-command mode, image.name and image.path must stay empty.

Example for hcloud:

		apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HCloudMachineTemplate
metadata:
  name: my-hcloud-template
spec:
  template:
    spec:
      type: cpx22
      imageURL: oci://example.com/yourimage:v1
      imageURLCommand: image-url-command-install-foo.sh
	

The command receives the following positional arguments:

1. imageURL — the OCI (or other) image URL
2. /root/bootstrap.data — path to the bootstrap data file written by CAPH
3. machine-name — name of the corresponding machine
4. root-devices — space-separated list of root device names (e.g. sda sdb)

Example:

		/root/image-url-command oci://example.com/yourimage:v1 /root/bootstrap.data my-md-bm-kh57r-5z2v8-zdfc9 'sda sdb'
	

The image format — whole-disk image, root-filesystem tarball, or anything else — is entirely your choice, as long as the imageURLCommand binary and the artifact at imageURL match each other. Both are user-configurable; you are responsible for keeping them in sync.

The command must be accessible by the controller pod below /shared. You can use an initContainer to copy the command to a shared emptyDir. For both hcloud and bare metal, the command field is only the basename of a command below /shared.

The env var OCI_REGISTRY_AUTH_TOKEN from the caph process will be set for the command, too.

By default, CAPH passes short device names (e.g. sda) as the last argument to the command. For bare metal machines you can set spec.installImage.deviceStringType to control this:

• "short" (or empty): passes the short device name, e.g. sda
• "wwn": passes the WWN from the rootDeviceHints, e.g. eui.00253885910c8cec

Example:

		spec:
  installImage:
    imageURLCommand: image-url-command-install-foo.sh
    deviceStringType: wwn
    image:
      url: oci://example.com/yourimage:v1
	

Using deviceStringType: wwn avoids fragile device-name lookups, because device names like sda can change across reboots while WWNs are stable identifiers. The deviceStringType field is not used for hcloud machines (hcloud VMs always boot from sda and disks have no WWN).

When multiple devices are configured (e.g. RAID via rootDeviceHints.raid.wwn), all device strings are passed as a single space-separated $4 argument. Scripts should split on whitespace.

The command must end with the last line on stdout containing IMAGE_URL_DONE. Otherwise the execution is considered to have failed.

Implementation detail: CAPH executes the command in the rescue system via ssh and nohup. Stdout and stderr are redirected to a file. CAPH continuously connects to the rescue system to see if the process is still running.

The controller uses url.ParseRequestURI (Go function) to validate the imageURL.

The full output (stdout and stderr) of the script is written to the controller log. On failure or timeout CAPH also creates a Warning event, but with a short message only, never the full output. If the script takes longer than 20 minutes, the controller cancels the provisioning.

Steps#

• CAPH copies the binary to the rescue system.
• The executable gets executed, the PID gets written to a file. Stdout and Stderr get redirected into a file.
• CAPH continuously reads /root/output.json (if present). The provisioner can write a message into the file which will be in the corresponding condition, so that users can see the current state of the process. This is optional.
• When the process has terminated, CAPH checks if IMAGE_URL_DONE is in the last line of the output. If not, the process is considered to have failed. The machine gets deprovisioned. CAPH creates a Warning event with a short message. The full output of the process and the output.json content (if it exists) are written to the controller log.
• When IMAGE_URL_DONE was found, the process is considered to have succeeded. The output of the process and the output.json content (if it exists) are written to the controller log.

output.json (optional)#

The command may write /root/output.json at any point during execution. The file is an option to give information from the node and to write it to the controller log. It's not about success or not.

CAPH reads only the message field from this file to update the provisioning condition on the machine (HCloudMachine or HetznerBareMetalHost). The message field is forwarded verbatim into the condition message. CAPH reads the file, and updates the condition (if needed) every ten seconds.

Outcome summary#

CAPH waits until the provisioning process in the rescue system has terminated. Then the captured stdout gets examined. If it does not contain IMAGE_URL_DONE, then the process has failed. Optionally output.json can be created by the process. The content of output.json does not change the final result (succeeded or failed).

Implemented in handleBootStateRunningImageCommand (hcloud) and actionImageInstallingImageURLCommand (baremetal).

Minimal example:

		{"message": "Downloading node image..."}
	

Any other fields in the JSON are ignored by CAPH but are written as-is to the controller log (see below). You can use them for your own structured debugging output.

Controller log on completion#

When the command finishes (success or failure), CAPH writes the full JSON content of the file to the controller log at key outputJSON. It is not exposed as a Kubernetes event, because the output can contain information that should not be shown to the cluster user.

Measured durations for hcloud#

The duration of the state RunningImageCommand depends heavily on your script.