Restrict SSH access

SSH on a Syself Linux node is key-only on port 100, and open to every source address by default. It comes up open when you create a cluster , and you adjust it afterwards: narrow it to a bastion host or block it completely with the steps below. It stays on by default because it is the only out-of-band way to reach a node, and that access matters when a node needs hands-on support.

If you restrict or block SSH, know what you give up: without SSH there is no direct way into a node. Support and debugging are then limited to what the cluster API and the log collector can reach.

Prerequisites#

• kubectl access to the workload cluster

How SSH is configured on a node#

• Port: 100, not 22.
• Authentication: key-only. PasswordAuthentication no and ChallengeResponseAuthentication no turn every password path off, and root logs in with a key only (PermitRootLogin prohibit-password). There is no password to guess, so a scanner that finds port 100 has nothing to brute-force.
• Accounts: AllowUsers root. root is the only login account. Any other username is refused before authentication.
• Keys: sshd reads authorized keys from one file, /etc/ssh/authorized_keys.d/root (AuthorizedKeysFile), written at provisioning time. It never reads ~/.ssh/authorized_keys.
• Limits: a client gets three key attempts (MaxAuthTries 3) inside a 20 second window (LoginGraceTime 20), then the connection closes.
• Firewall: Cilium's host firewall ships a built-in policy that allows port 100 from all addresses.

Why the open default is still safe#

An attacker who reaches port 100 needs the private half of the configured key. Everything that would let them work around that requirement is sealed:

• The sshd config sits on the verity-sealed base image. The kernel verifies every read of it, so nobody can switch password login on, raise the attempt limit, or repoint AuthorizedKeysFile on a running node. Weakening sshd takes a new OS image, not a file edit.
• The authorized key file sits on the sealed per-node configuration layer, also kernel-verified. Adding a key at runtime is a write to a read-only sealed layer: it fails, and the attempt lands in the audit log (the audit rules watch /etc/ssh/authorized_keys.d/).
• Root sessions leave an audit trail they cannot erase. PAM tags each login with a login id, and the audit rules record every command the session runs. The rules are baked into the sealed image and lock at boot, so even root cannot unload them, and deleting or renaming the audit log is itself recorded. See Retrieve audit logs .

One risk remains: sshd itself. A pre-authentication vulnerability in OpenSSH would hit the daemon as it ships. The sealed config keeps sshd from being weakened at runtime; it does not make the daemon invulnerable. Until a fixed image rolls out, the firewall is the layer you control: restricting port 100 to your bastion host shrinks who can talk to the daemon at all.

Syself manages this allow policy as a cluster component. If you edit it by hand, the cluster-stack operator resets it back on the next reconcile. Add your own CiliumClusterwideNetworkPolicy with an ingressDeny rule instead. In Cilium, a deny always overrides any allow, regardless of where the allow comes from. Your deny rule overrides the built-in allow without touching it.

Block SSH entirely#

To turn SSH off on every node, apply this CiliumClusterwideNetworkPolicy to the workload cluster. It denies TCP port 100 from every source, so no client can open an SSH connection.

		apiVersion: "cilium.io/v2"
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: "ssh-lockdown"
spec:
  nodeSelector:
    matchLabels: {} # applies to all nodes
  ingressDeny:
    - fromCIDRSet:
        - cidr: "0.0.0.0/0" # every source
      toPorts:
        - ports:
            - port: "100" # SSH runs on 100, not 22
              protocol: TCP
	

This blocks SSH at the firewall. The SSH daemon keeps running on the node; Cilium drops the packets before they reach it. There is no switch to stop the daemon itself, so "off" means "no connection gets through", not "the service is gone".

With a full block in place, use the Hetzner rescue system to reach a node in an emergency. It requires Hetzner console access. See Emergency node access . To turn SSH back on, delete the deny policy (see Restore SSH access).

Restrict SSH to a bastion host#

To keep SSH but only from a single jump host, deny every source except your bastion host's address. Apply this to the workload cluster. Replace 203.0.113.10/32 with the public IP of the bastion host your operators connect through.

		apiVersion: "cilium.io/v2"
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: "ssh-lockdown"
spec:
  nodeSelector:
    matchLabels: {} # applies to all nodes
  ingressDeny:
    - fromCIDRSet:
        - cidr: "0.0.0.0/0"
          except:
            - "203.0.113.10/32" # your bastion host
      toPorts:
        - ports:
            - port: "100"
              protocol: TCP
	

The except list carves out an exception in the deny rule: every source address that is NOT in except is denied. Only the bastion host falls through to the built-in allow and can still connect.

You can list more than one bastion host:

		except:
  - "203.0.113.10/32" # primary bastion host
  - "198.51.100.20/32" # secondary bastion host
	

Confirm it worked#

Run both checks: the rule loaded, and SSH is blocked.

First, check that Cilium loaded the deny rule. Run this inside the cilium pod on any node:

		$ kubectl -n kube-system exec -it ds/cilium -- cilium policy get | grep -A5 ssh-lockdown
	

You should see your ingressDeny rule in the output. If nothing prints, the policy did not load. Run kubectl get ciliumclusterwidenetworkpolicy ssh-lockdown and read its status.

The first check only proves the rule loaded, not that a client is blocked. So test the effect from a machine that should not reach SSH. Get a node's public IP:

		$ kubectl get nodes -o wide
	

Try to open port 100 against that IP from the blocked machine:

		$ nc -zv -w5 <node-external-ip> 100
	

Read the result this way:

• Times out (no response for 5 seconds): the packet was dropped. SSH is blocked from here. This is what a full block, or a source outside your except range, should do.
• Connects right away (succeeded / open): SSH is still reachable. The policy is not matching. Check that the port is "100" and that nodeSelector covers this node.
• Connection refused: you are hitting the wrong host or port, not the firewall. A refused connection is not a drop.

If you restricted to a bastion host, run the same test from the bastion too. From the bastion it should connect; from any other address it should time out.

Restore SSH access#

Delete the deny policy:

		$ kubectl delete ciliumclusterwidenetworkpolicy ssh-lockdown
	

The built-in allow policy is still in place, so port 100 is reachable from every source again right away. Nothing else to undo. Use this if you locked yourself out with a full block and need SSH back.

Tradeoffs#

For most deployments, restricting to a bastion host is the right balance. A full block makes sense when SSH is not part of normal operations and you have a reliable Hetzner console workflow for emergencies.

Control which SSH keys can log in (HCloud nodes)#

The SSH key attached to HCloud nodes is the one named in the hcloud-ssh-key-name field of the hetzner Secret that the cluster references. The value is the name of an SSH key in your Hetzner Cloud project, and that key must already be registered there. To change which key is authorized, update the field to a different key name and reprovision the affected nodes. See Prepare your Hetzner account for how the Secret is created.

For bare-metal nodes, SSH keys come from the robot-ssh Secret referenced in the cluster template. That Secret is not a topology variable and is not covered here.