A Deployment is built to run indefinitely; a Job is built to finish. It starts a pod, performs one piece of work (a migration, a nightly rollup), reports success or failure, and stops. The complication is where it runs. Nodes on this platform are recycled: by a Kubernetes upgrade you roll out, by autoscaling, or by a repair. Any Job running longer than a few minutes can therefore be evicted mid-run. Design for that first, ahead of parallelism or retries.

## Surviving node drains

Kubernetes does not checkpoint a Job pod for you. When a drain evicts it, the Job schedules a fresh pod elsewhere and the work restarts **from the beginning**; nothing is resumed. It must tolerate that: make it idempotent (safe to run twice from scratch, with no double-counting), or checkpoint progress externally and resume from the last mark. Treat an interrupted run as normal, not as the failure case.

Eviction is not instant. The pod receives a grace window to exit before the hard kill: `nodeDrainTimeoutSeconds`, a per-pool default of 180 seconds. A Job that needs longer to flush a partial write can [raise the timeout on its pool](/docs/hetzner/apalla/workloads/production/run-a-production-ready-workload) in the `Cluster` topology.

> [!WARNING]
> Trapping `SIGTERM` for a graceful shutdown does not provide unlimited time. If the wind-down runs past the pool's `nodeDrainTimeoutSeconds`, the drain hard-kills the pod mid-cleanup; 180 seconds is the ceiling until you raise it.

## A basic Job

```yaml title="job.yaml"
apiVersion: batch/v1
kind: Job
metadata:
  name: import
spec:
  backoffLimit: 4 # retry the work up to 4 times before failing the Job
  template:
    spec:
      restartPolicy: Never # fresh pod per attempt; keeps failed pods' logs
      containers:
        - name: import
          image: your/importer:tag
          command: ["./run-import"] # must be safe to re-run from scratch
```

To bound wall-clock time with `activeDeadlineSeconds`, or have finished Jobs delete themselves, see [Control retries and clean up finished jobs](/docs/hetzner/apalla/workloads/batch/backoff-and-ttl-cleanup).

## Parallelism and completions

Two fields scale one pod into many. `completions` is how many successful runs the Job needs; `parallelism` is how many run at once. Set `completions: 10` and `parallelism: 3`, and ten work items run three at a time. When each pod needs to know which slice it owns, add `completionMode: Indexed`: every pod reads a `JOB_COMPLETION_INDEX` (0 to 9 here) and selects its own partition.

## restartPolicy: Never or OnFailure

A Job pod must use `Never` or `OnFailure`, never `Always`. The choice depends on logs. `Never` starts a fresh pod per attempt and leaves the failed ones in place, so their logs stay readable, which is usually what you want while debugging. `OnFailure` restarts the container in place: fewer objects, but the failed container's logs are lost.

## Reading the result

Applying a Job says nothing about whether it ran. Its `COMPLETIONS` column does: `kubectl get job import` reads `1/1` once the work has finished and `0/1` while it is still running, or still failing. Those two states look identical. The distinguishing signal is the pods. With `restartPolicy: Never`, each attempt is a fresh pod, so a failing Job leaves a trail: `kubectl get pods -l job-name=import` shows failed pods stacking up while completions stay at `0/1`, whereas a slow Job shows a single pod still `Running`. Pull the logs from a failed pod for the reason, and check `backoffLimit` for how many attempts remain. (With `OnFailure` the signal is instead the restart count climbing on one pod, seen in the pod's `RESTARTS` column; `kubectl get job` has no such column.)