Retries and outlier detection are the Istio settings that absorb an unreliable network. A retry saves the request that fails immediately by sending it again. Outlier detection ejects a pod that keeps failing, so callers stop reaching it and traffic moves to the healthy pods. Use both: they solve different problems. This page assumes Istio is installed in sidecar mode; see [Handle unreliable networks with Istio](/docs/hetzner/apalla/network/mesh/handle-unreliable-networks-with-istio) for the model, and [Choose a service mesh](/docs/hetzner/apalla/network/mesh/choose-a-service-mesh) for whether you need a mesh in the first place.

## Configure retries per hop

When a request fails on a hop, Istio resends it. The retry can land on a different pod and take a different path through the network, so the user gets a slightly slower response instead of an error. Add a `VirtualService` on the caller side of each service-to-service hop:

```yaml title="virtualservice.yaml"
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: backend
spec:
  hosts: [backend]
  http:
    - route:
        - destination: {host: backend}
      timeout: 3s
      retries:
        attempts: 3
        perTryTimeout: 800ms
        retryOn: connect-failure,refused-stream,reset-before-request
```

- **`retryOn: connect-failure,refused-stream,reset-before-request`** means "the request never reached the application", the only cases that are safe to repeat. Use `reset-before-request`, not plain `reset`: `reset` also fires when the connection drops after the request was delivered, so the server may have already acted on it, while `reset-before-request` only retries a reset that happened before the request was sent. On a route that writes (charging a card, creating an order), keep it to exactly these causes so a partial request is never replayed.
- **`perTryTimeout` must be smaller than `timeout`.** If they are equal, there is no time left for a second attempt and retries silently never fire.
- **Keep `attempts` low.** Every retry multiplies load on the downstream service, so too many retries can make a struggling service worse.

## Configure outlier detection per service

Outlier detection watches real request outcomes. When one pod keeps returning errors, Istio stops sending it traffic for a while, then gradually tries again. Traffic shifts to the healthy pods on its own. Add a `DestinationRule` per service:

```yaml title="destinationrule.yaml"
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: backend
spec:
  host: backend
  trafficPolicy:
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 10s
      baseEjectionTime: 30s
      maxEjectionPercent: 50
```

- **`consecutive5xxErrors` and `interval`** decide how fast a bad pod is ejected: five 5xx responses inside a 10-second window here.
- **`baseEjectionTime`** is how long a pod stays out before Istio tries it again, growing each time it is re-ejected.
- **`maxEjectionPercent: 50`** stops Istio from removing more than half the pool at once, so the service keeps a quorum serving even when several pods struggle.

```mermaid
flowchart LR
  C["Caller"] --> P1["Pod 1 healthy"]
  C --> P2["Pod 2 healthy"]
  C -. ejected after 5 errors .-> P3["Pod 3 failing"]
```

The caller keeps reaching the healthy pods and stops sending to the failing one until `baseEjectionTime` passes and Istio tries it again.

Both settings need more than one replica. With a single replica there is no healthy pod to retry against or shift traffic to, so run at least two. Together, retries save the request that fails immediately, and outlier detection steers traffic away from a pod that is consistently unhealthy.

## Where to go next

- [Run a production-ready workload](/docs/hetzner/apalla/workloads/production/run-a-production-ready-workload)