Kubernetes Error: ImagePullBackOff — Cannot Pull Image

Events:
  Type     Reason   Age                From     Message
  ----     ------   ----               ----     -------
  Normal   Pulling  2m (x4 over 5m)    kubelet  Pulling image "myapp:v2.3.1"
  Warning  Failed   2m (x4 over 5m)    kubelet  Failed to pull image "myapp:v2.3.1": rpc error: code = Unknown desc = Error response from daemon: manifest for myapp:v2.3.1 not found: manifest unknown
  Warning  Failed   2m (x4 over 5m)    kubelet  Error: ErrImagePull
  Normal   BackOff  30s (x10 over 5m)  kubelet  Back-off pulling image "myapp:v2.3.1"
  Warning  Failed   30s (x10 over 5m)  kubelet  Error: ImagePullBackOff

ImagePullBackOff is the kubelet telling you it tried to pull your image, failed, retried, failed again, and is now waiting before the next attempt. It’s not actually the error — it’s the back-off state. The real error is one of manifest unknown, unauthorized, no matching manifest, toomanyrequests, or a network failure, and you find it in kubectl describe pod’s Events list.

The five causes are almost evenly distributed in production: typos, expired registry credentials, architecture mismatches, Docker Hub rate-limits, and egress problems. Each has a single, mechanical fix once you know which one applies — so spend 30 seconds reading Events before you change anything.

Why this happens

• Wrong image name or tag. Typo in the image reference, tag that doesn't exist (someone deleted `:latest`, or you referenced a tag from another repo), or the registry returned a manifest-unknown response. `kubectl describe pod` shows the exact registry error.
• Private registry auth missing or wrong. The image is in a private registry (ECR, GCR, GHCR, Docker Hub private repos) and the pod has no `imagePullSecrets` or the secret's docker-config is wrong/expired. ECR tokens expire every 12 hours unless you're using IRSA/Workload Identity.
• Architecture mismatch (arm64 vs amd64). Image was built for `linux/amd64` but the node is `linux/arm64` (Graviton, Apple Silicon-based dev cluster) or vice versa. The pull succeeds, then fails with 'no matching manifest for linux/arm64 in the manifest list entries.' Common with single-arch images and modern ARM nodes.
• Registry rate-limit (Docker Hub anonymous). Docker Hub limits anonymous pulls to 100/6h per IP. Busy clusters share an egress IP and get 429s as `toomanyrequests: You have reached your pull rate limit`. Fix is to authenticate (free tier gets 200/6h, paid more) or mirror images to your own registry.
• Network egress blocked or DNS broken. Pod's node can't reach the registry — VPC egress denied, NAT gateway saturated, or CoreDNS can't resolve the registry hostname. Less common than the above but produces an ImagePullBackOff with a TCP/DNS error in the Events.

How to fix it

Fixes are ordered by likelihood. Start with the first one that matches your context.

# Find the failing pod:
kubectl get pods -A | grep -E 'ImagePullBackOff|ErrImagePull'

# Read the events:
kubectl describe pod <pod-name> -n <namespace> | grep -A 30 Events:

# Or filter events directly:
kubectl get events -n <namespace> --sort-by=.lastTimestamp \
  | grep <pod-name>

# Try the pull as the cluster would see it:
docker pull myregistry.example/team/app:v2.3.1

# Inspect manifest without pulling layers:
crane manifest myregistry.example/team/app:v2.3.1

# Check what platforms a multi-arch manifest supports:
docker buildx imagetools inspect myregistry.example/team/app:v2.3.1

# 1. Create the secret (one-time, repeat per namespace):
# kubectl create secret docker-registry regcred \
#   --docker-server=myregistry.example \
#   --docker-username=<user> \
#   --docker-password=<pat-or-password> \
#   -n production

# 2. Reference it in the pod spec or service account:
apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-sa
  namespace: production
imagePullSecrets:
  - name: regcred
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app
  namespace: production
spec:
  template:
    spec:
      serviceAccountName: app-sa
      containers:
        - name: app
          image: myregistry.example/team/app:v2.3.1

# One-time setup:
docker buildx create --use --name multiarch
docker buildx inspect --bootstrap

# Build + push amd64 and arm64 in one manifest list:
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t myregistry.example/team/app:v2.3.1 \
  --push .

Detection and monitoring in production

Track ImagePullBackOff at cluster level — Prometheus + kube-state-metrics exposes `kube_pod_status_reason{reason="ImagePullBackOff"}`. Alert when any pod is in this state for >5 minutes; for production deployments, ImagePullBackOff after a deploy means the rollout will time out and traffic stays on the old version. Also alert on ECR token expiry (use a CronJob to refresh, or migrate to IRSA).