Kubernetes Troubleshooting Checklist: Diagnose Pods, Services, Ingress, and Deployments

Overview

Begin by defining the symptom in operational terms. “The application is down” could mean a container is restarting, a rollout has stalled, a Service has no endpoints, an Ingress is misrouted, or the application is healthy but returning errors. The first useful question is where the failure occurs in the request path:

1. Is the cluster and node capacity available?
2. Did the workload create the expected pods?
3. Are the pods scheduled, ready, and staying running?
4. Does the Service select those pods?
5. Can traffic reach the Service and the correct port?
6. Is the application itself accepting and processing requests?

Use a namespace explicitly in commands when possible. This prevents you from inspecting a similarly named resource in the wrong environment.

kubectl get pods -n <namespace>
kubectl get deploy,svc,ingress -n <namespace>
kubectl get events -n <namespace> --sort-by=.lastTimestamp

Capture the current state before changing anything. Save pod descriptions, recent events, deployment manifests, and relevant logs. A snapshot gives you a reference for comparison and makes it easier to explain what changed.

Checklist by scenario

1. A pod is not starting

First inspect the pod phase, reason, and recent events.

kubectl get pod <pod> -n <namespace> -o wide
kubectl describe pod <pod> -n <namespace>
kubectl get events -n <namespace> --field-selector involvedObject.name=<pod>

Match the observed state to the likely cause:

• Pending: Check scheduling events, node selectors, taints and tolerations, resource requests, and available capacity.
• ImagePullBackOff or ErrImagePull: Verify the image name and tag, registry reachability, and image pull credentials. Confirm that the image architecture is compatible with the target nodes.
• CrashLoopBackOff: Inspect current and previous container logs. Check the command, arguments, environment variables, mounted files, and application startup assumptions.
• CreateContainerConfigError: Look for missing Secrets, ConfigMaps, keys, or volume references.
• OOMKilled: Compare the container’s memory limit with its actual workload and inspect whether the application has a leak, an unexpectedly large input, or an unsuitable limit.

kubectl logs <pod> -n <namespace> -c <container>
kubectl logs <pod> -n <namespace> -c <container> --previous

2. Pods are running but not ready

A running pod is not necessarily able to receive traffic. Inspect readiness and liveness probe results in the pod description. Confirm that the probe path, port, scheme, headers, and startup timing match the application.

kubectl get pod <pod> -n <namespace> -o jsonpath='{.status.containerStatuses[*].ready}'
kubectl describe pod <pod> -n <namespace>

Test the health endpoint from inside the relevant network context when possible. A probe may fail because the application binds only to localhost, listens on a different port, requires authentication, or needs more time to initialize. Use a startup probe for slow initialization rather than weakening a liveness probe until it masks real failures.

3. A deployment is not rolling out

Check the deployment, ReplicaSet, pod counts, and rollout status.

kubectl rollout status deployment/<deployment> -n <namespace>
kubectl describe deployment <deployment> -n <namespace>
kubectl get rs,pods -n <namespace> -l app=<label>

Look for an unavailable replica, a failed readiness probe, insufficient resources, an invalid image, or a scheduling constraint. Compare the new ReplicaSet with the previous one. If the release is clearly harmful and your change process permits it, record the evidence before considering a rollback:

kubectl rollout history deployment/<deployment> -n <namespace>
kubectl rollout undo deployment/<deployment> -n <namespace>

Rollback is a mitigation, not a diagnosis. Follow it with a review of the manifest, image, configuration, and application logs.

4. A Service has no traffic

Start by checking whether the Service has endpoints.

kubectl get svc <service> -n <namespace>
kubectl describe svc <service> -n <namespace>
kubectl get endpointslice -n <namespace> -l kubernetes.io/service-name=<service>

If there are no endpoint addresses, compare the Service selector with the labels on ready pods. A selector typo, namespace mismatch, or pods failing readiness can produce an apparently healthy Service with no usable backends. If endpoints exist, verify the Service port and targetPort, then test connectivity from a temporary diagnostic pod or an existing client pod. Confirm that the application listens on the target port and on an address reachable from the cluster network.

5. Ingress or external access fails

Trace the path from the external address to the backend. Check the Ingress definition, controller events and logs, hostname rules, TLS configuration, and referenced Service.

kubectl describe ingress <ingress> -n <namespace>
kubectl get ingressclass
kubectl get svc <service> -n <namespace>

Separate DNS, TLS, routing, and application failures. A DNS record can point to the wrong address; TLS can reference the wrong Secret; an Ingress rule can use a hostname that does not match the request; or the backend can return an error after routing succeeds. Test the Service directly when possible so you do not treat an Ingress issue and an application issue as the same problem.

6. Configuration, Secrets, or storage are involved

Verify that the expected ConfigMap or Secret exists in the workload’s namespace and that the referenced key names match exactly. Do not print sensitive values into shared terminals, tickets, or logs. For storage failures, inspect pod events and persistent volume claims.

kubectl get configmap <name> -n <namespace>
kubectl get secret <name> -n <namespace>
kubectl describe pvc <claim> -n <namespace>
kubectl get pv

Check access modes, storage capacity, binding status, mount paths, and whether the application expects an existing directory or file. A configuration change may require a pod restart; confirm the workload actually recreated pods after the change.

What to double-check

• Context and namespace: Run kubectl config current-context and confirm the namespace before deleting or editing resources.
• Labels and selectors: Verify deployment selectors, pod labels, Service selectors, and Ingress backend names as a connected set.
• Ports: Distinguish containerPort, Service port, targetPort, and the port used by the health probe. Matching numbers are not required, but the mapping must be intentional.
• Resources: Check both requests and limits. Requests affect scheduling; limits affect runtime behavior.
• Recent events: Events are often the fastest way to identify failed mounts, scheduling decisions, probe failures, and image errors, but they are not a permanent audit log.
• Change history: Compare the current manifest, image digest or tag, configuration revision, and recent deployment activity with the last known-good state.
• Network controls: If traffic fails between namespaces or workloads, inspect NetworkPolicy rules and the labels used by their selectors.

For longer incidents, combine Kubernetes state with application logs and metrics. Structured logs can help correlate a request across services, while metrics can show whether the problem is isolated to one pod or affects the wider workload. Keep diagnostics proportional: collect enough evidence to test a hypothesis, then make one controlled change.

Common mistakes

Deleting a failing pod is sometimes useful, but it can remove the evidence that explains why it failed. Capture describe output, events, and logs first. Similarly, restarting every deployment can hide a dependency or capacity problem and make the incident harder to isolate.

Another common mistake is treating “Running” as “healthy.” Readiness determines whether a pod should receive traffic; liveness and startup probes serve different purposes. A Service also does not guarantee reachability: it depends on selectors, ready endpoints, ports, routing, and network policy.

Avoid editing live resources without recording the intended change. Emergency patches should be reflected in version-controlled configuration afterward, or the next deployment may overwrite the fix. Do not expose Secret contents while debugging, and avoid broad cluster-wide commands when a namespace-scoped command answers the question.

When to revisit

Revisit this checklist before a planned release, after changing an image, probe, resource limit, Service, Ingress, NetworkPolicy, Secret, ConfigMap, or storage definition, and whenever a cluster version or networking component changes. It is also worth reviewing after an incident: remove steps that produced no useful evidence, add commands for recurring failure modes, and document the last known-good configuration.

For the next incident, use this short sequence: confirm context and namespace, record the symptom and recent changes, inspect events, check workload readiness, trace Service endpoints, then test the application directly. Only after those checks should you restart, roll back, scale, or edit resources. This keeps Kubernetes troubleshooting repeatable and turns a one-off fix into a reusable operational practice.