A Kubernetes upgrade is safest when treated as a release-readiness exercise rather than a single maintenance command. This checklist helps you find deprecated APIs, verify version compatibility, test workloads and ingress traffic, prepare add-ons and node pools, and define a rollback or recovery decision before production changes begin.
Overview
Kubernetes upgrades can affect more than the control plane. Resource APIs may be removed or change behavior, controllers may require compatible versions, admission policies can reject previously valid manifests, and node upgrades can expose application assumptions about networking, storage, scheduling, or disruption. Managed Kubernetes services also add provider-specific prerequisites, maintenance windows, and upgrade sequencing.
Use this Kubernetes upgrade checklist for a minor-version upgrade, node-pool migration, managed-service version change, or post-upgrade validation cycle. The exact commands and supported version combinations depend on your Kubernetes distribution and provider, so confirm them against the release documentation for the target version before execution.
Define the upgrade boundary
- Record the current control-plane, node, and client versions.
- Write down the target version and the intended upgrade sequence.
- List affected clusters, environments, regions, node pools, and critical namespaces.
- Identify the change owner, approver, maintenance window, and incident contact.
- Set measurable success criteria, such as healthy nodes, passing smoke tests, normal error rates, and confirmed ingress traffic.
Capture a baseline before changing anything
Take a snapshot of the cluster’s normal operating state. Useful baseline data includes node readiness, pod health, pending workloads, deployment availability, persistent-volume status, ingress responses, important metrics, and recent warning events. A baseline makes it easier to distinguish upgrade-related regressions from pre-existing problems.
kubectl version
kubectl get nodes -o wide
kubectl get pods -A
kubectl get deployments -A
kubectl get statefulsets -A
kubectl get pvc -A
kubectl get events -A --sort-by=.lastTimestamp
Save command output in the change record, but remove credentials, tokens, and sensitive application data before sharing it. For pod-level symptoms during validation, use the Kubernetes pod troubleshooting decision tree.
Checklist by scenario
Scenario 1: Preparing for a minor-version upgrade
- Confirm the supported path. Check the provider or distribution upgrade rules, version-skew guidance, control-plane and node requirements, and any mandatory intermediate version.
- Inventory APIs in use. Review manifests, Helm charts, Git repositories, generated resources, and deployed objects. Search for API versions scheduled for removal or already unavailable in the target release.
- Test rendering against the target. Render Helm and Kustomize output, validate it with the target client or a representative test cluster, and inspect admission results rather than relying only on successful templating.
- Review extensions. Check the compatibility of ingress controllers, CSI and CNI components, autoscalers, policy engines, monitoring agents, service meshes, backup tools, and custom operators.
- Run a representative rehearsal. Upgrade a non-production cluster or isolated environment with similar add-ons, workload types, policies, and traffic patterns.
Scenario 2: Checking deprecated and removed APIs
Do not inspect only files in the main application repository. Deprecated objects can also come from Helm dependencies, operators, scheduled jobs, infrastructure automation, emergency manifests, or resources created by controllers.
kubectl api-resources
kubectl get --raw /openapi/v2 > openapi.json
kubectl get all -A -o yaml > cluster-workloads.yaml
Use version-aware tooling where available to identify deprecated API usage, and compare its output with the target release documentation. The commands above provide inventory and schema context; they are not a complete deprecation scanner. Check both the API version in source manifests and the stored version of live objects. Before production, apply updated manifests in a test environment and confirm that controllers can reconcile them.
Scenario 3: Upgrading nodes or migrating a node pool
- Confirm capacity for workload rescheduling, including CPU, memory, ephemeral storage, and specialized resources.
- Review taints, tolerations, node selectors, affinity rules, topology spread constraints, and architecture requirements.
- Check PodDisruptionBudgets for workloads that must remain available during draining.
- Verify that daemonsets, device plugins, storage drivers, and networking agents support the destination node image and Kubernetes version.
- Drain one representative node or pool according to the provider’s documented process, then observe scheduling and application health before expanding the change.
Scenario 4: Validating ingress and external traffic
Test from outside the cluster as well as from an internal pod. Confirm DNS resolution, TLS certificate selection, HTTP status codes, redirects, authentication, request size limits, timeouts, WebSocket or streaming behavior where relevant, and access to each critical host and path. Compare response latency and error rates with the pre-upgrade baseline.
For a structured investigation of routing, certificates, controller events, and DNS, use the Kubernetes ingress troubleshooting guide.
Scenario 5: Defining rollback and recovery
A Kubernetes rollback is not always a simple version reversal. Managed control planes, node images, persistent data, schema migrations, and add-on changes may have one-way steps. Define the recovery action for each failure mode before the window begins:
- Pause the rollout or stop upgrading additional nodes.
- Restore traffic to a known-good environment if one exists.
- Roll back an application or chart release when the workload, rather than the platform, is at fault.
- Replace or re-create affected nodes using the previous supported image when provider procedures permit it.
- Restore configuration or data from a tested backup when recovery requires it.
- Escalate to the provider or platform owner when the control plane cannot be reversed safely.
What to double-check
Version compatibility
Build a small compatibility matrix before approval. Include the current and target Kubernetes versions, client tools, node images, ingress controller, CNI, CSI drivers, autoscaling components, policy controllers, monitoring agents, operators, and backup software. Mark each item as tested, supported according to its maintainer, or requiring confirmation. Avoid treating an unverified combination as safe simply because the component starts.
| Area | Questions to answer | Evidence to keep |
|---|---|---|
| APIs | Are manifests and live objects using APIs available in the target? | Scanner output, rendered manifests, test applies |
| Workloads | Can deployments, stateful workloads, jobs, and daemonsets restart successfully? | Smoke-test results and rollout status |
| Networking | Do DNS, services, ingress, network policies, and egress paths behave normally? | External and internal request tests |
| Storage | Can volumes attach, mount, write, fail over, and recover as expected? | Driver checks and application-level tests |
| Policy | Do admission rules, security contexts, quotas, and PodDisruptionBudgets still permit intended operations? | Admission events and policy test results |
| Observability | Are logs, metrics, traces, alerts, and audit records still arriving? | Dashboard checks and alert test results |
Workload smoke-test template
For every critical service, record the test, expected result, observed result, owner, and timestamp. A practical sequence is:
- Deploy or restart one replica and verify readiness and liveness behavior.
- Send a representative health or API request.
- Exercise a dependency such as a database, queue, object store, or internal service.
- Confirm logs and metrics include the expected labels and no new recurring warnings.
- Test scaling, rescheduling, and termination behavior where availability depends on them.
If the test involves API payloads or authentication, keep test credentials separate from production secrets. The JSON API debugging guide and JWT debugging and validation guide can support safe request inspection without exposing live tokens.
Common mistakes
- Checking only application manifests: Operators, charts, controllers, and one-off automation can create affected resources too.
- Assuming a successful control-plane upgrade means the change is complete: Nodes, add-ons, workloads, traffic, storage, and observability still need validation.
- Ignoring admission behavior: A manifest may render correctly but be rejected by a policy controller, webhook, quota, or security rule.
- Draining too many nodes at once: Rescheduling can expose capacity limits and disrupt workloads protected by incomplete or misconfigured disruption budgets.
- Testing only a health endpoint: A process can be ready while its database connection, queue consumer, ingress route, or background job is broken.
- Calling application rollback a cluster rollback: Reverting a deployment does not reverse every platform or data change. Define recovery at each layer.
- Skipping the change record: Without command output, timings, decisions, and observed symptoms, incident response becomes slower and less reliable.
When to revisit
Review this checklist before every Kubernetes minor-version upgrade, node-pool migration, managed-service maintenance event, and major add-on change. Revisit it during quarterly platform planning and whenever your organization changes ingress, storage, networking, admission policy, observability, backup, or deployment tooling.
Refresh the API inventory and compatibility matrix when a new target version is selected. Repeat the workload and ingress smoke tests after the change, not only before it. If an upgrade fails, record the triggering symptom, the decision point, the recovery action, and the follow-up fix so the next runbook iteration reflects what the cluster actually needs.
Action for the next upgrade: create a copy of this checklist in the change record, fill in the current and target versions, assign owners for APIs, add-ons, workloads, traffic, storage, and recovery, then require evidence for each item before approving production execution. For broader planning, pair it with the Kubernetes upgrade planning checklist.