TL;DR
Kubernetes Cost Optimization: Requests, Bin Packing, and Spot Nodes without the fluff: focus on outcomes, measure them, and stop pretending slides are progress.
“Unoptimized Kubernetes clusters are just very expensive deployment scripts.”
Executive Summary
Kubernetes costs grow with scale. Most waste comes from oversized requests, inefficient bin packing, and paying on-demand for non-critical workloads. This guide shows practical ways to reduce spend while keeping reliability: right-size requests, tune autoscalers, bin-pack better, use spot capacity safely, optimize storage/network, and prove savings with dashboards tied to SLOs. Cost is a feature: you own it, you measure it, and you treat regressions like any other incident.
Measure Before You Optimize
- Cost visibility: Deploy Kubecost or cloud-native cost tools; attribute spend by namespace/team.
- Workload inventory: Identify high-request/low-usage workloads; find noisy neighbors.
- SLO context: Set guardrails—never optimize costs at the expense of SLOs.
- Unit economics: Track $/request, $/build, $/workflow—whatever matters. Without a baseline, “we saved money” is just a story.
- Stakeholders: Finance/BizOps should get dashboards too; cost trends without owners turn into finger pointing.
Runbook: Cost Optimization in Six Steps
- Instrument: Install Kubecost or cloud cost allocation; export to Grafana/Looker. Tag namespaces/jobs with owner labels.
- Prioritize: Identify top offenders (oversized CPUs, idle nodes, expensive storage). Tie each to a team owner.
- Right-size: Adjust requests/limits, update quotas, and add usage alerts. Use automation (Vertical Pod Autoscaler recommendations, Datadog/Kubecost suggestions) but review manually.
- Pack + Autoscale: Tune node groups, Cluster Autoscaler, and HPAs to ensure pods land efficiently. Remove zombie nodes.
- Add spot/batch pools: Introduce cheaper capacity with disruption safeguards. Migrate tolerant workloads gradually.
- Review + repeat: Monthly “FinOps retro” with metrics, action items, and SLO checks. Celebrate wins; keep a backlog.
“Cost work is never done. The minute you claim victory, someone ships a 4 vCPU init container with a while loop.”
Right-Size Requests and Limits
Requests drive scheduling and capacity planning; limits cap burst. Oversized requests create fragmentation and idle capacity.
- Use resource usage profiles from Prometheus to set realistic requests (P95 usage + headroom).
- Prefer requests without limits for CPU-bound stateless services to avoid throttling.
- Periodically recalibrate requests based on usage trends.
Example deployment snippet:
resources:
requests:
cpu: "250m"
memory: "256Mi"
limits:
memory: "512Mi"
Improve Bin Packing
Packing pods tightly onto nodes reduces idle capacity.
- Node sizes: Use fewer, larger nodes to reduce fragmentation (validate with workload patterns).
- Scheduling hints: Affinity/anti-affinity and topology spread constraints to prevent hot spots while enabling packing.
- Overcommit cautiously: For CPU, slight overcommit can improve utilization; monitor saturation carefully.
- Priority classes: Assign priorities so critical workloads preempt lower-priority pods instead of competing unpredictably.
- Bin packing tools: Consider Karpenter, Carpenter, or custom controllers that evaluate pending pods and launch right-sized nodes.
- Topology awareness: Use zonal node pools to avoid stranding capacity in a single AZ.
Tune Autoscalers
- HPA: Scale based on meaningful metrics (CPU, requests-per-second, custom).
- Cluster Autoscaler: Ensure it can add/remove nodes quickly; review min/max bounds.
- Buffer capacity: Maintain small headroom for sudden spikes; avoid large static reserves.
- Scheduled scaling: For predictable workloads (batch/nightly jobs), schedule node groups/HPAs ahead of time.
- Vertical Pod Autoscaler (VPA): Use “recommendation” mode to suggest better requests; apply gradually.
- Node auto-provisioning: Cloud providers (GKE Autopilot, EKS Karpenter) can launch right-sized nodes automatically. Validate cost vs control trade-offs.
Use Spot/Preemptible Nodes Safely
Spot capacity offers 60–90% discounts for tolerant workloads.
- Workload selection: Stateless, fault-tolerant, batch jobs.
- Disruption planning: PodDisruptionBudgets, multiple node groups, rapid rescheduling.
- Mixing pools: Combine spot and on-demand; prefer on-demand for critical stateful services.
Storage and Networking Costs
- Use appropriate storage classes; avoid overprovisioning IOPS.
- Right-size persistent volumes; reclaim unused PVCs.
- Review egress charges; prefer in-cluster communication and peering.
- Compress or tier logs; unbounded logging eats storage and egress (SIEM ingestion).
- Set retention policies for backups, snapshots, and object storage.
- Evaluate managed databases vs self-hosted—sometimes the cheapest cluster is the one you don’t run yourself.
Observability and Reporting
- Dashboards per team showing requests vs. usage, cost per service, and trends.
- Alerts on runaway costs or unexpected scaling behaviors.
- Monthly cost reviews tied to action items.
- Track SLOs against cost: if savings degrade latency/error budgets, surface that immediately.
- Provide “pull requests of shame” (playfully) showing diff → cost delta. Make cost part of code review.
Pitfalls
- Aggressive limits cause throttling and degraded latency.
- Ignoring SLOs leads to savings at the expense of user experience.
- One-size-fits-all nodes hurt packing and performance.
- Spot everywhere results in cascading evictions. Mix pools and set PodDisruptionBudgets.
- Unowned costs: Without clear ownership, savings die on the vine. Assign leaders per namespace/platform.
- Manual spreadsheets: Automate dashboards; humans forget to update spreadsheets.
Adoption Plan
- Install Kubecost; build baseline dashboards.
- Right-size top 10 workloads; measure impact.
- Tune HPA/Cluster Autoscaler; test under synthetic load.
- Add spot pools for batch/stateless services; validate disruption handling.
- Quarterly recalibration and cost reviews.
Tooling Stack
- Cost visibility: Kubecost, Cloud provider cost APIs, FinOps tools (CloudHealth, CloudZero).
- Resource insights: Prometheus, Datadog, Datadog Cluster Agent, Metric server, VPA (recommendation mode).
- Scheduling/autoscaling: Cluster Autoscaler, Karpenter, KEDA for event-driven scaling.
- Policy enforcement: Kyverno/OPA to block giant requests, enforce labels, and require owner annotations.
- Automation: Scripts or Terraform modules to adjust node groups, remove idle nodes, and rotate spot pools.
Keep the toolset small and productized; platform teams should offer a “cost toolkit” so app teams don’t reinvent per namespace.
Governance and Guardrails
- Budgets per team: Allocate budgets (monthly/quarterly); alert when 80% consumed. Empower teams to prioritize.
- Policy-as-code: Block Deployments with missing owner labels or requests beyond defined thresholds unless justified.
- FinOps review: Monthly meeting with platform + finance + product to review trends, wins, and backlog.
- Exception process: Provide a lightweight approval flow for workloads needing oversized resources; document expiration dates.
Cost work fails when governance is missing; treat it like reliability with owners and rituals.
Metrics That Matter
- Utilization: CPU/memory requested vs actual usage (%). Aim for 60–80% average without SLO impact.
- Node efficiency: Pods per node, bin packing score (Kubecost provides).
- Spot savings: % of workloads on spot/preemptible; interruption rate; time to recover.
- Cost per unit: $ per request/build/job; highlight improvements over time.
- Change impact: Track cost before/after optimization PRs; tie to backlog items.
Report these monthly; display them next to reliability metrics so teams internalize trade-offs.
30/60/90 Plan
- 30 days: Enable cost visibility; tag namespaces; build dashboards; identify top 5 offenders. Set reliability guardrails (SLO error budgets).
- 60 days: Right-size key workloads; tune autoscalers; introduce spot pools for batch/stateless services; implement policy blocks for giant requests. Present savings report with before/after graphs.
- 90 days: Automate request tuning (scripts/VPA); add bin-packing enhancements (larger nodes, scheduler tweaks); integrate cost alerts into Slack/PagerDuty; create quarterly FinOps review with Finance/product.
Case Study (Hypothetical)
A streaming company spends $400k/month on Kubernetes. Kubecost shows 40% of spend in two namespaces with idle CPU. Platform team right-sizes requests using Prometheus 95th percentile, cutting node count by 20%. Next, they add spot pools for batch encoding jobs with disruption budgets, saving another 15%. Cluster Autoscaler min/max tuned; unused GPU nodes removed. In 90 days, spend drops to $280k/month while SLO burn remains flat. Finance sees the delta; teams get cost dashboards in their OnCall channel.
Conclusion
Cost optimization is continual. Right-size requests, improve packing, and use spot responsibly—then prove it with data. With the right guardrails, you’ll reduce spend while keeping reliability and developer velocity.