Kubernetes Cost Optimization: Cutting Cluster Spend 30-50%

This walks through the three layers where Kubernetes clusters actually waste compute spend, pod requests/limits, node pool composition, and Cluster Autoscaler tuning, and how fixing all three together typically cuts costs 30-50%.

Where Kubernetes clusters actually bleed money

There are three independent layers, and most teams have only ever tuned one of them (usually node instance types):

Layer 1: Pod requests/limits    →  Are you asking for more than you use?
Layer 2: Node pool composition  →  Are you paying on-demand price for workloads that could run on spot?
Layer 3: Cluster Autoscaler     →  Does the cluster actually scale down when load drops?

Fixing only Layer 2 without fixing Layer 1 means you're over-provisioning on cheaper nodes, real savings, but partial. The 30-50% reductions come from fixing all three together.

Layer 1: Rightsizing requests and limits

Pull at least 7-14 days of real usage per workload before touching anything; a single busy day will lie to you. If you're running metrics-server (you should be) plus Prometheus, run a kubectl top sanity check, then query container_memory_working_set_bytes / container_cpu_usage_seconds_total over time.

Deploy the Vertical Pod Autoscaler in recommendation-only mode. Skip Auto mode on a first pass, it evicts and restarts pods, and you want the numbers before anything restarts in production:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-api-vpa
  namespace: payments
spec:
  targetRef:
    apiVersion: "apps/v1"
    kind: Deployment
    name: checkout-api
  updatePolicy:
    updateMode: "Off" # recommendation-only, nothing is evicted automatically
  resourcePolicy:
    containerPolicies:
      - containerName: "*"
        minAllowed:
          cpu: 50m
          memory: 64Mi
        maxAllowed:
          cpu: 2
          memory: 2Gi

After a week of VPA recommendations, apply them as static requests (or graduate to updateMode: "Auto" once you trust the numbers). This step alone is usually the biggest win of the three: clusters we've audited typically had CPU requests 2-4x actual p95 usage.

Layer 2: Spot and preemptible node pools

Not every workload belongs on spot. Segment by disruption tolerance, not by team.

• Stateless, horizontally-scaled services behind a load balancer → spot/preemptible, always.
• Anything with local state, long-running batch jobs without checkpointing, or a single-replica anything → on-demand.

resource "google_container_node_pool" "spot_workers" {
  name       = "spot-workers"
  cluster    = google_container_cluster.primary.name
  location   = var.region
 
  autoscaling {
    min_node_count = 0
    max_node_count = 20
  }
 
  node_config {
    machine_type = "e2-standard-4"
    spot         = true
 
    labels = {
      workload-class = "spot-tolerant"
    }
 
    taint {
      key    = "cloud.google.com/gke-spot"
      value  = "true"
      effect = "NO_SCHEDULE"
    }
  }
}

Pair the taint with a matching toleration + nodeSelector, and set a PodDisruptionBudget so a spot reclamation wave can't take down every replica at once:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: checkout-api-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: checkout-api

Spot/preemptible capacity typically runs 60-80% cheaper than on-demand for the same instance type.

Layer 3: Making Cluster Autoscaler actually scale down

The most common failure mode isn't "autoscaler doesn't scale up." It's that it never scales back down. Something on every node blocks eviction: a kube-system DaemonSet pod without a PodDisruptionBudget exception, a pod with no controller (bare Pod, not a Deployment), or local storage (emptyDir is fine; hostPath is not).

Worth checking explicitly:

# cluster-autoscaler deployment args
- --scale-down-utilization-threshold=0.5   # default is conservative; 0.5-0.6 is usually safe
- --scale-down-unneeded-time=10m
- --expander=least-waste                   # picks the node group that wastes the least resource
- --balance-similar-node-groups

The default random expander will scale up an oversized node group when a smaller one would've fit the pending pod.

HPA: avoid the thrash trap

Horizontal Pod Autoscaler tuned against raw CPU% alone tends to oscillate under bursty traffic, scaling up, then immediately back down, risking brief capacity gaps along the way. Start with a stabilization window and a realistic target utilization: 70%, not 50%.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: checkout-api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 25
          periodSeconds: 60

What this typically adds up to

A representative before/after from a mid-size cluster (42 nodes, mixed workload):

Before: 42 nodes, on-demand only, avg CPU request utilization 24%   → $38,900/mo compute
After:  27 nodes, 55% spot mix, avg CPU request utilization 61%     → $21,300/mo compute
 
Net reduction: 45% ($17,600/mo)

The 45% reduction came from fixing all three layers together, not from a new tool or a migration.