Kubernetes Cost Optimization on GKE

This covers the three layers where Kubernetes clusters waste compute spend (pod requests/limits, node pool composition and cluster autoscaler tuning), and why the large savings come from fixing all three together. The examples are for GKE Standard. If you run EKS, the Karpenter follow-up covers the AWS side.

Where Kubernetes clusters actually bleed money

There are three independent layers, and most teams have only ever tuned one of them (usually node instance types):

Layer 1: Pod requests/limits    →  Are you asking for more than you use?
Layer 2: Node pool composition  →  Are you paying on-demand price for workloads that could run on spot?
Layer 3: Cluster Autoscaler     →  Does the cluster actually scale down when load drops?

Fixing Layer 2 without Layer 1 means you're over-provisioning on cheaper nodes. That saves real money, but only part of what's available. The big reductions come from fixing all three layers together, because each one multiplies the effect of the others.

Layer 1: Rightsizing requests and limits

Pull at least 7-14 days of real usage per workload before touching anything, because a single busy day gives misleading numbers. If you run metrics-server (you should) plus Prometheus, do a quick kubectl top sanity check, then compare usage over time against what each container requests. container_cpu_usage_seconds_total is a counter, so always wrap it in rate():

# CPU cores actually used, per container (graph the p95 over 7-14 days)
sum by (namespace, pod, container) (rate(container_cpu_usage_seconds_total{container!=""}[5m]))
 
# Memory actually used, per container
max by (namespace, pod, container) (container_memory_working_set_bytes{container!=""})

Deploy the Vertical Pod Autoscaler in recommendation-only mode. Skip the eviction-based modes on a first pass. They restart pods, and you want to see the numbers before anything restarts in production:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-api-vpa
  namespace: payments
spec:
  targetRef:
    apiVersion: "apps/v1"
    kind: Deployment
    name: checkout-api
  updatePolicy:
    updateMode: "Off" # recommendation-only, nothing is evicted automatically
  resourcePolicy:
    containerPolicies:
      - containerName: "*"
        minAllowed:
          cpu: 50m
          memory: 64Mi
        maxAllowed:
          cpu: 2
          memory: 2Gi

On GKE, VPA is built in. Enable it on the cluster (vertical_pod_autoscaling { enabled = true } on the google_container_cluster) instead of installing the upstream components.

After a week or two of VPA recommendations (kubectl describe vpa checkout-api-vpa -n payments shows the Target per container), apply them as static requests in your manifests, or switch to updateMode: "Recreate" once you trust the numbers (InPlaceOrRecreate if your VPA version supports in-place resize; older releases call the eviction mode Auto, which is now deprecated). Don't let VPA and an HPA both act on CPU for the same workload, because they will fight each other. In my experience this step is usually the biggest win of the three. It's common to find CPU requests set at several times the observed p95 usage.

Layer 2: Spot node pools

Not every workload belongs on spot. Segment by disruption tolerance, not by team.

• Stateless, horizontally-scaled services behind a load balancer → Spot, as the default.
• Anything with local state, long-running batch jobs without checkpointing, or a single-replica anything → on-demand.

resource "google_container_node_pool" "spot_workers" {
  name     = "spot-workers"
  cluster  = google_container_cluster.primary.name
  location = var.region # regional pool: nodes spread across the region's zones
 
  autoscaling {
    # total_* limits apply to the whole pool; min_node_count/max_node_count
    # would be *per zone* on a regional pool (20 per zone = 60 nodes in 3 zones)
    total_min_node_count = 0
    total_max_node_count = 20
    location_policy      = "ANY" # take Spot capacity from whichever zone has it
  }
 
  node_config {
    machine_type = "e2-standard-4"
    spot         = true
 
    labels = {
      workload-class = "spot-tolerant"
    }
 
    taint {
      key    = "cloud.google.com/gke-spot"
      value  = "true"
      effect = "NO_SCHEDULE"
    }
  }
}

GKE labels Spot nodes with cloud.google.com/gke-spot: "true" automatically. Pair the taint with a matching toleration and a nodeSelector in the workload's pod template:

    spec:
      nodeSelector:
        cloud.google.com/gke-spot: "true"
      tolerations:
        - key: cloud.google.com/gke-spot
          operator: Equal
          value: "true"
          effect: NoSchedule

And set a PodDisruptionBudget so a drain or scale-down can't take down every replica at once:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: checkout-api-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: checkout-api

The PDB protects against voluntary evictions (drains, autoscaler scale-down). A Spot preemption isn't an eviction, so it ignores the PDB. What protects you from preemption is running enough replicas spread across zones, and location_policy plus a zone topologySpreadConstraint help with that. Google lists Spot VM discounts of roughly 60-91% off on-demand pricing. The actual figure varies by machine type and region, so check current pricing for yours.

Layer 3: Getting the cluster autoscaler to scale down

The common failure mode is an autoscaler that scales up fine and never scales back down, because something on the node can't be moved. On GKE, the usual blockers are:

• a pod with no controller (a bare Pod, not a Deployment/StatefulSet/Job);
• a kube-system pod that isn't a DaemonSet and has no PodDisruptionBudget (think a metrics or DNS add-on scheduled onto a worker node);
• a PDB that allows zero disruptions (minAvailable equal to the replica count);
• a pod annotated cluster-autoscaler.kubernetes.io/safe-to-evict: "false", or whose affinity, node selector or requests mean it can't fit anywhere else.

DaemonSet pods don't block scale-down; they run on every node anyway. For a pod you know can be moved safely, annotate its pod template with cluster-autoscaler.kubernetes.io/safe-to-evict: "true", or give kube-system workloads a PDB.

GKE runs the cluster autoscaler for you and doesn't expose its flags. What you can change is the cluster's autoscaling profile:

resource "google_container_cluster" "primary" {
  # ... rest of the cluster config
 
  cluster_autoscaling {
    enabled             = false # node auto-provisioning off; per-pool autoscaling still applies
    autoscaling_profile = "OPTIMIZE_UTILIZATION"
  }
}

OPTIMIZE_UTILIZATION scales down more aggressively and prefers packing pods onto nodes that are already well allocated. The default BALANCED profile keeps more spare capacity around so pods schedule faster. I'd use OPTIMIZE_UTILIZATION for batch and cost-sensitive clusters. For latency-sensitive services with spiky traffic, watch pending-pod time after switching.

HPA: avoid the thrash trap

Horizontal Pod Autoscaler tuned against raw CPU% alone tends to oscillate under bursty traffic, scaling up, then immediately back down, risking brief capacity gaps along the way. Start with a stabilization window and a realistic target utilization: 70%, not 50%.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: checkout-api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 25
          periodSeconds: 60

How the three layers add up

The effects of the layers multiply, so fixing one in isolation usually disappoints. Here's a rough way to estimate your own ceiling before doing the work:

new_compute_cost ≈ old_compute_cost
                   × (total CPU requested after / total CPU requested before)   # Layer 1 + 3
                   × (1 − spot_share × spot_discount)                            # Layer 2

Treat the result as an upper bound. Bin-packing is never perfect, DaemonSets and system reservations don't shrink, and not every workload can move to Spot. Measure the real before/after with GKE cost allocation or your billing export.