Kubernetes FinOps: Cutting Compute Spend by 40%

This picks up after the rightsizing pass, replacing static node pools and Cluster Autoscaler tuning with Karpenter's just-in-time, bin-packing node provisioning so the cluster provisions the actual instance shape each pod needs instead of forcing everything onto a handful of predefined shapes. If you haven't done the rightsizing work yet, start with the cost optimization guide.

What Karpenter does differently

Karpenter removes the node-group abstraction entirely. Instead, it watches for unschedulable pods, looks at what they actually need, and chooses the cheapest instance type and size that satisfies the constraints from the entire eligible set, then calls the EC2 Fleet API directly. No ASG. No node group per shape.

A pending pod that needs 400m CPU and 512Mi memory gets a small instance. A pod that needs 8 vCPU and 32GB gets something else entirely. In the same NodePool, in the same few seconds, Karpenter is solving a bin-packing problem against live EC2 pricing and capacity.

This also means spot becomes a default posture rather than a second node group you have to build and babysit. A single NodePool can express "prefer spot, fall back to on-demand," with Karpenter handling the instance-type diversification that makes spot actually reliable: something teams running Cluster Autoscaler with manually-managed spot ASGs typically skip, pinning to two or three instance types and eating far more interruptions than necessary.

The NodePool and EC2NodeClass

Two CRDs do the work. EC2NodeClass describes the AWS-specific details: AMI, subnets, security groups, IAM role. NodePool describes the scheduling constraints and the disruption behavior.

apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: default
spec:
  amiFamily: AL2023
  role: "karpenter-node-role"
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: "prod-cluster"
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: "prod-cluster"
  blockDeviceMappings:
    - deviceName: /dev/xvda
      ebs:
        volumeSize: 50Gi
        volumeType: gp3
        deleteOnTermination: true

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general-purpose
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
        - key: karpenter.k8s.aws/instance-generation
          operator: Gt
          values: ["4"]
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      expireAfter: 336h
  limits:
    cpu: 1000
    memory: 1000Gi
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 1m

capacity-type: In [spot, on-demand] with no further weighting tells Karpenter to prefer whichever is cheapest for the instance types that satisfy the pod's requirements, and to fall back to on-demand automatically whenever spot capacity is unavailable, whether at launch or after an interruption. expireAfter: 336h forces nodes to be replaced every two weeks even if nothing else triggers it, capping how long a node can silently drift from the latest AMI and forcing regular re-bin-packing.

Workloads opt into this pool via nodeSelector or a topologySpreadConstraint against the labels Karpenter assigns:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout-api
spec:
  replicas: 6
  template:
    spec:
      nodeSelector:
        karpenter.sh/capacity-type: spot
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: checkout-api
      containers:
        - name: checkout-api
          image: registry.internal/checkout-api:1.4.2
          resources:
            requests:
              cpu: "500m"
              memory: "1Gi"

Consolidation: the part Cluster Autoscaler never did well

Cluster Autoscaler scales down by evicting a node once it's fully empty and a scale-down delay has passed; it never moves pods around to create an empty node, or asks whether the fleet is now sub-optimally packed. Karpenter's consolidation loop does both, continuously: it simulates whether running pods could fit onto fewer or cheaper nodes, and if so cordons, drains, and terminates the excess, letting the scheduler re-pack the rest onto a tighter footprint. Unlike the plain WhenEmpty policy, consolidationPolicy: WhenEmptyOrUnderutilized also targets underutilized nodes, the difference between a fleet that shrinks after a traffic spike and one that stays bloated indefinitely.

Consolidation is disruptive by design: anything that can't tolerate an unplanned reschedule needs to say so explicitly:

apiVersion: apps/v1
kind: Pod
metadata:
  name: batch-report-job
  annotations:
    karpenter.sh/do-not-disrupt: "true"
spec:
  containers:
    - name: report
      image: registry.internal/report-generator:2.1.0

Spot interruption handling, natively

The older pattern, Cluster Autoscaler plus manually-managed spot ASGs, relied on aws-node-termination-handler running as a DaemonSet, polling the instance metadata service for the two-minute interruption notice, then cordoning and draining. It works, but it's a separate component to deploy, upgrade, and monitor, and it reacts after AWS has already decided to reclaim the instance.

Karpenter watches for the same interruption signals, rebalance recommendations and the two-minute notice, delivered through an SQS queue fed by EventBridge rules it manages. On receiving one, it immediately begins draining the node and, in the step manual setups usually skip, starts provisioning the replacement before the old node is terminated, rather than waiting for the gap to surface on the next Cluster Autoscaler scan.

What this looks like in the FinOps numbers

The rightsizing pass and the provisioning-model change compound rather than substitute for each other: realistic requests give Karpenter accurate bin-packing inputs, and Karpenter turns those into actual node-count reduction instead of leaving slack in oversized static groups. On engagements where the requests work was already done but the cluster was still on Cluster Autoscaler with two or three static node groups, moving to Karpenter alone has typically taken another 30-40% off the compute line, almost entirely from consolidation and better spot diversification.

Adopting this doesn't touch a single application repo. It's a control-plane swap: new autoscaling component, NodePool/EC2NodeClass definitions, PodDisruptionBudgets that mean what they say, and consolidation left running. The real cost shows up afterward, in ownership: someone now maintains NodePool config the way someone used to own ASGs, and someone audits do-not-disrupt before it turns into a hiding place for idle capacity.