Five Levers to a 70% Lower Cloud Bill

Lever 1: Placement that matches risk to workload

The first and biggest saving is simply deciding what is allowed to run on spot. A spot node can be reclaimed with two minutes' notice, so the question for every tier is blunt: if this vanishes mid-flight, do we lose money or just a little time?

In Dev, almost everything answers "just time." AI, application, and tooling workloads all run on spot; only databases stay on-demand. Prod is barely different: AI, tooling, and the bulk of the application tier still ride spot, and the database tier is the only one held fully on-demand. The one refinement is that applications flagged critical: true don't move wholesale off spot. They pin a percentage of their pods to on-demand as a floor, using Karpenter's capacity-spread ratio split, so even the critical apps stay majority-spot.

This isn't an exotic bet. Stateless pools on spot with an automatic on-demand fallback is a well-worn pattern, and when the instance-type spread is wide enough the fallback almost never has to fire. Our own ~80%-spot fleet is where most of the ~70% saving comes from; the rest of this page is about protecting and extending it.

• Interruption-tolerant tiers (accelerated AI training/batch, stateless apps, platform tooling) ride spot in both environments
• In Prod, only apps flagged critical: true pin a fraction of pods to on-demand (via the ratio split); the majority still run on spot
• Stateful databases are the one tier never on spot: a reclaim mid-write is a data risk no discount justifies
• A native SQS interruption queue catches the 2-minute reclaim notice so Karpenter drains and replaces the node before the workload notices

Lever 1, continued: A weighted ladder of NodePools

Placement is expressed as a set of Karpenter NodePools with different weights. Weight is a tie-breaker: when a pod matches more than one pool, Karpenter tries the higher-weight pool first, though it will fall through to a lower-weight pool rather than leave a pod pending. Databases sit at the top so stateful pods bind to their on-demand pool first; the application and tooling pools sit lower, spot-first with a deliberately wide instance-type spread so there is always deep spot capacity to land on.

Within a pool that allows both capacity types, Karpenter's price-capacity-optimized allocation picks the cheapest available offering, almost always spot, and only reaches for on-demand when spot runs dry. When we want a hard on-demand floor for a specific app rather than "whatever's cheapest," the on-demand/spot ratio split does it: two NodePools share a synthetic capacity-spread label (spot gets four values, on-demand gets one) and a pod topology spread over that label lands the workload roughly 4:1 on spot. That's the mechanism behind critical: true staying majority-spot: a floor, not a wholesale move. (The ladder and weights below are representative, illustrating the pattern rather than dumping production config.)

# Representative pattern illustrating the on-demand/spot ratio split,
# not a literal dump of production config.

# Spot pool: 4 capacity-spread values -> 4 of every 5 virtual domains.
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: apps-spot
spec:
  weight: 20
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot"]
        - key: capacity-spread
          operator: In
          values: ["2", "3", "4", "5"]
---
# On-demand pool: 1 capacity-spread value -> the remaining 1 in 5.
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: apps-on-demand
spec:
  weight: 20
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["on-demand"]
        - key: capacity-spread
          operator: In
          values: ["1"]
---
# Only critical-flagged pods spread across the domains, landing ~4:1 on
# spot. Everything else just takes the cheapest capacity, almost always spot.
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: capacity-spread
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              critical: "true"

Lever 2: Consolidation keeps the cheap fleet cheap

Provisioning cheap nodes is only half the job; the other half is relentlessly removing the ones you no longer need. Karpenter's consolidation is the single biggest ongoing lever after spot itself. It continuously deletes empty nodes, and (under WhenEmptyOrUnderutilized) simulates removing a node and rescheduling its pods, replacing two half-full nodes with one better-packed (often cheaper) instance.

The part people miss is that consolidation isn't only on-demand → spot. Spot-to-spot consolidation will replace a spot node with a cheaper spot instance type without ever leaving spot, but only when the NodePool offers at least fifteen instance types. That floor exists to stop a "race to the bottom," where Karpenter keeps swapping onto the single cheapest, least-stable instance and pays it all back in churn. It's also why the allocation strategy is price-capacity-optimized, not lowest-price: it trades a sliver of discount for markedly lower interruption rates.

Lever 3: Scaling down is a cost strategy

Scalability usually gets sold as the ability to absorb load. The cost win is the opposite direction: giving compute back the moment it isn't in use. Three autoscalers do this on three different axes, and none of them is the cluster autoscaler that adds and removes nodes. These shape what runs on the nodes we already pay for.

The Vertical Pod Autoscaler right-sizes CPU and memory requests so we stop reserving capacity nothing uses; its newer InPlace and InPlaceOrRecreate modes lean on Kubernetes in-place pod resize to tighten a running pod without the eviction churn that used to make VPA risky on stateful workloads. The Cluster Proportional Autoscaler scales cluster-shaped add-ons like CoreDNS to the node and core count, so they aren't pinned at peak on a fleet that breathes between 300 and 400 nodes. And KEDA takes event-driven workers where the native HPA can't, all the way to zero, waking them straight off queue depth when work arrives.

Lever 4: Turn the bill into a per-workload signal

A monthly AWS invoice is a single number; it can't tell you which team, service, or one careless resource request is driving it. OpenCost (the CNCF's vendor-neutral cost-allocation spec) rebuilds that number from the inside. It reads Prometheus and cAdvisor, prices every node's CPU, memory, GPU, disk, network, and load balancers from cloud or custom rates, and splits each asset's cost into three buckets: the workloads that actually used it, the idle capacity nobody used, and cluster overhead.

The allocation model has one detail that changes behaviour: a workload is billed for max(request, usage). Reserve four cores and use one, and OpenCost charges you for four, and books the other three as idle cost against your namespace, not the cluster's. That makes waste impossible to hide. The /allocation API becomes a worklist: aggregate the same spend by namespace, controller, pod, label, or node, flip on shareIdle to see who's really carrying the slack, and the biggest over-request is the next thing VPA should tighten.

It isn't only in-cluster, either. The Cloud Costs integration reads the AWS Cost and Usage Report, so the actual amortized figures (spot discounts included) land in the same view as the list-price estimates. That's what closes the loop: placement, consolidation, and scaling all become measurable, and the ~80%-spot ratio itself turns into a number we watch per node pool rather than one we assume.

• max(request, usage) billing surfaces over-provisioning as named idle cost, straight into the VPA worklist
• One spend, sliced any way: namespace, controller, pod, label, or node; shareIdle attributes the slack
• Cloud Costs reads the AWS CUR for real amortized cost, so the spot discount shows up, not just list price
• Idle % per node vs per cluster says when to consolidate harder (Lever 2) versus right-size (Lever 3)

Lever 5: Migrating legacy stacks onto Kubernetes

Here's the honest limit of everything above: it only helps a workload that runs on Kubernetes. Spot, consolidation, VPA, KEDA, OpenCost. None of it touches something sitting on ECS. So the last lever is unglamorous and very effective: find the legacy stacks and move them.

The clearest example was our OpenCTI threat-intelligence platform, which I'd originally stood up on AWS ECS. Migrating it to EKS brought it under the same spot-first, consolidated, right-sized regime as everything else and saved $130,200 a year on the AWS bill. One workload, one migration. It has run in production since 2022 with zero infrastructure incidents, so the saving didn't come at the cost of reliability.

Five levers, one bill

No single trick gets you from all-on-demand to a 60–70% lower bill. Spot sets the baseline, consolidation and scaling defend it, migration widens what it covers, and OpenCost tells you which lever to pull next. Pulled together, ~80% spot across the fleet turns into a durably cheaper platform that still holds its SLA, which is the only version of cost-cutting worth doing.