Node Auto-Provisioning (NAP) lets GKE create and delete node pools automatically based on what your pending pods need — the right machine type and size, on demand. Pair it with Spot VMs and you get large clusters that cost a fraction of on-demand while staying reliable. This guide shows the modern Custom Compute Classes approach, with a lab.

Prerequisites

  • A GKE cluster on a recent version (1.30.3-gke.1451000+ for Custom Compute Classes; GA in 1.31).
  • gcloud and kubectl configured for the cluster.
  • A stateless, multi-replica workload to schedule (databases stay on regular nodes).

Two levels of autoscaling

  • The Cluster Autoscaler adds/removes nodes within existing node pools.
  • Node Auto-Provisioning goes further: it creates whole new node pools (choosing machine family and size) when no existing pool fits the pending pods — and removes them when idle.

Enable NAP with resource limits so it can't run away:

gcloud container clusters update my-cluster --enable-autoprovisioning \
  --min-cpu 4 --max-cpu 200 --min-memory 8 --max-memory 800

Spot VMs — up to ~90% cheaper

Spot VMs use spare capacity at a deep discount. The catch: Google can preempt them with ~30 seconds' notice. So they're perfect for fault-tolerant, restartable work (stateless services with several replicas, batch jobs) and wrong for a single-replica database.

The modern way: Custom Compute Classes

Recent GKE adds Custom Compute Classes — the cleanest way to do Spot-first with a fallback. You declare a prioritized list of node shapes, and GKE walks it top-down, provisioning from the first priority that has capacity and falling back to the next when it doesn't. No tolerations, no affinity weights:

apiVersion: cloud.google.com/v1
kind: ComputeClass
metadata:
  name: cost-optimized
spec:
  priorities:                   # ordered fallback — GKE tries top to bottom
    - machineFamily: n2
      spot: true                # 1st choice: cheap Spot
    - machineFamily: n2d
      spot: true                # 2nd: Spot on another family — more capacity to find
    - machineFamily: n2
      spot: false               # last resort: on-demand (regular) so pods always run
  activeMigration:
    optimizeRulePriority: true  # when Spot frees up, move pods back to the cheaper tier
  nodePoolAutoCreation:
    enabled: true               # let node auto-provisioning create the pools these rules need

Pods opt in by name — one label, and every team shares the same policy:

spec:
  nodeSelector:
    cloud.google.com/compute-class: cost-optimized

Why this beats the older trick:

  • Real ordered fallback, not just "prefer" — chain Spot → other Spot families → on-demand.
  • Falls back on stockouts and preemption: if Spot capacity is gone, GKE drops to the next priority automatically and your pods keep running.
  • Active migration back to Spot: optimizeRulePriority moves workloads back onto the cheaper tier when Spot returns, so a one-time stockout doesn't strand you on expensive on-demand.
  • One named policy selected with a nodeSelector, instead of copy-pasting tolerations + affinity into every workload.

Compute classes work on both Standard and Autopilot.

The classic way (older clusters)

Before compute classes, you expressed Spot-first with preferred node affinity plus a toleration:

spec:
  tolerations:
    - key: cloud.google.com/gke-spot
      operator: Equal
      value: "true"
      effect: NoSchedule
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:   # PREFER spot, don't require it
        - weight: 100
          preference:
            matchExpressions:
              - {key: cloud.google.com/gke-spot, operator: In, values: ["true"]}

preferred (not required) is the trick, but it's a single hint — no multi-tier fallback and no migration back to Spot. On recent GKE, prefer compute classes.

Bin-packing, node sizing & consolidation

Autoscaling saves money in two directions — adding capacity and removing waste. GKE's autoscalers work on your pods' resource requests, so honest requests are the foundation of everything here (a pod that requests 10x what it uses forces GKE to provision 10x the nodes). Two behaviors matter:

  • Bin-packing. The scheduler and autoscaler try to pack pods tightly onto nodes. With Node Auto-Provisioning, GKE also picks a machine shape that fits the pending pods well, rather than forcing everything onto one preset type — less stranded capacity.
  • Consolidation / scale-down. When nodes sit underused, the autoscaler drains and removes them, and can reschedule pods to pack them onto fewer nodes. This is where much of the saving comes from — but it only works if pods tolerate being moved (PodDisruptionBudgets and graceful shutdown, below).

Node size is a real trade-off: many small nodes give fine-grained scaling and blast-radius isolation but more per-node overhead; fewer large nodes pack better and cost less per core but scale in coarser steps and concentrate risk. NAP lets you set bounds and picks within them, but understanding the trade-off helps you set sane min/max limits.

Combining Spot with reservations & committed-use

Spot-first (the compute class above) is the cheap default, but production usually blends three capacity types:

  • Spot for the fault-tolerant bulk — cheapest, interruptible.
  • On-demand as the fallback tier so pods always run (your compute class's last priority).
  • Committed-use discounts (CUDs) / reservations for your steady baseline — you commit to a level of usage for 1–3 years and pay 30–60% less. Reservations also guarantee capacity, which Spot never does.

The mature pattern: cover your predictable baseline with committed-use/reserved capacity, run the elastic, interruptible portion on Spot, and keep on-demand as the safety net. A compute class can express Spot → on-demand fallback while your reservations quietly discount the baseline underneath.

Autopilot vs Standard for autoscaling

GKE has two modes, and they change who does the scaling:

  • Standard — you manage node pools (or let NAP manage them). You get full control over machine types, DaemonSets, and node-level tweaks. More power, more responsibility.
  • Autopilot — Google manages nodes entirely; you just deploy pods and pay per pod resource request. Node provisioning, bin-packing, scaling, and security hardening are automatic, and Spot/compute classes are still available. Less control, far less operational surface.

For teams that don't need node-level control, Autopilot removes most of this article's toil — but the concepts (requests drive scaling, Spot-first fallback, PDBs, graceful shutdown) still apply, because they're about your workloads, not the node management.

Observability: watch scale events and cost

Autoscaling is only trustworthy if you can see it working. Watch:

  • Scale events — kubectl get events and the Cluster Autoscaler/NAP logs show why nodes were added or removed and why any pod is stuck Pending (the #1 thing to check when scaling "isn't happening").
  • Pending pods and node utilization — dashboards (kube-prometheus-stack) for real CPU/memory vs requests reveal over-requesting, the main source of waste.
  • Preemption rate — how often Spot nodes are reclaimed, so you can judge whether a workload is Spot-appropriate.
  • Cost — GKE cost allocation / a tool like OpenCost attributes spend per namespace/team, closing the loop so you can prove the Spot-first setup is actually saving money.

Which workloads belong on Spot (and which don't)

Spot-first only works if the right things run on Spot. The test is simple: can this workload survive a node vanishing with ~30 seconds' notice?

Good on Spot:

  • Stateless services with several replicas behind a load balancer — lose one, the rest serve while it reschedules.
  • Batch and CI/CD jobs — restartable by nature; a preempted job just re-runs.
  • Data processing / ML training with checkpointing — resumes from the last checkpoint.
  • Dev/test environments — interruptions are cheap there.

Keep on regular (on-demand/reserved) nodes:

  • Databases and stateful singletons — a preemption mid-write is exactly what you don't want.
  • Anything with a single replica or that can't be safely restarted.
  • Long jobs with no checkpointing — a preemption near the end wastes hours.

The graceful-handling techniques below (replicas, PDBs, SIGTERM) are what make a workload Spot-appropriate. If you can't make it tolerate preemption, it belongs on regular nodes — and that's fine; the compute class's on-demand fallback tier exists for exactly this.

A cost model: what Spot-first actually saves

It's worth seeing the numbers. Suppose a workload needs ~100 vCPUs of steady capacity:

  • All on-demand: baseline cost — call it 100%.
  • All Spot (≈ 60–90% off): roughly 10–40% of baseline — but zero capacity guarantee, so unsafe alone.
  • Spot-first with on-demand fallback (the compute class): most pods land on Spot most of the time, and the on-demand tier only fills in during stockouts/preemptions. Realistically you run the large majority on Spot, landing near 20–40% of the all-on-demand cost with no availability hit.
  • Add committed-use for the baseline: the steady portion gets another 30–60% off and guaranteed capacity, while the elastic portion rides Spot.

The exact figure depends on Spot availability for your machine family and region, but the shape is dramatic: for fault-tolerant workloads, Spot-first routinely cuts compute cost by more than half, and combining it with committed-use for the baseline is how large clusters stay affordable. The one prerequisite for all of it — honest resource requests — is also what makes the savings real rather than theoretical.

The foundation: honest resource requests

Every autoscaling decision in this article rests on one thing — accurate pod resource requests. GKE schedules and provisions based on what pods request, not what they use. Get requests wrong and everything downstream breaks:

  • Over-request (ask for 500m CPU, use 20m) and GKE provisions far more nodes than needed — you pay for Spot and on-demand capacity that sits idle, and the "savings" from Spot-first evaporate under the waste.
  • Under-request (or omit requests) and pods get packed too tightly, then throttle or OOM-kill under load, and the autoscaler can't reason about real needs.

Right-size from real usage: read actual consumption with kubectl top pods and Prometheus, and let the Vertical Pod Autoscaler in recommendation mode suggest requests based on observed usage. Set requests to real usage plus sensible headroom, and set limits to protect the node. This single discipline is what makes Spot-first, bin-packing, and consolidation actually deliver the savings they promise — without it, you're optimizing node cost while quietly wasting half of every node.

Make it graceful

  • Run multiple replicas across nodes (pod anti-affinity) so a preemption never takes all of them.
  • Set a PodDisruptionBudget so voluntary disruptions keep a minimum available.
  • Handle SIGTERM and the ~30s shutdown so in-flight work drains cleanly.
  • Keep anything truly stateful (databases) on regular nodes.

Lab: Spot-first with regular fallback

  1. Create a GKE cluster on a recent version and enable node auto-provisioning with CPU/memory limits.
  2. Apply the cost-optimized ComputeClass above — Spot priorities with an on-demand last resort.
  3. Deploy a stateless app with 3 replicas and nodeSelector: cloud.google.com/compute-class: cost-optimized.
  4. Watch NAP create a Spot node pool and place the pods there: kubectl get nodes -L cloud.google.com/gke-spot.
  5. Simulate a stockout: cordon and delete the Spot pool. Confirm the pods fall back to on-demand automatically — no downtime — then re-enable Spot and watch optimizeRulePriority migrate them back to the cheaper tier.
  6. Compare the tiers' pricing to see the savings.

Prefer the older path? Swap steps 2–3 for the toleration + preferred-affinity manifest — same drill, minus the multi-tier fallback and auto-migration.

Verify it worked

  • kubectl get nodes -L cloud.google.com/gke-spot,cloud.google.com/compute-class shows Spot nodes labeled with your class.
  • Pods are Running on Spot nodes; after you delete the Spot pool they reschedule to on-demand with no outage (multiple replicas + PDB).
  • Billing/labels confirm the cheaper Spot tier is the default.

Troubleshooting

  • Pods stay Pending → NAP limits too low, or no priority can be satisfied. Raise --max-cpu/memory and check kubectl describe pod events.
  • Pods never land on Spot → the workload doesn't select the compute class, or the cluster version predates Custom Compute Classes (use the classic affinity path).
  • Everything went to on-demand and stayed there → a Spot stockout triggered fallback but optimizeRulePriority isn't set, so nothing migrated back. Enable it.
  • Preemption caused a blip → too few replicas or no PDB / SIGTERM handling. Add replicas, a PDB, and graceful shutdown.

Where to go next

Our free course walks this lab end to end on real GKE: enabling NAP, the Spot-first/regular-fallback compute class, and a graceful-preemption drill — so you cut cluster cost dramatically without sacrificing availability.