Horizontal Pod Autoscaler in Production: Scaling Kubernetes Workloads the Right Way

The Horizontal Pod Autoscaler (HPA) is one of the most valuable controllers in Kubernetes, and also one of the most misunderstood. In a tutorial it is two commands and a happy graph. In production it interacts with resource requests, metrics pipelines, stabilization windows, and your cloud bill in ways that are easy to get wrong. This guide walks through how HPA actually behaves and how to run it with confidence.

What the Horizontal Pod Autoscaler does

HPA automatically adjusts the number of replicas of a workload (a Deployment, ReplicaSet, or StatefulSet) based on observed metrics. When load rises, it adds Pods; when load falls, it removes them. It does not change the resources of a single Pod — that is the job of the Vertical Pod Autoscaler. HPA scales out and in, not up and down.

How it works under the hood

The HPA controller runs inside the control plane and reconciles on a loop (every 15 seconds by default). On each tick it:

1. Reads the current metric value for the target via the metrics APIs.
2. Computes a desired replica count from the ratio of current to target metric.
3. Applies scaling behavior rules and stabilization windows.
4. Patches the scale subresource of the workload.

The core formula is straightforward:

desiredReplicas = ceil(currentReplicas * (currentMetricValue / desiredMetricValue))

If you run 4 replicas averaging 80% CPU and your target is 50%, HPA wants ceil(4 * (80 / 50)) = 7 replicas.

Where the metrics come from

For CPU and memory, HPA relies on metrics-server, a lightweight aggregator that scrapes the kubelet. Without it, CPU/memory autoscaling silently does nothing. Install it and verify:

kubectl top pods -n my-app

Beyond CPU and memory, HPA supports two richer metric types:

• Custom metrics — per-Pod or per-object application metrics (requests per second, queue depth) exposed through a custom metrics adapter such as Prometheus Adapter.
• External metrics — values from outside the cluster (a cloud queue length, a managed message broker backlog).

A sample HPA manifest

The quickest way to create one is kubectl autoscale:

kubectl autoscale deployment api-main \
  --cpu-percent=60 --min=3 --max=20

For anything real, prefer a checked-in manifest using the stable autoscaling/v2 API, which supports multiple metrics and tunable behavior:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-main
  namespace: kubexer-prd
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-main
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 75
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100
          periodSeconds: 15
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 10
          periodSeconds: 60

When multiple metrics are listed, HPA computes a desired count for each and takes the largest, so a single hot signal can drive scale-up.

Scaling behavior and stabilization windows

The behavior block is what separates a calm autoscaler from a flapping one. The stabilization window tells HPA to consider the highest recommendation over a trailing period before scaling down, which prevents thrashing when load is spiky. A common pattern is to scale up fast and scale down slowly: a zero-second up window for responsiveness, and a 300-second down window so you do not shed capacity the moment a spike ends.

Policies cap the rate of change. The example above doubles the replica count at most every 15 seconds going up, but removes only 10% per minute going down.

The prerequisite everyone forgets: resource requests

Utilization targets are computed as a percentage of the Pod's CPU/memory requests. If your containers have no resources.requests set, utilization is undefined and CPU-based HPA will not work. Always set requests:

resources:
  requests:
    cpu: 250m
    memory: 256Mi
  limits:
    cpu: "1"
    memory: 512Mi

Right-size requests to a realistic baseline. Requests that are too high make the workload look idle and HPA scales down too aggressively; requests that are too low make it look hot and you over-provision.

Common pitfalls

• No metrics-server. The single most common cause of an HPA that reports <unknown> targets.
• Missing resource requests. Utilization math has no denominator.
• Fighting another controller. Never hardcode replicas in a Deployment that an HPA also manages, and watch out for GitOps tools that reset it.
• maxReplicas hitting a cluster ceiling. HPA can ask for Pods the cluster cannot schedule. Pair it with the Cluster Autoscaler or Karpenter so node capacity follows.
• Flapping. Tune stabilization windows instead of disabling autoscaling out of frustration.

Why it matters in production

Done well, HPA gives you three things at once: cost efficiency (you pay for capacity only when traffic justifies it), resilience (load spikes are absorbed by new replicas instead of latency and errors), and operational calm (no one is paged at 2am to bump a replica count). The trade-off is that you now have a feedback loop to observe and tune.

That observability is where a dedicated cockpit helps. In Kubexer you can watch a Deployment's replica count react in real time, inspect the events the HPA emits, and correlate restarts and CPU against scaling decisions in one place — which turns HPA tuning from guesswork into something you can actually see.

Wrapping up

Start with sane requests, install metrics-server, set conservative min/max bounds, and add a scale-down stabilization window before you trust it with peak traffic. Observe a full traffic cycle, then tighten the behavior policies. HPA rewards teams that treat it as a tunable control loop rather than a fire-and-forget setting.