Autoscaling

You can scale a Deployment on memory with a plain autoscaling/v2 HorizontalPodAutoscaler and no extra tooling: set a Resource metric named memory, give every container a memory request, and have metrics-server running. The manifest below does exactly that. The rest of this article is about what the HPA actually does with that manifest — how the number is computed, what changes when you add CPU next to it, why it scales down slower than you expect, and what every error message in kubectl describe hpa means.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: worker-memory
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: worker
  minReplicas: 2
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 75   # % of the memory REQUEST, averaged across pods

Apply it, wait one sync period (15 seconds by default), and kubectl get hpa worker-memory should show memory: <unknown>/75% replaced by a real percentage. If it stays <unknown>, jump to the troubleshooting section — the cause is almost always a missing request or a missing metrics-server.

This is the how-to page. If you want the argument for why memory is usually the wrong signal for stateless services, that is a separate article: Kubernetes HPA best practices. Here I assume you have decided memory is your signal, or you need to understand a memory HPA someone else wrote.

How the HPA Computes Memory Utilization

Three facts explain almost every “why did it do that” question about memory-based HPA.

Utilization is a percentage of the request, not the limit. The HPA controller reads each pod’s memory usage from the resource metrics API (metrics.k8s.io, served by metrics-server), divides it by the sum of the memory requests of that pod’s containers, and averages the result across all pods of the target. A pod with requests.memory: 512Mi using 400Mi is at 78% utilization even if its limit is 2Gi. If any container in the pod has no memory request, utilization for that pod is undefined and the HPA does not act on the metric at all — which is the single most common reason a memory HPA silently does nothing.

The formula is a ratio, not a threshold. Every 15 seconds the controller computes:

desiredReplicas = ceil( currentReplicas × ( currentMetricValue / desiredMetricValue ) )

With 4 replicas averaging 90% against a 75% target, that is ceil(4 × 90/75) = ceil(4.8) = 5. With 4 replicas averaging 30%, it is ceil(4 × 30/75) = ceil(1.6) = 2. The HPA does not “add one pod when over the line”; it jumps straight to the replica count that would bring the average back to target, capped by maxReplicas, minReplicas and the scaling policies described later.

There is a tolerance band around the target. If the ratio currentMetricValue / desiredMetricValue is within ±10% of 1.0 (between 0.9 and 1.1), the HPA does nothing. That prevents flapping when memory hovers around the target. The 10% is a cluster-wide default set on kube-controller-manager with --horizontal-pod-autoscaler-tolerance. Since Kubernetes 1.37 the tolerance is also configurable per HPA and per direction through spec.behavior.scaleUp.tolerance and spec.behavior.scaleDown.tolerance (the HPAConfigurableTolerance feature: alpha in 1.33, beta in 1.35, stable and locked on in 1.37). On memory this matters more than on CPU, because memory moves slowly: a wide scale-down tolerance keeps a worker fleet from oscillating on a slow leak-and-GC cycle.

Two more details that bite on memory specifically. Pods that are not Ready are excluded from the average, and pods with no metrics yet are also excluded — so a burst of fresh pods that are still warming up does not drag the average down. And there is a CPU-only grace period (--horizontal-pod-autoscaler-cpu-initialization-period, 5 minutes by default) that has no memory equivalent: a new pod’s memory is counted as soon as metrics-server reports it, which for a JVM means “at full heap, immediately”.

Kubernetes HPA Memory and CPU Example

Most production HPAs that use memory use it alongside CPU. The manifest is just two entries in metrics:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-cpu-memory
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80

The part people get wrong is the combination logic. Multiple metrics are OR, not AND. The controller computes a desired replica count for each metric independently and then takes the maximum. If CPU says “4 replicas” and memory says “9 replicas”, you get 9. The Deployment scales up when either metric is over its target, and it only scales down when both say fewer replicas would be fine.

This is why “I added memory to my HPA and it never scales down any more” is such a common complaint. CPU drops to nothing at night and proposes 3 replicas; memory, being sticky, is still at 70% of request across 9 pods and proposes 8. The maximum wins. The Deployment sits at 8 replicas until memory actually falls — which for a runtime that holds its heap is never. If that is your situation, the fix is not in the HPA: either the memory request is wrong (see the requests and limits guide), or memory should be a safety valve at 90% rather than a scaling signal at 80%.

A useful pattern is exactly that asymmetric configuration: CPU as the real driver at 60%, memory at 90% purely so a leak or a runaway cache spreads load before pods start getting OOM-killed. In normal operation memory never proposes more replicas than CPU does; in a leak it does, and the fleet grows while you investigate.

averageUtilization vs averageValue

Utilization targets are relative to requests. AverageValue targets are absolute. Both work for memory:

metrics:
  - type: Resource
    resource:
      name: memory
      target:
        type: AverageValue
        averageValue: 1536Mi   # scale so that the average pod uses ~1.5Gi

Use AverageValue when the request is deliberately not a good baseline — for example, a batch worker whose request is set low so it schedules on any node, but whose real working set is known and stable per unit of work. Use Utilization when you want the HPA to follow whatever the request is, so that right-sizing the request automatically re-tunes the autoscaler.

There is a trap with AverageValue and the ratio formula: the target is compared against the average per pod, and the replica count is still derived from the ratio. A target of 1536Mi with pods averaging 1700Mi gives a ratio of 1.107 — just outside the 10% tolerance — and a scale-up. A target of 1536Mi with pods averaging 1600Mi gives 1.04 and nothing happens. If you want the HPA to react to smaller deviations, that is what the per-HPA tolerance field is for.

There is also a third target.type, Value, which compares the total across all pods rather than the average. It is rarely what you want for memory.

Controlling Scale-Down: behavior for a Slow Metric

Memory does not fall the moment traffic stops. Caches stay warm, runtimes hold heap, and a pod that was at 85% ten minutes ago may still be at 80%. The default scale-down behavior — a 300-second stabilization window that uses the highest desired replica count seen in the last five minutes, then removes up to 100% of pods per 15 seconds — is aggressive once the window clears. For memory-driven workloads, slow it down explicitly:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: worker-memory
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: worker
  minReplicas: 2
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 75
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100
          periodSeconds: 60        # at most double the fleet per minute
      selectPolicy: Max
    scaleDown:
      stabilizationWindowSeconds: 600   # look back 10 minutes, not 5
      policies:
        - type: Pods
          value: 1
          periodSeconds: 120       # remove at most one pod every 2 minutes
      selectPolicy: Min

Read this as: scale up fast, scale down one pod at a time and only when the last ten minutes agree. The selectPolicy: Min on scale-down means that if you list several policies, the most conservative one wins. Setting selectPolicy: Disabled on scaleDown turns scale-down off entirely — occasionally the right answer for a memory HPA that exists only to absorb leaks, where you would rather scale down through a deploy than have the HPA do it.

The stabilization window is a rolling maximum of desired replica counts, not a delay. If memory dropped enough to justify 3 replicas eight minutes ago and enough to justify 2 replicas now, a 600-second window still holds the fleet at whatever the maximum proposal in the window was. That is what keeps a GC pause from triggering a scale-down.

Troubleshooting: What the Errors Mean

Everything you need is in kubectl describe hpa <name>, in the Conditions and Events sections, plus kubectl top pod to see what metrics-server is actually reporting. The messages below are the ones you will actually see.

unable to get metrics for resource memory: no metrics returned from resource metrics api — the HPA asked metrics.k8s.io for the pods’ memory and got an empty answer. Either metrics-server is not installed (kubectl get apiservice v1beta1.metrics.k8s.io should show Available: True), or it is installed but cannot scrape the kubelets (the classic symptom is kubectl top pod returning error: Metrics API not available, fixed on many local clusters by adding --kubelet-insecure-tls to the metrics-server args), or the pods are so new that no sample exists yet. The event reason is FailedGetResourceMetric and the ScalingActive condition will be False.

FailedGetResourceMetric with missing request for memory (sometimes phrased missing request for memory in container <name> of Pod <name>) — at least one container in the target pods has no resources.requests.memory. The HPA cannot compute a percentage of nothing, so it ignores the metric. Sidecars are the usual culprit: your app has a request, the injected proxy or log shipper does not. Add a request to every container, or switch that metric to type: ContainerResource with container: <your-app> so only the container you care about is measured.

the HPA was unable to compute the replica count: ... — this is the generic wrapper; the text after the colon is the real cause and is one of the two messages above, or failed to get memory utilization when the target’s pods all failed to report. Check kubectl top pod -l <selector> — if that works and the HPA still fails, the HPA’s selector and the Deployment’s selector disagree (the HPA uses the scale subresource’s selector, so a Deployment with a changed matchLabels can leave the HPA looking at the wrong pods).

<unknown> in kubectl get hpa that never resolves — same causes as above, in this order: no metrics-server, no request on some container, pods not Ready. If TARGETS shows a value for CPU but <unknown> for memory, it is the request.

ScalingLimited: True with TooManyReplicas or TooFewReplicas — not an error. The computed replica count hit maxReplicas or minReplicas. If memory sits at 95% with the HPA pinned at maxReplicas, the fleet is undersized or the request is too small; adding replicas is not going to help.

ScalingActive: False with ScalingDisabled — the target Deployment has replicas: 0, or the HPA was created against a resource that does not implement the scale subresource. The HPA does not scale from zero on resource metrics.

It scales up but never down — re-read the combination rule above. With CPU and memory both present, memory must also fall below target. Then check the stabilization window and any selectPolicy: Disabled. Then check whether kubectl top pod shows memory genuinely staying high; if it does, the HPA is behaving correctly and the workload is what needs looking at.

A quick diagnostic sequence that covers all of these:

kubectl get apiservice v1beta1.metrics.k8s.io           # Available: True?
kubectl top pod -n default -l app=worker                 # do numbers come back?
kubectl get deploy worker -o jsonpath='{.spec.template.spec.containers[*].resources.requests.memory}'
kubectl describe hpa worker-memory | sed -n '/Conditions/,/Events/p'
kubectl get hpa worker-memory -o jsonpath='{.status.currentMetrics}' | jq

The last line is the underrated one: status.currentMetrics shows the exact average utilization the controller computed, which is the number it is plugging into the formula. When the HPA’s arithmetic looks wrong, it is nearly always because that number is not what you assumed — usually because the average includes a pod with a very different request than the others.

When Memory Actually Works as a Signal

Memory-based HPA works when memory per pod is a function of the work in flight and is released when that work finishes. That describes a narrower set of workloads than most people assume, but it is not empty:

• Queue consumers and stream processors that buffer messages in memory: more backlog means more memory per pod, and adding pods drains the backlog.
• In-memory caches and session stores where you want to add capacity before eviction starts, not after latency degrades.
• Image, PDF, or video processing workers whose per-request working set is large and short-lived.
• Anything written in a runtime that returns memory to the OS promptly under low load — Rust, Go with GOMEMLIMIT set sensibly, most C++ services.

It does not work for JVM services, which allocate their heap up front and keep it, or for Go services without a memory limit, whose GC targets a percentage of live heap rather than an absolute ceiling. For those, memory utilization is a property of the configuration, not of the load, and the HPA has nothing to react to. The best practices article goes through the runtime-by-runtime detail; the short version is that if a graph of memory against requests per second is flat, memory is not your signal.

One combination to avoid outright: a VerticalPodAutoscaler in Auto mode and an HPA on the same resource. VPA raises the request, which lowers the utilization percentage, which makes the HPA scale in, which raises per-pod usage, which makes VPA raise the request again. Run VPA in Off (recommendations only) or Initial mode on any Deployment that has a memory HPA. And remember the HPA only ever adds pods — when it hits maxReplicas because there is no room on the nodes, that is the node autoscaler’s problem, covered in Cluster Autoscaler vs Karpenter.

CPU vs Memory as an HPA Signal

Frequently Asked Questions

Conclusion

A memory HPA is three lines of YAML and one requirement — a memory request on every container — but the behavior behind those lines is a ratio against requests, a 10% tolerance band, a maximum-wins rule across metrics, and a five-minute rolling maximum on the way down. Once you hold those four facts, every surprising thing a memory HPA does becomes predictable. Set memory as a safety valve next to CPU unless you have a graph proving memory tracks load; slow the scale-down down; and when it misbehaves, read status.currentMetrics before you touch the target.

Kubernetes has introduced as its alpha version in its Kubernetes 1.27 release the Vertical Pod Autoscaling capability to provide the option for the Kubernetes workload to be able to scale using the “vertical” approach by adding more resources to an existing pod. This increases the autoscaling capabilities of your Kubernetes workloads that you have at your disposal such as KEDA or Horizontal Pod Autoscaling.

Vertical Scaling vs Horizontal Scaling

Vertical and horizontal scaling are two approaches used in scaling up the performance and capacity of computer systems, particularly in distributed systems and cloud computing. Vertical scaling, also known as scaling up or scaling vertically, involves adding more resources, such as processing power, memory, or storage, to a single instance or server. This means upgrading the existing compute components or migrating to a more powerful infrastructure. Vertical scaling is often straightforward to implement and requires minimal changes to the software architecture. It is commonly used when the system demands can be met by a single, more powerful infrastructure.

On the other hand, horizontal scaling, also called scaling out or scaling horizontally, involves adding more instances or servers to distribute the workload. Instead of upgrading a single instance, multiple instances are employed, each handling a portion of the workload. Horizontal scaling offers the advantage of increased redundancy and fault tolerance since multiple instances can share the load. Additionally, it provides the ability to handle larger workloads by simply adding more machines to the cluster. However, horizontal scaling often requires more complex software architectures, such as load balancing and distributed file systems, to efficiently distribute and manage the workload across the machines.

In summary, vertical scaling involves enhancing the capabilities of a single object, while horizontal scaling involves distributing the workload across multiple instances. Vertical scaling is easier to implement but may have limitations in terms of the maximum resources available on a single machine. Horizontal scaling provides better scalability and fault tolerance but requires more complex software infrastructure. The choice between vertical and horizontal scaling depends on factors such as the specific requirements of the system, the expected workload, and the available resources.

Why Kubernetes Vertical AutoScaling?

This is an interesting topic because we have been living in a world where the state was that was always better to scale out (using Horizontal Scaling) rather than scaling up (using Vertical Scaling) and especially this was one of the mantras you heard in cloud-native developments. And, that hasn’t changed because horizontal scaling provides much more benefits than vertical scaling and it is well covered with the Autoscaling capabilities or side-projects such as KEDA. So, in that case, why is Kubernetes including this feature and why are we using this site to discuss it?

Because with the transformation of Kubernetes to be the de-facto alternative to any deployment you do nowadays, the characteristic and capabilities of the workloads that you need to handle have extended and that’s why you need to use different techniques to provide the best experience to each of the workloads types

How Kubernetes Vertical Autoscaling?

Here you will find all the documentation about this new feature that as commented is still in the “alpha” stage to is something to try as an experimental mode rather than using it at the production level HPA Documentation

Vertical Scaling works in the way that you will be able to change the resources assigned to the pod, CPU, and memory without needing to restart the pod and change the manifest declaration and that’s a clear benefit of this approach. As you know, until now if you want to change the resources applied to a workload you need to update the manifest document and restart the pod to apply the new changes.

To define this you need to specify the resizePolicy by adding a new section to the manifest pod as you can see here:

apiVersion: v1
kind: Pod
metadata:
  name: qos-demo-5
  namespace: qos-example
spec:
  containers:
  - name: qos-demo-ctr-5
    image: nginx
    resizePolicy:
    - resourceName: cpu
      restartPolicy: NotRequired
    - resourceName: memory
      restartPolicy: RestartContainer
    resources:
      limits:
        memory: "200Mi"
        cpu: "700m"
      requests:
        memory: "200Mi"
        cpu: "700m"

For example in this case we define for the different resource names the policy that we want to apply, if we’re going to change the cpu assigned it won’t require a restart but in case we’re changing the memory it would require a restart.

That implied that if would like to change the CPU assigned you can directly patch the manifest as you can see in the snippet below and that provides an update of the assigned resources:

 kubectl -n qos-example patch pod qos-demo-5 --patch '{"spec":{"containers":[{"name":"qos-demo-ctr-5", "resources":{"requests":{"cpu":"800m"}, "limits":{"cpu":"800m"}}}]}}'

When to use Vertical Scaling are the target scenarios?

It will depend on a lot of different scenarios from the use-case but also from the technology stack that your workload is using to know what of these capabilities can apply. As a normal thing, the CPU change will be easy to adapt to any technology but the memory one would be more difficult depending on the technology used as in most of the technologies the memory assigned is defined at the startup time.

This will help to update components that have changed their requirements as an average scenario or when you’re testing new workloads with live load and you don’t want to disrupt the current processing of the application or simply workloads that don’t support horizontal scaling because are designed on a single-replica mode

 Conclusion

In conclusion, Kubernetes has introduced Vertical Pod Autoscaling, enabling Kubernetes vertical autoscaling of workloads by adding resources to existing pods. Kubernetes Vertical autoscaling allows for resource changes without restarting pods, providing flexibility in managing CPU and memory allocations.

Kubernetes Vertical autoscaling offers a valuable option for adapting to evolving workload needs. It complements horizontal scaling by providing flexibility without the need for complex software architectures. By combining vertical and horizontal scaling approaches, Kubernetes users can optimize their deployments based on specific workload characteristics and available resources.

📚 Want to dive deeper into Kubernetes? This article is part of our comprehensive Kubernetes Architecture Patterns guide, where you’ll find all fundamental and advanced concepts explained step by step.