You can scale a Deployment on memory with a plain autoscaling/v2 HorizontalPodAutoscaler and no extra tooling: set a Resource metric named memory, give every container a memory request, and have metrics-server running. The manifest below does exactly that. The rest of this article is about what the HPA actually does with that manifest — how the number is computed, what changes when you add CPU next to it, why it scales down slower than you expect, and what every error message in kubectl describe hpa means.
Related reading: HPA scale-down behavior and stabilizationWindowSeconds explained.
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: worker-memory
namespace: default
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: worker
minReplicas: 2
maxReplicas: 20
metrics:
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 75 # % of the memory REQUEST, averaged across podsApply it, wait one sync period (15 seconds by default), and kubectl get hpa worker-memory should show memory: <unknown>/75% replaced by a real percentage. If it stays <unknown>, jump to the troubleshooting section — the cause is almost always a missing request or a missing metrics-server.
This is the how-to page. If you want the argument for why memory is usually the wrong signal for stateless services, that is a separate article: Kubernetes HPA best practices. Here I assume you have decided memory is your signal, or you need to understand a memory HPA someone else wrote.
How the HPA Computes Memory Utilization
Three facts explain almost every “why did it do that” question about memory-based HPA.
Utilization is a percentage of the request, not the limit. The HPA controller reads each pod’s memory usage from the resource metrics API (metrics.k8s.io, served by metrics-server), divides it by the sum of the memory requests of that pod’s containers, and averages the result across all pods of the target. A pod with requests.memory: 512Mi using 400Mi is at 78% utilization even if its limit is 2Gi. If any container in the pod has no memory request, utilization for that pod is undefined and the HPA does not act on the metric at all — which is the single most common reason a memory HPA silently does nothing.
The formula is a ratio, not a threshold. Every 15 seconds the controller computes:
desiredReplicas = ceil( currentReplicas × ( currentMetricValue / desiredMetricValue ) )With 4 replicas averaging 90% against a 75% target, that is ceil(4 × 90/75) = ceil(4.8) = 5. With 4 replicas averaging 30%, it is ceil(4 × 30/75) = ceil(1.6) = 2. The HPA does not “add one pod when over the line”; it jumps straight to the replica count that would bring the average back to target, capped by maxReplicas, minReplicas and the scaling policies described later.
There is a tolerance band around the target. If the ratio currentMetricValue / desiredMetricValue is within ±10% of 1.0 (between 0.9 and 1.1), the HPA does nothing. That prevents flapping when memory hovers around the target. The 10% is a cluster-wide default set on kube-controller-manager with --horizontal-pod-autoscaler-tolerance. Since Kubernetes 1.37 the tolerance is also configurable per HPA and per direction through spec.behavior.scaleUp.tolerance and spec.behavior.scaleDown.tolerance (the HPAConfigurableTolerance feature: alpha in 1.33, beta but disabled by default in 1.35, stable and locked on in 1.37). On memory this matters more than on CPU, because memory moves slowly: a wide scale-down tolerance keeps a worker fleet from oscillating on a slow leak-and-GC cycle.
Two more details that bite on memory specifically. Pods that are not Ready are excluded from the average, and pods with no metrics yet are also excluded — so a burst of fresh pods that are still warming up does not drag the average down. And there is a CPU-only grace period (--horizontal-pod-autoscaler-cpu-initialization-period, 5 minutes by default) that has no memory equivalent: a new pod’s memory is counted as soon as metrics-server reports it, which for a JVM means “at full heap, immediately”.
Kubernetes HPA Memory and CPU Example
Most production HPAs that use memory use it alongside CPU. The manifest is just two entries in metrics:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: api-cpu-memory
namespace: default
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: api
minReplicas: 3
maxReplicas: 30
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 80The part people get wrong is the combination logic. Multiple metrics are OR, not AND. The controller computes a desired replica count for each metric independently and then takes the maximum. If CPU says “4 replicas” and memory says “9 replicas”, you get 9. The Deployment scales up when either metric is over its target, and it only scales down when both say fewer replicas would be fine.
This is why “I added memory to my HPA and it never scales down any more” is such a common complaint. CPU drops to nothing at night and proposes 3 replicas; memory, being sticky, is still at 70% of request across 9 pods and proposes 8. The maximum wins. The Deployment sits at 8 replicas until memory actually falls — which for a runtime that holds its heap is never. If that is your situation, the fix is not in the HPA: either the memory request is wrong (see the requests and limits guide), or memory should be a safety valve at 90% rather than a scaling signal at 80%.
A useful pattern is exactly that asymmetric configuration: CPU as the real driver at 60%, memory at 90% purely so a leak or a runaway cache spreads load before pods start getting OOM-killed. In normal operation memory never proposes more replicas than CPU does; in a leak it does, and the fleet grows while you investigate.
averageUtilization vs averageValue
Utilization targets are relative to requests. AverageValue targets are absolute. Both work for memory:
metrics:
- type: Resource
resource:
name: memory
target:
type: AverageValue
averageValue: 1536Mi # scale so that the average pod uses ~1.5GiUse AverageValue when the request is deliberately not a good baseline — for example, a batch worker whose request is set low so it schedules on any node, but whose real working set is known and stable per unit of work. Use Utilization when you want the HPA to follow whatever the request is, so that right-sizing the request automatically re-tunes the autoscaler.
There is a trap with AverageValue and the ratio formula: the target is compared against the average per pod, and the replica count is still derived from the ratio. A target of 1536Mi with pods averaging 1700Mi gives a ratio of 1.107 — just outside the 10% tolerance — and a scale-up. A target of 1536Mi with pods averaging 1600Mi gives 1.04 and nothing happens. If you want the HPA to react to smaller deviations, that is what the per-HPA tolerance field is for.
There is also a third target.type, Value, which compares the total across all pods rather than the average. It is rarely what you want for memory.
Controlling Scale-Down: behavior for a Slow Metric
Memory does not fall the moment traffic stops. Caches stay warm, runtimes hold heap, and a pod that was at 85% ten minutes ago may still be at 80%. The default scale-down behavior — a 300-second stabilization window that uses the highest desired replica count seen in the last five minutes, then removes up to 100% of pods per 15 seconds — is aggressive once the window clears. For memory-driven workloads, slow it down explicitly:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: worker-memory
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: worker
minReplicas: 2
maxReplicas: 20
metrics:
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 75
behavior:
scaleUp:
stabilizationWindowSeconds: 0
policies:
- type: Percent
value: 100
periodSeconds: 60 # at most double the fleet per minute
selectPolicy: Max
scaleDown:
stabilizationWindowSeconds: 600 # look back 10 minutes, not 5
policies:
- type: Pods
value: 1
periodSeconds: 120 # remove at most one pod every 2 minutes
selectPolicy: MinRead this as: scale up fast, scale down one pod at a time and only when the last ten minutes agree. The selectPolicy: Min on scale-down means that if you list several policies, the most conservative one wins. Setting selectPolicy: Disabled on scaleDown turns scale-down off entirely — occasionally the right answer for a memory HPA that exists only to absorb leaks, where you would rather scale down through a deploy than have the HPA do it.
The stabilization window is a rolling maximum of desired replica counts, not a delay. If memory dropped enough to justify 3 replicas eight minutes ago and enough to justify 2 replicas now, a 600-second window still holds the fleet at whatever the maximum proposal in the window was. That is what keeps a GC pause from triggering a scale-down.
Troubleshooting: What the Errors Mean
Everything you need is in kubectl describe hpa <name>, in the Conditions and Events sections, plus kubectl top pod to see what metrics-server is actually reporting. The messages below are the ones you will actually see.
unable to get metrics for resource memory: no metrics returned from resource metrics api — the HPA asked metrics.k8s.io for the pods’ memory and got an empty answer. Either metrics-server is not installed (kubectl get apiservice v1beta1.metrics.k8s.io should show Available: True), or it is installed but cannot scrape the kubelets (the classic symptom is kubectl top pod returning error: Metrics API not available, fixed on many local clusters by adding --kubelet-insecure-tls to the metrics-server args), or the pods are so new that no sample exists yet. The event reason is FailedGetResourceMetric and the ScalingActive condition will be False.
FailedGetResourceMetric with missing request for memory (sometimes phrased missing request for memory in container <name> of Pod <name>) — at least one container in the target pods has no resources.requests.memory. The HPA cannot compute a percentage of nothing, so it ignores the metric. Sidecars are the usual culprit: your app has a request, the injected proxy or log shipper does not. Add a request to every container, or switch that metric to type: ContainerResource with container: <your-app> so only the container you care about is measured.
the HPA was unable to compute the replica count: ... — this is the generic wrapper; the text after the colon is the real cause and is one of the two messages above, or failed to get memory utilization when the target’s pods all failed to report. Check kubectl top pod -l <selector> — if that works and the HPA still fails, the HPA’s selector and the Deployment’s selector disagree (the HPA uses the scale subresource’s selector, so a Deployment with a changed matchLabels can leave the HPA looking at the wrong pods).
<unknown> in kubectl get hpa that never resolves — same causes as above, in this order: no metrics-server, no request on some container, pods not Ready. If TARGETS shows a value for CPU but <unknown> for memory, it is the request.
ScalingLimited: True with TooManyReplicas or TooFewReplicas — not an error. The computed replica count hit maxReplicas or minReplicas. If memory sits at 95% with the HPA pinned at maxReplicas, the fleet is undersized or the request is too small; adding replicas is not going to help.
ScalingActive: False with ScalingDisabled — the target Deployment has replicas: 0, or the HPA was created against a resource that does not implement the scale subresource. The HPA does not scale from zero on resource metrics.
It scales up but never down — re-read the combination rule above. With CPU and memory both present, memory must also fall below target. Then check the stabilization window and any selectPolicy: Disabled. Then check whether kubectl top pod shows memory genuinely staying high; if it does, the HPA is behaving correctly and the workload is what needs looking at.
A quick diagnostic sequence that covers all of these:
kubectl get apiservice v1beta1.metrics.k8s.io # Available: True?
kubectl top pod -n default -l app=worker # do numbers come back?
kubectl get deploy worker -o jsonpath='{.spec.template.spec.containers[*].resources.requests.memory}'
kubectl describe hpa worker-memory | sed -n '/Conditions/,/Events/p'
kubectl get hpa worker-memory -o jsonpath='{.status.currentMetrics}' | jqThe last line is the underrated one: status.currentMetrics shows the exact average utilization the controller computed, which is the number it is plugging into the formula. When the HPA’s arithmetic looks wrong, it is nearly always because that number is not what you assumed — usually because the average includes a pod with a very different request than the others.
When Memory Actually Works as a Signal
Memory-based HPA works when memory per pod is a function of the work in flight and is released when that work finishes. That describes a narrower set of workloads than most people assume, but it is not empty:
- Queue consumers and stream processors that buffer messages in memory: more backlog means more memory per pod, and adding pods drains the backlog.
- In-memory caches and session stores where you want to add capacity before eviction starts, not after latency degrades.
- Image, PDF, or video processing workers whose per-request working set is large and short-lived.
- Anything written in a runtime that returns memory to the OS promptly under low load — Rust, Go with
GOMEMLIMITset sensibly, most C++ services.
It does not work for JVM services, which allocate their heap up front and keep it, or for Go services without a memory limit, whose GC targets a percentage of live heap rather than an absolute ceiling. For those, memory utilization is a property of the configuration, not of the load, and the HPA has nothing to react to. The best practices article goes through the runtime-by-runtime detail; the short version is that if a graph of memory against requests per second is flat, memory is not your signal.
One combination to avoid outright: a VerticalPodAutoscaler in Auto mode and an HPA on the same resource. VPA raises the request, which lowers the utilization percentage, which makes the HPA scale in, which raises per-pod usage, which makes VPA raise the request again. Run VPA in Off (recommendations only) or Initial mode on any Deployment that has a memory HPA. And remember the HPA only ever adds pods — when it hits maxReplicas because there is no room on the nodes, that is the node autoscaler’s problem, covered in Cluster Autoscaler vs Karpenter.
CPU vs Memory as an HPA Signal
| CPU | Memory | |
|---|---|---|
| Tracks request load for stateless services | Usually yes | Usually no |
| Compressible (throttled, not killed) | Yes | No — OOMKill |
| Falls quickly when load drops | Seconds | Minutes to never, runtime-dependent |
| Warm-up grace period in the HPA | Yes, cpu-initialization-period | None |
| Sensitive to request accuracy | Moderately | Extremely |
| Safe default target | 60–70% | 80–90% as a safety valve; 70–75% only when memory is proven load-proportional |
| Typical failure mode | Scales late on latency-bound work | Scales up and never returns, or never scales at all |
Frequently Asked Questions
Can Kubernetes HPA scale on memory and CPU at the same time?
Yes. List both as Resource metrics under spec.metrics in an autoscaling/v2 HorizontalPodAutoscaler. The controller computes a desired replica count for each metric separately and applies the largest one, so the Deployment scales up when either CPU or memory exceeds its target and scales down only when both are below target. A common production pattern is CPU at 60% as the driver and memory at 90% as a safety valve.
Does HPA use memory limits or requests?
Requests. averageUtilization is the pod’s memory usage divided by the sum of its containers’ memory requests, averaged across the target’s pods. Limits are never part of the calculation. If any container in a pod has no memory request, the HPA cannot compute utilization for that pod and reports missing request for memory, and it will not scale on that metric until the request is added.
Why is my memory HPA not scaling down?
Three usual reasons. First, if the HPA also has a CPU metric, the maximum of the two proposals wins, so memory has to fall below its target too. Second, the default 300-second scale-down stabilization window uses the highest desired replica count seen in that window, so a single high sample keeps the fleet up for five minutes. Third, and most often, the memory simply is not falling: JVM heaps and Go runtimes without GOMEMLIMIT hold memory after load drops, so the HPA is reporting the truth.
What does “unable to get metrics for resource memory: no metrics returned from resource metrics api” mean?
The HPA queried the metrics.k8s.io API for the target pods’ memory and got nothing back. Check that metrics-server is installed and its APIService is Available, that kubectl top pod returns numbers for the target pods, and that the pods have been running long enough for a sample to exist. On local clusters the fix is often adding --kubelet-insecure-tls to the metrics-server deployment.
Should I use averageUtilization or averageValue for memory?
Use averageUtilization when the memory request is a meaningful baseline, so that right-sizing the request automatically re-tunes the HPA. Use averageValue (an absolute quantity such as 1536Mi) when the request is deliberately set low for scheduling reasons or when you know the real working set per pod and want to target it directly. Both feed the same ratio formula and are subject to the same 10% tolerance.
Can HPA scale a Deployment to zero on memory?
Not on resource metrics. With HPAScaleToZero enabled (beta and on by default since Kubernetes 1.37) an HPA can set minReplicas: 0, but only when at least one object or external metric is configured; with only CPU or memory metrics there are no pods to measure, so the HPA cannot decide when to scale back up. Scale-to-zero on queue depth or request rate is a job for KEDA.
Conclusion
A memory HPA is three lines of YAML and one requirement — a memory request on every container — but the behavior behind those lines is a ratio against requests, a 10% tolerance band, a maximum-wins rule across metrics, and a five-minute rolling maximum on the way down. Once you hold those four facts, every surprising thing a memory HPA does becomes predictable. Set memory as a safety valve next to CPU unless you have a graph proving memory tracks load; slow the scale-down down; and when it misbehaves, read status.currentMetrics before you touch the target.
