Warrior, now we're learning one of the most useful Kubernetes production features:
HPA automatically increases or decreases the number of Pods based on workload demand.
Until now, you manually did:
kubectl scale deployment node-api --replicas=5But real applications have changing traffic:
Morning → 2 Pods
Afternoon → 5 Pods
Traffic peak → 10 Pods
Night → 2 PodsYou don't want someone manually changing replicas every time.
That's what HPA does.
HPA stands for:
Horizontal Pod Autoscaler
"Horizontal" means:
More Pods
←───────→rather than making one Pod bigger.
Example:
Before:
Pod 1
Pod 2
High traffic:
Pod 1
Pod 2
Pod 3
Pod 4
Pod 5So:
HPA changes the number of Pod replicas.
This distinction is very important.
Add more Pods:
Pod 1
Pod 2
Pod 3
Pod 4This is HPA's job.
Give a Pod more resources:
Before:
CPU = 250m
Memory = 256Mi
After:
CPU = 1
Memory = 1GiThat's vertical scaling.
Conceptually:
Horizontal
↓
More Pods
Vertical
↓
Bigger PodsImagine your Node.js API normally gets:
100 requests/minuteYou have:
2 PodsSuddenly:
10,000 requests/minuteTwo Pods may become overloaded.
HPA can detect increased resource utilization and increase replicas:
2 Pods
↓
4 Pods
↓
8 PodsWhen demand decreases:
8 Pods
↓
4 Pods
↓
2 Pods HPA
│
│ observes metrics
▼
Metrics API
│
▼
Pod Metrics
│
▼
CPU / Memory
│
▼
Desired Replicas
│
▼
Deployment
│
┌─────────┼─────────┐
▼ ▼ ▼
Pod 1 Pod 2 Pod 3HPA doesn't directly create containers itself.
It changes the workload's desired replica count, typically through the target resource such as a Deployment.
Suppose:
spec:
replicas: 2HPA may effectively adjust the Deployment toward:
replicas = 5Then:
Deployment
↓
ReplicaSet
↓
5 PodsRemember the hierarchy:
HPA
↓
Deployment
↓
ReplicaSet
↓
PodsHPA needs data.
For example:
CPU utilization
Memory utilization
Custom metrics
External metricsThe simplest example is CPU.
Suppose:
Target CPU = 70%If Pods consistently exceed that target:
CPU ↑
↓
HPA
↓
Replicas ↑If utilization falls:
CPU ↓
↓
HPA
↓
Replicas ↓For basic CPU/memory HPA, Kubernetes commonly uses the Metrics API, often provided by Metrics Server.
You can check:
kubectl top podsIf this doesn't work, Metrics Server may not be installed/configured.
You might see an error similar to:
Metrics API not availableRun:
kubectl top podsExample:
NAME CPU(cores) MEMORY(bytes)
node-api-abc 120m 180Mi
node-api-def 150m 190Mi
node-api-ghi 110m 175MiAnd:
kubectl top nodesshows node-level usage.
Suppose your Deployment has:
resources:
requests:
cpu: "250m"and HPA target:
70%If the container is using:
175mthen:
175 / 250 = 70%So:
CPU utilization = 70%This is why Chapter 13 was important.
CPU utilization for HPA is commonly calculated relative to the CPU request.
Suppose:
requests:
cpu: "250m"Actual usage:
200mUtilization:
200 / 250 = 80%But if request were:
500mthe same 200m usage would be:
200 / 500 = 40%Same actual CPU.
Different HPA utilization.
Therefore:
Badly chosen resource requests can lead to misleading autoscaling behavior.
Suppose:
Deployment:
node-apiYou can create an HPA with:
kubectl autoscale deployment node-api \
--cpu-percent=70 \
--min=2 \
--max=10Meaning:
Minimum Pods = 2
Maximum Pods = 10
Target CPU = 70%Check:
kubectl get hpaIn production, declarative YAML is often preferable.
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: node-api-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: node-api
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70Apply:
kubectl apply -f hpa.yamlCheck:
kubectl get hpascaleTargetRef#This:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: node-apimeans:
HPA should control the replica count of
node-api.
So:
HPA
│
│ controls
▼
Deployment/node-apiminReplicas#minReplicas: 2means:
HPA shouldn't normally scale below 2 replicas.
Even if CPU is very low:
10%
↓
HPA
↓
2 Podsnot:
0 PodsmaxReplicas#maxReplicas: 10means:
HPA won't scale this target beyond 10 replicas through this HPA configuration.
So:
Traffic ↑
↓
Pods ↑
↓
...
10 Pods
↓
Maximum reachedIf traffic continues increasing, the application may still become overloaded.
That's why autoscaling isn't magic.
Suppose:
3 PodsEach has:
CPU request = 250mCurrent CPU:
Pod A = 200m
Pod B = 225m
Pod C = 250mApproximate utilization:
200 / 250 = 80%
225 / 250 = 90%
250 / 250 = 100%Average:
~90%Target:
70%HPA may decide to increase replicas.
A simplified mental model is:
desired replicas
≈
current replicas ×
(current utilization / target utilization)Suppose:
Current replicas = 3
Current utilization = 90%
Target = 60%Then approximately:
3 × (90 / 60)
= 4.5Kubernetes rounds according to its scaling logic and constraints, so it may move toward:
5 replicasThis is a simplified explanation, not the complete HPA algorithm.
HPA isn't:
CPU = 71%
↓
Instantly add PodKubernetes periodically evaluates metrics and uses stabilization and scaling behavior to avoid excessive oscillation.
So:
Traffic spike
↓
Metrics collected
↓
HPA evaluates
↓
Desired replica count changes
↓
Deployment creates Pods
↓
Pods start
↓
Readiness passes
↓
Traffic reaches new PodsThis takes time.
This connects directly to Chapter 14.
Suppose HPA creates:
Pod DPod D starts:
Pod D
Starting...Readiness:
NotReadyTherefore:
Service
├── Pod A ✅
├── Pod B ✅
├── Pod C ✅
└── Pod D ❌Once Pod D is ready:
Service
├── Pod A ✅
├── Pod B ✅
├── Pod C ✅
└── Pod D ✅That's how HPA and readiness work together.
These three concepts form a very important production triangle:
HPA
│
scale replicas
│
▼
Deployment
│
▼
Pods
│
┌──────┴──────┐
│ │
Resources Readiness
│ │
▼ ▼
Scheduling TrafficRemember this architecture.
You can also scale based on memory.
Example:
metrics:
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 70Meaning:
Target average memory utilization = 70%But memory-based scaling needs careful design.
For example, an application with a steadily growing cache may continually consume memory without traffic increasing.
So:
CPU and memory are signals, not necessarily direct measurements of user traffic.
HPA can use multiple metrics.
For example:
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 80Conceptually:
CPU → 70%
Memory → 80%
│
▼
HPAWhen multiple metrics are configured, HPA evaluates the desired replica count for each and uses the largest recommended count.
CPU and memory aren't always the best scaling signals.
Imagine your API receives:
10,000 requests/secondCPU may still be relatively low.
You might want:
Requests per secondas the scaling signal.
This is where custom metrics can become useful.
Examples:
Requests/second
Queue length
Messages waiting
Active sessions
Application-specific metricYou can also scale based on metrics from outside the cluster.
Examples might include:
Cloud queue depth
External monitoring system
Managed service metricThis requires the appropriate metrics adapter/integration.
Since you work with AI systems, imagine an inference API:
POST /generateSuppose:
Requests ↑
Queue length ↑CPU might not be the best scaling signal.
Instead:
Queue length > 100
↓
Increase inference workersThis is where custom/external metrics become powerful.
A production AI system might use:
Queue depth
GPU utilization
Requests per second
Latency
Tokens/secdepending on the serving architecture and autoscaling system.
This is a critical distinction.
Suppose:
HPA
↓
10 Podsbut your cluster doesn't have enough capacity.
You might get:
Some Pods
↓
PendingHPA increased the number of Pods.
It did not create additional worker nodes.
That's the job of:
Cluster Autoscaler or other node autoscaling mechanisms.
We'll learn that in Chapter 16.
Very important:
Scales:
PodsScales:
NodesArchitecture:
Traffic
│
▼
HPA
│
▼
More Pods
│
▼
Not enough node capacity
│
▼
Cluster Autoscaler
│
▼
More NodesThis is the foundation of elastic Kubernetes clusters.
Suppose:
Cluster
├── Node 1
└── Node 2Traffic increases.
HPA:
3 Pods
↓
8 PodsBut nodes only have capacity for:
5 PodsSo:
5 Pods → Running
3 Pods → PendingCluster Autoscaler detects unschedulable Pods:
Pending Pods
↓
Cluster Autoscaler
↓
Add Node
↓
Pending Pods scheduledFinal:
Node 1
Node 2
Node 3
8 PodsCheck HPA:
kubectl get hpaDetailed:
kubectl describe hpa node-api-hpaWatch:
kubectl get hpa -wCheck metrics:
kubectl top podsCheck deployment:
kubectl get deployment node-apiCheck Pods:
kubectl get podskubectl get hpa#You might see:
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS
node-api-hpa Deployment/node-api 75%/70% 2 10 4Let's decode:
75% / 70%
↑ ↑
current targetMeaning:
Current utilization = 75%
Target = 70%HPA may want to increase replicas.
Suppose:
kubectl get hpashows:
TARGETS: <unknown>/70%Possible causes include:
Metrics API unavailable
Metrics Server problem
Missing resource requests
Configuration issuesCheck:
kubectl describe hpa node-api-hpaAlso:
kubectl top podsSuppose your Deployment:
containers:
- name: api
image: my-apihas no CPU request.
Then CPU utilization-based HPA may not work as expected because percentage utilization requires a meaningful CPU request baseline.
So a good pattern is:
resources:
requests:
cpu: "250m"
memory: "256Mi"before using utilization-based HPA.
Imagine:
Traffic spike
↓
10 PodsTraffic immediately falls:
Traffic ↓Should Kubernetes instantly remove 8 Pods?
Probably not.
Rapid scale up/down can cause:
Oscillation
Cold starts
Unnecessary churn
Performance instabilityKubernetes HPA supports scaling behavior controls and stabilization mechanisms.
For example, conceptually:
Scale up:
Fast
Scale down:
SlowerThis can help prevent aggressive scale-down behavior.
behavior#Modern HPA uses:
behavior:Example:
behavior:
scaleDown:
stabilizationWindowSeconds: 300Meaning:
Avoid immediately scaling down based on transient metric changes; use a stabilization window.
You can also control scale-up/down policies.
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: node-api-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: node-api
minReplicas: 2
maxReplicas: 10
behavior:
scaleUp:
stabilizationWindowSeconds: 0
scaleDown:
stabilizationWindowSeconds: 300
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70This says:
Minimum = 2
Maximum = 10
CPU target = 70%
Scale up:
quickly
Scale down:
stabilize for ~5 minutesExact behavior depends on the complete HPA configuration and controller evaluation.
Imagine:
Node.js APIDeployment:
resources:
requests:
cpu: "250m"
memory: "256Mi"
limits:
cpu: "1"
memory: "512Mi"HPA:
minReplicas: 2
maxReplicas: 10Target:
CPU = 70%Traffic:
Low
↓
2 Pods
Medium
↓
4 Pods
High
↓
7 Pods
Very high
↓
10 PodsIf cluster capacity is insufficient:
HPA
↓
More Pods
↓
Pending Pods
↓
Cluster Autoscaler
↓
More NodesThis is the complete production flow:
TRAFFIC
│
▼
Service
│
┌───────────┼───────────┐
▼ ▼ ▼
Pod A Pod B Pod C
✅ ✅ ❌
Ready Ready NotReady
│ │
└──────┬────┘
│
Traffic
▲
│
HPA
│
CPU / Memory
│
▼
More / fewer PodsThis is a very important Kubernetes pattern.
Horizontal Pod Autoscaler automatically adjusts the number of Pod replicas of a scalable workload based on observed metrics.
More Podsrather than:
Bigger PodUsually a scalable workload such as a Deployment by adjusting its desired replica count.
Examples:
CPU
Memory
Custom metrics
External metricsA component commonly used to provide basic resource usage metrics through the Kubernetes Metrics API.
Because CPU utilization targets are commonly calculated relative to CPU requests.
No.
HPA scales Pods.
Cluster/node autoscaling handles node capacity.
HPA
↓
Pods
Cluster Autoscaler
↓
NodesTechnically, if configured with:
maxReplicas: 100yes, but actual cluster capacity and application behavior determine whether those Pods can run successfully.
To avoid oscillation, cold starts, unnecessary Pod churn, and unstable capacity.
HPA
│
├── minReplicas
├── maxReplicas
├── metrics
└── target
│
▼
Deployment
│
▼
PodsCPU-based scaling:
CPU utilization ↑
↓
HPA
↓
Replicas ↑When demand falls:
CPU utilization ↓
↓
HPA
↓
Replicas ↓But:
HPA → Pods
Cluster Autoscaler → NodesYou now have the full picture:
INTERNET
│
▼
Ingress
│
▼
Service
│
┌─────────────┼─────────────┐
▼ ▼ ▼
Pod 1 Pod 2 Pod 3
│ │ │
└─────────────┼─────────────┘
▲
│
HPA
▲
│
CPU / Memory / Metrics
│
▼
Replica adjustment
│
▼
More / fewer Pods
│
Capacity problem?
│
▼
Cluster Autoscaler
│
▼
More NodesHPA scales Pods; Cluster Autoscaler scales Nodes.
We'll go one level deeper:
Pod Pending
↓
Insufficient node capacity
↓
Cluster Autoscaler
↓
New worker node
↓
Scheduler
↓
Pod gets placedWe'll cover node pools, scale-up, scale-down, cloud Kubernetes behavior, EKS/GKE/AKS concepts, and how HPA + Cluster Autoscaler work together.
Mark it as complete to track your mastery.