Servers & Application Architecture

1. Chapter Overview#

In the previous chapters, we followed a request from the user all the way to HTTP communication:

User
 ↓
Browser / Mobile App
 ↓
DNS
 ↓
IP
 ↓
TCP / QUIC
 ↓
TLS
 ↓
HTTP
 ↓
Load Balancer / Reverse Proxy
 ↓
Application

Now we need to understand the application side of that architecture.

When an HTTP request reaches a server:

What actually happens?

What is a server?

What is a process?

What is a thread?

What is a worker?

What is a web server?

What is an application server?

How can one machine handle thousands of requests?

How do we make an application horizontally scalable?

These questions are fundamental to system design.

2. What Is a Server?#

The word server can mean different things depending on context.

At the simplest level:

A server is a computer or software process that provides a service to clients.

For example:

Browser
   ↓
Web Server

or:

Application
   ↓
Database Server

or:

Application
   ↓
Redis Server

The word "server" can therefore refer to:

1. a physical/virtual machine
2. an operating-system process
3. a software application
4. a role in an architecture

This distinction is important.

3. Server as a Machine#

A server machine is simply a computer running workloads.

It has resources such as:

CPU
RAM
Disk
Network

For example:

Server
├── 8 CPU cores
├── 32 GB RAM
├── 500 GB SSD
└── 1 Gbps network

A cloud VM is also essentially a virtualized server environment.

Examples of server environments include:

• virtual machines
• containers
• bare-metal machines
• cloud instances
• Kubernetes pods/nodes

4. Server as Software#

A server program listens for requests.

For example:

Application starts
      ↓
Bind to port 8080
      ↓
Listen for connections
      ↓
Receive request
      ↓
Process request
      ↓
Send response

In Node.js:

import http from "http";
 
const server = http.createServer((req, res) => {
  res.end("Hello");
});
 
server.listen(3000);

The important concept is:

listen → receive → process → respond

5. IP Address + Port#

A network service normally listens on a port.

Example:

192.168.1.10:3000

Here:

192.168.1.10 → IP address
3000         → port

A server can run multiple services:

:80    → HTTP
:443   → HTTPS
:5432  → PostgreSQL
:6379  → Redis

The operating system uses ports to direct network traffic to the appropriate process.

6. What Happens When a Request Reaches a Server?#

A simplified flow:

Network
   ↓
Operating System
   ↓
Listening Socket
   ↓
Server Process
   ↓
Request Handler
   ↓
Business Logic
   ↓
Database / Cache / Other Services
   ↓
Response

This is a key system-design mental model.

7. Socket#

A socket is an endpoint used for network communication.

Conceptually:

Client Socket
      ↕
Network
      ↕
Server Socket

A server can listen on a socket such as:

0.0.0.0:8080

When connections arrive, the operating system manages the underlying network state and makes data available to the server process.

8. Process#

A process is a running instance of a program.

For example:

Node.js application
       ↓
Process

A process has its own:

• memory address space
• resources
• execution context

Conceptually:

Process
├── Code
├── Heap
├── Stack
├── Open files
├── Sockets
└── Other resources

9. Thread#

A thread is an execution path within a process.

A process can have:

Process
├── Thread 1
├── Thread 2
├── Thread 3
└── Thread 4

Threads within the same process can share process memory.

This makes communication between threads efficient, but shared memory also creates concurrency challenges.

Examples:

• race conditions
• locks
• deadlocks
• synchronization problems

10. Process vs Thread#

This is a simplified model; actual behavior depends on the operating system and runtime.

11. CPU Cores#

Suppose a machine has:

8 CPU cores

It can execute multiple threads concurrently.

A useful mental model:

CPU
├── Core 1
├── Core 2
├── Core 3
├── Core 4
├── Core 5
├── Core 6
├── Core 7
└── Core 8

Operating systems schedule runnable threads onto CPU cores.

Important distinction:

A program can have many threads even when the machine has fewer CPU cores.

The OS schedules them over time.

12. Concurrency vs Parallelism#

These are often confused.

Concurrency#

Multiple tasks are in progress during overlapping periods.

Task A ───────
      Task B ───────
            Task C ───────

Parallelism#

Multiple tasks actually execute simultaneously on different CPU cores.

Core 1 → Task A
Core 2 → Task B
Core 3 → Task C

A system can be concurrent without everything executing simultaneously.

13. Web Server#

A web server is software that handles HTTP-related communication, especially serving web content and forwarding requests.

Examples include:

• Nginx
• Apache HTTP Server
• Caddy

A web server can handle:

• static files
• TLS termination
• compression
• caching
• reverse proxying
• connection handling
• routing

Example:

Browser
   ↓
Nginx
   ├── /images → static files
   └── /api    → application server

14. Application Server#

An application server runs application/business logic.

For example:

HTTP Request
     ↓
Application Server
     ↓
Validate request
     ↓
Authenticate user
     ↓
Business logic
     ↓
Database
     ↓
Response

In modern frameworks, the distinction between web server and application server can be blurred.

For example, a framework may include its own HTTP server while Nginx sits in front as a reverse proxy.

15. Web Server vs Application Server#

A simplified distinction:

In production, one component can perform multiple roles.

Do not treat the terminology as an absolute architectural boundary.

16. Reverse Proxy#

A reverse proxy receives client requests and forwards them to backend services.

Client
  ↓
Reverse Proxy
  ↓
Application

Examples:

• Nginx
• HAProxy
• Envoy

A reverse proxy can provide:

• TLS termination
• routing
• load balancing
• compression
• caching
• access control
• request limits
• observability

17. Why Put a Reverse Proxy in Front?#

Suppose:

Internet
   ↓
Application Server

Now imagine adding:

Internet
   ↓
Reverse Proxy
   ↓
Application Server

The proxy can absorb infrastructure concerns.

For example:

Client
 ↓ HTTPS
Nginx
 ↓ HTTP/internal TLS
App

The application may no longer need to directly manage all public-facing TLS and connection behavior.

18. Application Process#

Suppose we have:

Node.js application

Starting it creates a process.

Machine
 └── Node Process
      └── Application

That process consumes:

CPU
RAM
File descriptors
Network sockets

As traffic increases, these resources can become bottlenecks.

19. One Process Handling Requests#

A simplistic architecture:

Client 1 ─┐
Client 2 ─┤
Client 3 ─┼──→ Application Process
Client 4 ─┤
Client 5 ─┘

The application receives requests and schedules their processing according to its runtime model.

Different languages/runtimes handle concurrency differently.

Examples:

• Node.js event loop
• Java thread pools
• Go goroutines
• Python worker/process models
• .NET thread pool

System design should focus on the resource and concurrency model rather than assuming all runtimes behave identically.

20. Node.js Example#

Node.js uses an event-driven architecture.

A simplified model:

Incoming Request
      ↓
Event Loop
      ↓
Application Code
      ↓
Async I/O
      ↓
Event Loop
      ↓
Response

For I/O-heavy workloads, Node.js can handle many concurrent operations without creating one OS thread per request.

However:

JavaScript execution on the main Node.js event-loop thread is not magically parallel.

CPU-heavy work can block the event loop.

21. Why CPU-Heavy Work Is Dangerous in Node.js#

Suppose:

app.get("/report", (req, res) => {
  // Very expensive CPU calculation
});

If this blocks the event loop:

Request A
   ↓
CPU-heavy work
   ↓
Event loop blocked
 
Request B → waits
Request C → waits
Request D → waits

Therefore CPU-intensive work may need:

• worker threads
• separate processes
• background workers
• specialized services
• queues

22. Worker#

A worker is a unit responsible for processing work.

For example:

API Server
   ↓
Queue
   ↓
Workers
├── Worker 1
├── Worker 2
├── Worker 3
└── Worker 4

Workers are especially useful for background tasks:

• email
• image processing
• video encoding
• report generation
• data processing
• notifications

23. Worker Pool#

Instead of creating a new worker for every task, a system can maintain a pool.

Worker Pool
├── Worker 1
├── Worker 2
├── Worker 3
└── Worker 4

Jobs are distributed among available workers.

Benefits:

• bounded concurrency
• resource control
• improved throughput
• reduced startup overhead

24. Connection Pool#

Database connections are expensive resources.

Instead of:

Request
 ↓
Create DB connection
 ↓
Query
 ↓
Close connection

for every request, applications commonly use a connection pool.

Application
     ↓
Connection Pool
├── DB Connection 1
├── DB Connection 2
├── DB Connection 3
└── DB Connection 4
     ↓
Database

Requests reuse connections.

25. Why Connection Pools Matter#

Suppose:

1,000 concurrent requests

and every request creates a new database connection.

The database may become overwhelmed.

Instead:

1,000 application requests
          ↓
Connection Pool
          ↓
50 database connections
          ↓
Database

The exact pool size depends on workload and database capacity.

A pool is therefore a form of resource throttling.

26. Connection Pool Saturation#

Suppose:

Pool size = 20
Active queries = 20

Request #21 may have to wait.

If waiting requests continue to accumulate:

Requests
  ↓
Connection Pool
  ↓
Waiting Queue
  ↓
Timeouts

This can create cascading latency.

Therefore connection pool configuration is a system-design concern.

27. Thread Pool#

Many runtimes maintain worker threads for operations such as:

• filesystem operations
• DNS operations
• cryptographic work
• blocking integrations

The exact implementation differs by runtime.

The general concept is:

Application
     ↓
Thread Pool
├── Thread 1
├── Thread 2
├── Thread 3
└── Thread 4

If all workers are busy, additional work waits.

28. Request Lifecycle Inside an Application#

Let's follow a request.

HTTP Request
     ↓
Reverse Proxy
     ↓
Application Server
     ↓
Router
     ↓
Middleware
     ↓
Authentication
     ↓
Validation
     ↓
Controller / Handler
     ↓
Business Logic
     ↓
Cache / Database / External API
     ↓
Response Serialization
     ↓
HTTP Response

This is one of the most useful mental models for backend system design.

29. Routing#

Routing determines which application code handles a request.

Example:

GET /users/123

may map to:

UserController.getUser(123)

Another:

POST /orders

may map to:

OrderController.createOrder()

30. Middleware#

Middleware runs during request processing.

Example:

Request
 ↓
Logging middleware
 ↓
Authentication middleware
 ↓
Validation middleware
 ↓
Route handler
 ↓
Response

Middleware can perform:

• authentication
• logging
• request IDs
• validation
• rate limiting
• error handling
• metrics

31. Business Logic#

Business logic contains the actual application rules.

For example:

Create Order
 ↓
Check user
 ↓
Check inventory
 ↓
Calculate price
 ↓
Apply discount
 ↓
Create order
 ↓
Publish event

Do not put all business logic directly into HTTP handlers.

A cleaner architecture separates:

HTTP Layer
     ↓
Application / Domain Layer
     ↓
Data Access Layer

32. Layered Architecture#

A common structure:

┌──────────────────────────┐
│ Presentation / API Layer │
├──────────────────────────┤
│ Business / Service Layer │
├──────────────────────────┤
│ Data Access Layer        │
├──────────────────────────┤
│ Database                 │
└──────────────────────────┘

Example:

Controller
   ↓
Service
   ↓
Repository
   ↓
Database

This separation makes code easier to test and evolve.

33. Controller#

The controller handles the HTTP-facing concerns.

Example:

POST /users

Controller responsibilities may include:

• reading request
• validating input
• calling service
• converting result to HTTP response

It should not necessarily contain every business rule.

34. Service Layer#

The service layer implements application behavior.

Example:

UserService.createUser()

It may:

validate business rules
 ↓
check duplicate email
 ↓
hash password
 ↓
create user
 ↓
publish event

35. Repository / Data Access Layer#

The repository handles persistence concerns.

Example:

UserRepository.findByEmail()
UserRepository.create()
UserRepository.findById()

The service should not need to know every SQL implementation detail.

Conceptually:

Controller
   ↓
Service
   ↓
Repository
   ↓
Database

36. Monolith#

A monolith is an application deployed as a single application unit.

Example:

                Monolith
┌──────────────────────────────────┐
│ Users                            │
│ Orders                           │
│ Payments                         │
│ Notifications                   │
│ Admin                            │
└──────────────────────────────────┘
                ↓
             Database

All major application functionality is packaged and deployed together.

37. Modular Monolith#

A modular monolith keeps one deployment unit but separates internal domains.

Monolith
├── User Module
├── Order Module
├── Payment Module
├── Notification Module
└── Admin Module

This can provide stronger internal organization without immediately introducing distributed-system complexity.

38. Microservices#

In a microservice architecture:

User Service
Order Service
Payment Service
Notification Service

Each service can potentially be deployed independently.

                 API Gateway
                      ↓
       ┌──────────────┼──────────────┐
       ↓              ↓              ↓
 User Service    Order Service   Payment Service
       ↓              ↓              ↓
   User DB        Order DB       Payment DB

Microservices provide flexibility but introduce distributed-system complexity.

39. Monolith vs Microservices#

There is no universal rule that one architecture is always correct.

The architecture should match the problem.

40. Why Not Start With Microservices?#

Suppose your application has:

5 users
3 developers
simple CRUD

Creating:

15 microservices
20 databases
Kafka
Service mesh
API gateway

can add enormous operational complexity without solving a real problem.

A well-structured monolith may be sufficient.

A common evolution path is:

Modular Monolith
       ↓
Identify scaling/team/domain boundaries
       ↓
Extract selected services
       ↓
Microservices where justified

41. Horizontal Scaling#

Suppose one application instance can handle:

1,000 requests/sec

Traffic becomes:

5,000 requests/sec

Instead of making one machine five times larger:

Machine A
5× CPU

we can run:

Server 1
Server 2
Server 3
Server 4
Server 5

and distribute requests.

                 Load Balancer
                /    |    |    \
               ↓     ↓    ↓     ↓
             App1  App2  App3  App4

This is horizontal scaling.

42. Vertical Scaling#

Instead, we can increase the resources of one server.

Before:
4 CPU
16 GB RAM
 
After:
16 CPU
64 GB RAM

This is vertical scaling.

It is often simpler initially, but there are hardware/resource limits.

43. Stateless Application Servers#

Horizontal scaling becomes easier when application instances are stateless.

Instead of:

App1
 └── User session in local memory

prefer:

App1 ─┐
App2 ─┼──→ Shared Session Store
App3 ─┘

or use stateless authentication when appropriate.

Then any instance can handle the next request.

44. Load Balancer + Application Servers#

A typical architecture:

                Internet
                    ↓
              Load Balancer
              /     |     \
             ↓      ↓      ↓
           App1    App2    App3
             \      |      /
              \     |     /
                 Cache
                    ↓
                 Database

The load balancer distributes requests across instances.

We will study load balancing in detail in Chapter 07.

45. Health Checks#

A load balancer should avoid sending traffic to unhealthy instances.

Two common concepts:

Liveness#

Is the process alive?

Readiness#

Is the application ready to receive traffic?

Example:

GET /health/live
GET /health/ready

An application may be alive but not ready.

For example:

Process running
Database connection unavailable

The process is alive, but it may not be ready to serve requests.

46. Graceful Shutdown#

Suppose an application instance is being replaced.

Bad shutdown:

Kill process
 ↓
Active requests fail

Graceful shutdown:

Stop accepting new requests
        ↓
Finish active requests
        ↓
Close connections
        ↓
Flush important buffers
        ↓
Exit

This reduces request failures during deployments and scaling events.

47. Zero-Downtime Deployment Concept#

Suppose:

Version A
App1
App2
App3

Deploy Version B:

App1 → Version B
App2 → Version B
App3 → Version B

A load balancer can gradually remove old instances from service.

Conceptually:

Traffic
  ↓
Old instances + new instances
  ↓
Gradually shift
  ↓
Only new instances

Later we will study deployment strategies more deeply.

48. Server Bottlenecks#

An application server can be limited by:

CPU#

CPU = 100%

Memory#

RAM exhausted

Network#

Network bandwidth saturated

File descriptors#

Too many open files/sockets.

Connection pools#

All DB connections busy.

External dependencies#

A slow dependency causes application requests to wait.

Lock contention#

Threads/processes wait for shared resources.

49. Backpressure#

Backpressure means slowing or limiting incoming work when downstream capacity is limited.

Example:

Traffic
  ↓
API
  ↓
Queue
  ↓
Workers

Suppose workers can process:

1,000 jobs/sec

but incoming traffic produces:

10,000 jobs/sec

The queue grows.

A system needs a strategy:

• limit requests
• reject work
• queue work
• scale workers
• shed lower-priority work

Without backpressure, queues and memory can grow until the system fails.

50. Load Shedding#

Sometimes the correct action during overload is to reject work.

Example:

System capacity = 10,000 req/sec
Incoming = 20,000 req/sec

Trying to process everything may cause:

CPU exhaustion
memory exhaustion
timeouts
cascading failures

Instead, the system may reject some requests:

429 Too Many Requests
503 Service Unavailable

This protects the remaining capacity.

51. Graceful Degradation#

A system can sometimes provide reduced functionality instead of failing completely.

Example:

Recommendation Service unavailable

Instead of:

Entire product → failure

the application can return:

Product page
+
"No recommendations currently available"

This is graceful degradation.

52. Caching at the Application Layer#

A server may use a cache:

Request
 ↓
Application
 ↓
Cache hit?
 ├── Yes → return
 └── No
      ↓
   Database
      ↓
   Store cache
      ↓
   Return

This reduces database load and latency.

We will study caching in depth in Chapters 15 and 16.

53. Server-Side Rendering#

For web applications, the server may generate HTML.

Browser
 ↓
HTTP Request
 ↓
Web/Application Server
 ↓
Generate HTML
 ↓
Response

Alternatively, the server may return JSON to a frontend application.

Browser
 ↓
API
 ↓
JSON
 ↓
React / Next.js

The architecture depends on the application requirements.

54. Static vs Dynamic Work#

A server can serve static content:

logo.png
style.css
app.js

or dynamic content:

GET /profile
 ↓
Database lookup
 ↓
Generate response

Static content is often moved to a CDN or object storage.

Dynamic requests usually require application processing.

55. Server Architecture Example#

A simple production architecture:

                    Internet
                       ↓
                    CDN
                       ↓
                Load Balancer
                       ↓
             ┌─────────┼─────────┐
             ↓         ↓         ↓
           App1      App2      App3
             │         │         │
             └────┬────┴────┬────┘
                  ↓         ↓
                Redis    Database

The application servers are stateless.

Shared state is stored in external systems.

56. More Realistic Architecture#

                           Internet
                              │
                              ▼
                         CDN / Edge
                              │
                              ▼
                       Load Balancer
                              │
                              ▼
                        API Gateway
                              │
                  ┌───────────┼───────────┐
                  ▼           ▼           ▼
               App 1        App 2       App 3
                  │           │           │
                  └───────────┼───────────┘
                              │
                   ┌──────────┼──────────┐
                   ▼          ▼          ▼
                 Redis     Database    Queue
                                         │
                              ┌──────────┼──────────┐
                              ▼          ▼          ▼
                           Worker 1   Worker 2   Worker 3

This architecture combines:

• edge delivery
• load balancing
• stateless applications
• caching
• persistent storage
• asynchronous processing

57. Synchronous vs Asynchronous Application Work#

A request can perform work synchronously:

Client
 ↓
API
 ↓
Process
 ↓
Database
 ↓
Response

Or move expensive work to a background queue:

Client
 ↓
API
 ↓
Queue
 ↓
202 Accepted

Then:

Queue
 ↓
Worker
 ↓
Database / Storage

This allows the API to return quickly.

58. When to Use Background Workers#

Good candidates include:

• sending emails
• sending notifications
• generating PDFs
• video processing
• image resizing
• large imports
• analytics processing
• report generation

The client does not necessarily need to wait for these operations.

59. Application Architecture Principles#

Good application architecture usually aims for:

Separation of concerns#

Different responsibilities remain separated.

Statelessness where practical#

Avoid unnecessary local state.

Clear boundaries#

Separate domains and dependencies.

Controlled resource usage#

Use connection pools and bounded concurrency.

Timeouts#

Do not wait forever.

Observability#

Logs, metrics, and traces should make failures diagnosable.

Graceful failure#

Avoid taking down the entire system because one dependency failed.

Simplicity#

Do not introduce distributed complexity without a reason.

60. Common Mistakes#

Mistake 1 — Thinking "server" means only a physical machine#

A server can mean a machine, process, or service.

Mistake 2 — Assuming one request equals one thread#

Different runtimes use different concurrency models.

Mistake 3 — Assuming more threads always means more performance#

Too many threads can cause:

• context switching
• memory usage
• contention
• resource exhaustion

Mistake 4 — Ignoring connection pools#

Database connections are limited resources.

Mistake 5 — Storing important state only in local memory#

Horizontal scaling and instance replacement can lose it.

Mistake 6 — Using microservices automatically#

Microservices introduce:

• network failures
• deployment complexity
• distributed transactions
• observability requirements
• operational overhead

Mistake 7 — No graceful shutdown#

Deployments and scaling events can interrupt active requests.

Mistake 8 — No health checks#

A load balancer may continue sending traffic to unhealthy instances.

Mistake 9 — Ignoring backpressure#

Unlimited incoming work can exhaust resources.

61. Interview Questions#

Q1. What happens inside a server when an HTTP request arrives?#

A good answer:

Network stack
 ↓
Socket
 ↓
Server process
 ↓
HTTP parser
 ↓
Router
 ↓
Middleware
 ↓
Authentication/validation
 ↓
Business logic
 ↓
Cache/database/dependencies
 ↓
Response serialization
 ↓
HTTP response

Q2. What is the difference between a process and a thread?#

A process provides an isolated execution environment and memory space.

Threads execute within a process and generally share its memory.

Q3. Why are stateless servers easier to scale?#

Because any healthy instance can process the next request without requiring locally stored session state.

Q4. Why use a connection pool?#

To reuse expensive database connections and limit the number of concurrent database connections.

Q5. What happens if the connection pool is exhausted?#

Requests may wait for a connection and eventually timeout or fail.

Q6. Why use a reverse proxy?#

It can centralize TLS termination, routing, load balancing, caching, request policies, and other infrastructure concerns.

Q7. Monolith or microservices?#

The answer depends on requirements.

Consider:

• team size
• domain boundaries
• scaling requirements
• deployment independence
• operational maturity
• failure isolation
• transaction requirements

Q8. Why are stateless applications useful behind load balancers?#

Because the load balancer can send requests to any healthy instance without depending on a particular instance's local state.

Q9. What is graceful shutdown?#

Stop accepting new work, finish active work where possible, close resources, and then terminate.

Q10. What is backpressure?#

A mechanism for preventing incoming work from overwhelming downstream capacity.

62. Practical Exercise#

Design a server architecture for a simple e-commerce application.

Requirements:

• users
• products
• orders
• payments
• notifications

Start with:

Client
 ↓
Application
 ↓
Database

Then improve it to:

Client
 ↓
Load Balancer
 ↓
Application Servers
 ↓
Cache
 ↓
Database

Then add:

Queue
 ↓
Notification Workers

Identify:

1. Which components are stateless?
2. Which components hold state?
3. Which components can scale horizontally?
4. Where can bottlenecks occur?
5. What happens if the database becomes slow?
6. What happens if one application server crashes?
7. What happens if notification workers are overloaded?

63. Mini Project — Production Task Service#

Build a conceptual architecture:

                  Client
                    ↓
              Load Balancer
                    ↓
             ┌──────┴──────┐
             ↓             ↓
          App 1          App 2
             │             │
             └──────┬──────┘
                    ↓
                  Redis
                    ↓
               PostgreSQL
                    ↓
                  Queue
                    ↓
                 Workers

Implement or document:

• health endpoint
• readiness endpoint
• graceful shutdown
• database connection pool
• request timeout
• structured logging
• request ID
• background worker
• retry strategy
• error handling

64. Chapter Summary#

You should now understand:

Server Fundamentals#

• server as machine
• server as process
• ports
• sockets
• processes
• threads
• CPU cores
• concurrency
• parallelism

Application Infrastructure#

• web servers
• application servers
• reverse proxies
• workers
• worker pools
• thread pools
• connection pools

Application Architecture#

• request lifecycle
• routing
• middleware
• controllers
• services
• repositories
• layered architecture
• modular monolith
• monolith
• microservices

Scalability#

• horizontal scaling
• vertical scaling
• stateless application servers
• load balancers
• health checks
• graceful shutdown
• zero-downtime concepts

Reliability#

• backpressure
• load shedding
• graceful degradation
• timeouts
• resource limits
• connection pool saturation

65. The Big Picture#

We can now expand our mental model:

                         INTERNET
                            │
                            ▼
                       CDN / EDGE
                            │
                            ▼
                     LOAD BALANCER
                            │
                            ▼
                      REVERSE PROXY
                            │
                            ▼
                     APPLICATION
                            │
              ┌─────────────┼─────────────┐
              │             │             │
              ▼             ▼             ▼
           Routing      Middleware     Workers
              │             │
              └───────┬─────┘
                      ▼
                Business Logic
                      │
              ┌───────┼────────┐
              ▼       ▼        ▼
            Cache   Database   Queue
                               │
                               ▼
                            Workers

The critical insight is:

A production application is not simply "a server running code."

It is a collection of processes, resources, dependencies, and control mechanisms that must work together under load and failure.

66. Chapter 05 Master Checklist#

Before moving forward, make sure you can explain:

• What a server is
• Server vs server software
• IP address and port
• Socket
• Process
• Thread
• Process vs thread
• CPU core
• Concurrency
• Parallelism
• Web server
• Application server
• Reverse proxy
• Application process
• Node.js event loop concept
• CPU-heavy work and event-loop blocking
• Worker
• Worker pool
• Database connection pool
• Thread pool
• Request lifecycle
• Routing
• Middleware
• Controller
• Service layer
• Repository/data-access layer
• Layered architecture
• Monolith
• Modular monolith
• Microservices
• Horizontal scaling
• Vertical scaling
• Stateless application servers
• Health checks
• Readiness vs liveness
• Graceful shutdown
• Zero-downtime deployment concept
• Server bottlenecks
• Backpressure
• Load shedding
• Graceful degradation
• Synchronous vs asynchronous work

67. What Comes Next?#

Chapter 06 — Scaling Applications#

Now that we understand what happens inside an application server, we are ready for one of the central topics of System Design:

How do we make an application handle increasing traffic?

We will learn:

1 Server
   ↓
Vertical Scaling
   ↓
Multiple Servers
   ↓
Horizontal Scaling
   ↓
Load Balancing
   ↓
Stateless Architecture
   ↓
Caching
   ↓
Database Scaling
   ↓
Async Processing
   ↓
Distributed Architecture

We will cover:

• why applications need to scale
• traffic growth
• vertical scaling
• horizontal scaling
• stateless architecture
• scaling bottlenecks
• application-level scaling
• database bottlenecks
• cache-assisted scaling
• asynchronous scaling
• autoscaling
• capacity planning basics
• scale-up vs scale-out
• single-server limits
• scaling failure modes
• practical scaling strategies
• a complete application scaling case study

After Chapter 06, we will move into Chapter 07 — Load Balancers, where we will deeply study how traffic is distributed across multiple application servers.