Vertical vs Horizontal Scaling

System Design FundamentalsEntrysystem-design

The Question

What is the difference between vertical and horizontal scaling?

What a Strong Answer Covers

• vertical = bigger machine
• "horizontal = more machines
• SPOF.

Senior-Level Answer

Scaling is how you increase a system's capacity to handle more load. The two fundamental approaches differ in *where* you add resources.

**Vertical scaling** (scaling up) means adding more resources to an existing machine — more CPU cores, more RAM, faster disks, or a bigger network card. It is operationally simple: you upgrade the hardware or VM size, and the application often needs no changes. However, it has a hard ceiling — there is a maximum machine size, and at the high end, the cost grows super-linearly (a machine with 4x the RAM often costs more than 4x). Crucially, a single larger machine is still a single point of failure, so vertical scaling does not improve availability.

**Horizontal scaling** (scaling out) means adding more machines and distributing load across them. This is the foundation of cloud-native architecture. It provides effectively unlimited capacity (add nodes as needed), and multiple nodes mean that a single node failure does not take down the service — improving fault tolerance. The trade-offs are complexity: stateless services scale horizontally almost trivially, but stateful services (databases, session stores) require careful partitioning (sharding), consistent hashing, or consensus protocols.

For stateless application tiers (web servers, API servers), horizontal scaling is the default choice. Auto-scaling groups or Kubernetes HPA can add and remove instances based on CPU or request rate, making it cost-efficient and resilient. For databases, the traditional path is to scale vertically first (because it requires no application changes), then scale out reads with replicas, and finally shard writes horizontally only when necessary — sharding adds significant complexity to queries and transactions.

In practice, most large systems use a combination. The database primary is sized vertically to its practical maximum, reads are distributed across horizontal replicas, and the stateless application layer scales horizontally with auto-scaling. The choice at any layer depends on the marginal cost of the next vertical upgrade vs. the operational cost of adding another node.

The key insight is that horizontal scaling requires the application to be designed for it — shared-nothing architecture, externalized session state, idempotent operations, and distributed tracing become necessary.

Key Differences

What Separates a 2/3 from a 3/3

Common Mistakes

• Claiming vertical scaling improves availability — it does not; you still have a single point of failure.
• Treating horizontal scaling as free — it requires stateless design, load balancing, and distributed state management.
• Not knowing that databases typically scale vertically first before horizontal sharding.
• Confusing horizontal scaling with replication — read replicas improve read throughput but do not scale writes.

Follow-Up Questions

• How does Kubernetes Horizontal Pod Autoscaler decide when to add pods? — HPA watches metrics (CPU utilization, custom metrics via Metrics Server or Prometheus adapter) and scales replica count to maintain a target utilization ratio.
• What makes a service 'stateless' and why does it matter for horizontal scaling? — No in-process state that must be co-located with a specific instance — sessions in Redis, not in memory. Any instance can handle any request.
• When would you choose to shard a database rather than continue scaling vertically? — When write throughput or dataset size exceeds single-node capacity, or when vertical cost curve exceeds sharding operational cost. Sharding adds query routing complexity.
• What is the trade-off between adding more small nodes vs. fewer large nodes in a distributed system? — More small nodes: better fault isolation, cheaper per node. Fewer large nodes: less network overhead, simpler to manage, but blast radius of a node failure is larger.

Related Questions

• Redis Caching Patterns
• API Versioning
• SLOs vs SLAs
• Availability — Five 9s
• Latency vs Throughput