System Design Fundamentals

Software Architecture

Software Architecture is the high-level structure of a software system and the way its major components communicate and work together.

Functional Requirement

Functional requirements describe what the system must do.

They describe the features and behaviors of the system.

Non-Functional Requirement.

Non-functional requirements define the quality, performance, reliability, scalability, security, and operational characteristics of a system.

Functional vs Non Functional

Functional = WHAT?
Non-Functional = HOW WELL?

Important NFRs

• Scalability

• Availability

• Reliability

• Performance / Latency

• Throughput

• Security

• Durability

• Maintainability

1. What does the system need to do?

2. How well does it need to do it?

3. How should we structure the system to achieve both?

1. Functional Requirements

2. Non-Functional Requirements

3. Software Architecture

Scalability

The ability of a system to handle increasing load by adding resources while maintaining acceptable performance.

Load

Load is the amount of work coming into your system.

As load increases, the system needs more capacity.

Scalability vs Performance

Scalability - How well does the system continue to perform as workload increases?

1K users     → 50 ms
1M users     → 100 ms

Performance - How fast can the system handle a given workload?

Request → Server → Response --- 50 ms

Main Types of Scaling

Vertical Scaling — Scale Up

Horizontal Scaling — Scale Out

Vertical Scaling

Make one machine more powerful. By increasing CPU, RAM, Disk, Network capacity, etc.,

Vertical Scaling - Advantages

• Simple

• Less architectural complexity

• Usually easier to implement

Vertical Scaling - Disadvantages

• Hardware has limits

• Can become expensive

• Single point of failure may remain

• Doesn't scale indefinitely

Horizontal Scaling

Instead of making one server bigger, add more servers.

Horizontal Scaling - Advantages

• Can scale much further

• Better fault tolerance

• Easy to add/remove machines

• Very useful for cloud systems

Horizontal Scaling - Disadvantages

• More complexity

• Requires load balancing

• Data consistency becomes harder

• Distributed-system problems appear

Stateless Servers

The server doesn't store user-specific session information locally. Any server can handle the request. This makes horizontal scaling easy. For modern distributed systems, Prefer stateless application servers whenever practical.

Stateful Servers

The server keeps important user state locally. You may need: Shared session storage, Redis, Database, Sticky sessions

Database Scalability

Database scalability is the ability of a database system to handle growing amounts of data, user traffic, and transaction requests without losing performance or availability. Application servers aren't the only thing that needs scaling. The database can become the bottleneck.

Database Scalability - Vertical Scaling

Make the database server bigger. Simple, but eventually limited.

DB
4 CPU → 32 CPU
8 GB  → 128 GB

Database Scalability - Read Replicas

Create copies of the database for read operations.

Auto Scaling

Cloud systems can automatically increase/decrease resources based on demand.

Scalability vs Elasticity

Scalability:

Can the system handle growth?

Elasticity:

Can the system automatically adjust capacity as load changes?

Bottlenecks

A system is only as scalable as its bottleneck.

Common bottlenecks:

CPU, Memory, Database, Disk I/O, Network, Lock contention, Connection pools, External APIs, Single-threaded components

Find the bottleneck before adding more resources.

Scale Up vs Scale Out

The Scalability Toolkit

                 SCALABILITY
                      │
       ┌──────────────┼──────────────┐
       ↓              ↓              ↓
    Compute         Database       Content
       │              │              │
  ┌────┴────┐    ┌────┴────┐       CDN
  ↓         ↓    ↓         ↓
Scale Up  Scale Out
                Read      Sharding
              Replicas
                │
              Cache

If an interviewer asks "What is scalability?"

Scalability is the ability of a system to handle increasing workload by adding resources while maintaining acceptable performance. We can scale vertically by increasing the capacity of a machine, or horizontally by adding more machines. In distributed systems, horizontal scalability is commonly achieved using load balancers, stateless services, caching, database replication/sharding, CDNs, asynchronous processing, and auto-scaling.

What is Reliability?

Reliability is the ability of a system to perform its intended functions correctly and consistently over a specified period, even when failures may occur.