Distributed systems

Distributed System (textbook definition)

A collection of independent, autonomous hosts connected through a communication network that communicate and coordinate their actions exclusively by passing messages.

No shared memory (architectural constraint)

Processes in a distributed system must use the network stack for all communication; there is no shared memory between hosts.

No shared clock (architectural constraint)

There is no single global "tick" to perfectly synchronize events across a distributed system.

No shared operating system (architectural constraint)

Nodes in a distributed system typically run their own local OS instances (e.g., Linux, Windows).

Tanenbaum & Van Steen's Definition of Distributed System

A collection of autonomous computing elements, connected by a network, which appear to its users as a single coherent system.

Autonomous (in distributed systems)

Nodes fail independently of one another.

Coherent (in distributed systems)

Users should not need to know which machine handled their request; the system presents a unified interface.

Leslie Lamport's View of Distributed Systems

"You know you have one when the crash of a computer you have never heard of stops you from getting any work done." Highlights the reality of partial failure and the need for state consistency.

Node (distributed systems)

An independent processing entity with local memory; in modern environments often a virtualized instance or container.

Link (distributed systems)

A communication channel that facilitates message passing between nodes; unlike internal bus lines, links are unreliable (latency, packet loss, partitions).

Motivations for Distribution

Resource Sharing (hardware/software/data access), Collaboration (sharing info across distances), and Inherent Distribution (systems naturally spread out, e.g., national ATM networks).

Advantage: Performance/Price of Distributed Systems

Microprocessors (distributed) have a low, cost-effective performance/price ratio compared to mainframes (centralised), which have a high, expensive ratio.

Advantage: Total Power of Distributed Systems

Distributed systems can achieve high total power (e.g., 10,000 CPUs) whereas mainframes are limited by single-chip cycle speed.

Advantage: Higher Availability

Achieved through redundancy (multiple server nodes and storage devices) so the system survives even if individual components fail.

Advantage: Incremental Growth

The ability to "scale out" by adding resources gradually, avoiding disruptive "forklift upgrades" required in centralized systems.

Concurrency (core characteristic)

In distributed environments, autonomous components execute simultaneously, handling separate tasks/requests across machines; requires managing race conditions and resource contention.

Lack of a Global Clock (core characteristic)

Because there is no global "correct time," local timestamps can't order events across machines; logical or vector clocks are used to determine causality.

Independent Failures / Partial Failure

Unlike centralized systems with a Single Point of Failure (SPOF), distributed systems face partial failure — one node may be down while others remain healthy; software must be "failure-aware."

Heterogeneity (core challenge)

A distributed system must bridge differences in hardware (ARM vs x86), operating systems, networks (protocols/media), programming languages, and data formats.

Middleware

The software layer that sits above the OS to mask heterogeneity; often implemented as API Gateways, Message Brokers, or Service Meshes (e.g., Istio, Linkerd, RabbitMQ).

Security (core challenge in distributed systems)

Mechanisms/frameworks that protect data in transit and validate server identities so clients aren't communicating with malicious nodes (TLS, Authentication, Authorization, Certificates).

Failure Detection

Using heartbeats or timeouts to find failed nodes.

Failure Masking

Hiding failures by retrying requests or using redundant replicas.

Failure Recovery

Bringing a failed node back into a consistent state with the rest of the cluster.

Transparency (distributed systems)

The critical concept of concealing the physical distribution of processes and resources so users experience the system as if it were local.

Access Transparency

Hides differences in data representation and how resources are accessed; read local and remote files similarly (e.g., RMI/RPC, NFS).

Location Transparency

Hides where a resource is physically located; user uses a URL, not an IP address.

Migration Transparency

Allows resources to move without user impact (e.g., moving a service between cloud regions).

Replication Transparency

Hides that a resource is copied for performance or reliability (e.g., multiple database copies).

Concurrency Transparency (type)

Hides that multiple users are accessing the same resource simultaneously (e.g., many users editing at once).

Failure Transparency (type)

Hides that a node crashed and was recovered in the background (e.g., request succeeds despite a node crash).

Vertical Scaling (Scale Up)

Adding more CPU or RAM to a single machine; has a physical and financial ceiling.

Horizontal Scaling (Scale Out)

Adding more nodes to the system; the preferred method for modern web-scale applications, offering theoretically infinite growth.

Scalability (Load and Capacity)

Measures how well a system handles increased load by increasing total capacity; in a well-architected system, 2x nodes should yield roughly 2x throughput.

Bottleneck (scalability)

A centralized component (e.g., single auth server, single non-partitionable SQL database) that limits scaling and represents a Single Point of Failure (SPOF).

Client-Server Architecture

Foundational asymmetric model: the Client actively initiates requests (user-facing); the Server passively listens, manages resources, and responds.

Multi-tier (N-tier) Architecture

Decomposes a system into logical layers: Presentation Tier (UI), Application Logic Tier (business rules), and Data Tier (persistent storage).

Peer-to-Peer (P2P) Architecture

A decentralized model where every node acts as both client and server (a "servent"); no central authority, resilient to censorship and traffic spikes (e.g., BitTorrent, Blockchain).

Microservices Architecture

The system is decomposed into small, independent services that communicate via lightweight protocols (e.g., REST); each can use a different language, be deployed independently, and scaled based on its own load.

Cloud-Native Architecture

Uses virtualized infrastructure on demand (containers, Kubernetes, service discovery, auto-scaling) to provision nodes quickly and handle scaling requirements.

Inter-Process Communication (IPC)

Communication between processes over the network; the exchange of data between two or more separate, independent processes/threads — it's what makes distributed systems distributed.

IPC "Island Analogy"

Each process is an island with its own address space, code, and data; the OS acts as a bridge, and the API is the "bridge deck" enabling high-level IPC programming.

Reasons Processes Cooperate

Information Sharing (e.g., banking database), Computation Speedup (e.g., Google Search), Modularity/Reliability (e.g., Netflix microservices), Convenience (e.g., WhatsApp messaging).

Shared Memory Model (IPC)

Processes communicate via a common memory region. Pros: fast/low latency, efficient data sharing, simple to manage. Cons: complex synchronisation, scalability limits, security/access control issues.

Message Passing Model (IPC)

Processes communicate by sending/receiving messages. Pros: loose coupling, safety from concurrency bugs, fault isolation. Cons: network delays, message loss, serialization overhead.

Send(destination, message)

Basic messaging operation: sends a message to a destination process.

Receive(source, buffer)

Basic messaging operation: receives a message from a source and stores it in a buffer.

Connect() / Disconnect() (messaging)

Additional messaging operations: Connect() establishes a logical communication channel between processes; Disconnect() terminates the channel and releases resources. Not all systems need explicit connections (e.g., UDP is connectionless; TCP/gRPC are connection-oriented).

Host Address (IP)

Identifies the machine on the network.

Port Number

Identifies the specific service (process) on a machine; a port typically has one receiver but can accept messages from many senders.

Service Name

A human-friendly name that maps to a port (e.g., http, smtp, ssh).

URL (higher-level addressing)

Combines protocol, host, port (optional), and path to identify a service (e.g., https://api.example.com:443/orders).

Message Header

Contains metadata about a message: message type, source/destination ID, length, sequence number, priority, checksum.

Message Body

The actual content or payload being transmitted in a message.

Direct Communication

The sender communicates directly with the intended receiver; sender must know the receiver's identity/address; typically tightly coupled and synchronous (e.g., HTTP, gRPC, TCP).

Indirect Communication

Processes exchange messages through an intermediary (message broker, queue, event bus); sender and receiver don't need to be online simultaneously; loosely coupled, usually asynchronous, high scalability/fault tolerance (e.g., Kafka, RabbitMQ, JMS).

Unicast

One sender sends a message to exactly one receiver (e.g., HTTP request, TCP socket).

Multicast

One sender sends a message to multiple interested receivers (e.g., service discovery, replicated systems, event notifications).

Why Multicast Matters

Used for Replication (updating multiple servers), Service Discovery, Event Distribution (updates to many subscribers), and Fault Tolerance (keeping replicas consistent).

Synchronous Communication

Sender is blocked, waiting for a reply, from the moment the call is sent until the response is received (receiver also blocks while processing).

Asynchronous Communication

Sender's call is non-blocking; sender is free to do other work while waiting; the reply/response is handled later (e.g., via callback or polling).

Indefinite Blocking (failure mode)

A process might "hang" forever if a network link fails during a blocking call.

Timeout (failure handling)

Returns a special error code if a process blocks for too long, allowing recovery logic to run.

Deadlock (IPC)

Occurs when blocking operations are issued in the wrong sequence, e.g., Process 1 waits for Process 2 while Process 2 waits for Process 1.

Key IPC Design Questions

Direct or indirect communication? Synchronous or asynchronous? Buffered or unbuffered? Fixed-size or variable-size messages? These choices affect performance, scalability, and reliability.

Buffering (messages)

Allows messages to be stored temporarily, delivered later, and processed asynchronously; benefits include absorbing traffic bursts, improving performance, and decoupling sender/receiver — at the cost of extra memory usage.

The Representation Problem

Different computer architectures represent data differently (e.g., Intel x86 vs AMD64 vs ARM), which complicates transmitting high-level data structures as network bytes.

Byte-ordering (Endianness)

A heterogeneity challenge: Big-endian (e.g., UNIX SPARC) vs Little-endian (e.g., Wintel) systems represent multi-byte data differently.

Marshalling

The process of flattening a structured, in-memory data object into a standard external format (e.g., JSON) so it can be transmitted as a sequence of bytes across the network.

Unmarshalling

The process of rebuilding the original structured data object from the standard-format bytes received over the network.

JSON (serialization format)

Text format that is human readable and easy to debug, but produces larger messages.

XML (serialization format)

Text format that is self-describing but verbose.

Protocol Buffers / Apache Avro

Binary serialization formats that produce smaller messages and enable faster processing than text formats like JSON/XML.

Transport Layer (in IPC)

Provides end-to-end communication between applications on different hosts, creating a reliable communication channel on top of IP; uses port numbers to address specific processes/services.

TCP (Transmission Control Protocol)

Connection-oriented, reliable byte-stream protocol.

UDP (User Datagram Protocol)

Connectionless, "best-effort" datagram service — fast but unreliable.

How TCP Achieves Reliability

Segmentation (splits large messages into segments), Sequence Numbers (unique ID per segment), Acknowledgements (three-way handshake: SYN, SYN-ACK, ACK), and Retransmission (if no ACK arrives before timeout).

HTTP (as Application Layer IPC)

Operates at Layer 7 (Application) of the OSI model; uses a request-response pattern over TCP; commonly used to expose REST APIs in microservices and web applications.

HTTP Methods (CRUD)

POST = Create/Insert, GET = Read/Select (never changes resource), PUT = Update/Override existing resource, DELETE = Remove resource.

Common HTTP Status Codes

200 OK (success), 201 Created (success for POST), 400 Bad Request (missing parameters), 401 Unauthorized (missing authentication), 404 Not Found (resource doesn't exist), 500 Server Error (server-side problem).

RESTful Services

Resource-oriented communication where resources are identified by URLs; principles include Stateless, Cacheable, Scalable, and Platform Independent.

API Gateway

A single entry point for all client requests that routes them to appropriate microservices; handles authentication, authorization, rate limiting, logging, caching, and request routing.

Publish-Subscribe (Messaging)

Publishers send events to a message broker (e.g., Apache Kafka) organized by topics; subscribers receive events. Decouples producers and consumers; scalable, resilient, ideal for event-driven architectures (e.g., Kafka, RabbitMQ, Azure Service Bus).

Remote Invocation

A mechanism for executing functions on a remote machine that gives the illusion the function executes locally; a high-level abstraction built on top of message-passing (IPC).

IPC vs Remote Invocation

IPC is a low-level mechanism for exchanging data via message passing (send/receive). Remote Invocation (RPC/RMI/gRPC) is a high-level abstraction built on IPC that lets a process invoke a method on another machine as if it were local.

Remote Procedure Call (RPC)

An extension of conventional procedure calls that allows a program to execute a procedure on a remote machine as if it were a local function call, hiding networking details (sockets, formatting) under the hood.

Key Benefits of RPC

Simplifies distributed application development, promotes code reusability, provides language/platform independence (with proper IDL), and improves maintainability and productivity.

Client Stub

A client-side proxy that receives client requests, marshals the procedure name/arguments, sends the request to the server, receives the response, unmarshals the result, and returns it to the client.

Server Stub

A server-side proxy that receives client requests, unmarshals incoming parameters, invokes the appropriate procedure, receives the execution result, marshals the return value, and sends the response to the client.

External Data Representation (XDR)

A standard format used to ensure data integrity during transmission between machines that represent data differently; key idea is converting local data into a portable format before transmission (e.g., Protocol Buffers in gRPC).

Binding (Remote Invocation)

The process of determining the location and identity of a remote procedure, allowing the client to obtain a service handle, network address, or endpoint — like looking up a contact in a phone directory before calling.

Static Binding

The server's address is hardcoded into the client at compile time; efficient but inflexible.

Dynamic Binding

The server registers its interface with a Binder (naming/directory service); the client queries the Binder to "look up" the service handle before making the call.

Service Contract

An agreement between client and server on how they will understand each other, typically defined using an Interface Definition Language (IDL).

Interface Definition Language (IDL)

A language-independent way to specify procedure names, input parameters, and return types; an IDL compiler (stub generator) automatically creates client and server stubs (e.g., .proto files in gRPC/Ballerina).

Synchronous Remote Call

The client thread blocks and waits for the response; simple to program but inefficient since waiting wastes client thread resources; good for short operations/low concurrency.

Asynchronous RPC

The client does not block; it continues other work while the response is delivered later via a callback/event; better resource utilization and higher concurrency, but requires an asynchronous programming model.

Deferred Synchronous RPC

The client continues other work immediately after the request, then later polls or waits for the result; simpler than fully asynchronous (no callbacks) but still more efficient than pure blocking.

gRPC Timeout / Deadline

A time limit set by the client for an RPC call to complete; if the server doesn't respond in time, the client aborts the call and returns an error (e.g., DEADLINE_EXCEEDED).

Retry Mechanism (gRPC)

The client automatically retries a failed RPC request to overcome transient issues (network blips, temporary unavailability), using a retry policy (max attempts, delay, backoff multiplier) and retryable status codes.