Comprehensive Study Notes on Distributed Systems, Architectural Models, and Processing Techniques

Overview of Distributed Systems and Real-World Examples

• Primary Purpose: Distributed systems are implemented to ensure large-scale data handling, improve system performance, and guarantee overall reliability.
• Real-World Examples:
  • Online Banking Systems: Banks utilize distributed systems where local branches and ATMs share data with central servers while maintaining independent operations. This structure ensures that transactions can continue without interruption.
  • E-Commerce Platforms: Platforms such as Amazon or Flipkart distribute the processing of orders, payments, and inventory management across multiple servers. This is necessary to handle high volumes of traffic and provide a smooth user experience.
  • Social Media Platforms: Applications like Facebook, Twitter, and Instagram employ multiple data centers to store and process massive amounts of user-generated data efficiently.
  • Online Gaming Systems: Multiplayer games, specifically naming PUBG, distribute game states across multiple servers to reduce latency and ensure a real-time gaming experience for players.

Types of Distributed Systems

Distributed systems are classified based on node organization, communication methods, and the distribution of tasks across the system.

• Client-Server Systems:
  • A centralized distributed system where a central server provides services, and multiple clients request those services over a network.
  • Roles: The Server manages data and processing, while Clients send requests and receive the processed responses.
  • Example: Gmail or online banking systems.
• Peer-to-Peer (P2P) Systems:
  • A system where all nodes are equal and can function as both client and server simultaneously without a central authority.
  • Characteristics: No central server exists; nodes share resources directly with one another.
  • Example: BitTorrent, blockchain networks.
• Clustered Systems:
  • Consists of a group of closely connected computers that work together as a single unified system to enhance performance and reliability.
  • Characteristics: Nodes operate in a tightly connected network; tasks are shared for high performance.
  • Example: Google search clusters, supercomputers.
• Cloud-Based Distributed Systems:
  • These use cloud infrastructure where computing resources are distributed across multiple data centers and accessed via the internet.
  • Characteristics: Resources are scalable on demand; services are internet-accessible.
  • Example: AWS (Amazon Web Services), Microsoft Azure, Google Cloud.

Operational Mechanics of Distributed Systems

• Node Independence: Each node runs its own specific application and maintains its own local data.
• Communication: Nodes interact through the network using specialized protocols or middleware services.
• Workload Distribution: Data and tasks are spread across nodes to facilitate parallel processing.
• Local Processing: Nodes process data locally and then share the results with other nodes as required.
• Distributed Storage: Data may be stored in distributed databases rather than a single centralized repository.
• System Objectives: The system is designed to ensure coordination, consistency, and fault tolerance across every node.

Distributed Applications: Architecture and Strengths

A distributed application is a software program whose components run on multiple, independent computers connected through a network.

• How it Works: Unlike a traditional monolithic software (which exists entirely on a single machine), distributed applications break processes into loosely coupled, communicating parts.
  • Client Side: This is the interface the user interacts with locally, such as a web browser or mobile application.
  • Server/Backend Side: This consists of the database, processing engines, or microservices operating across remote servers, cloud platforms, or P2P networks.
• Key Advantages:
  • Scalability: Supports horizontal scaling, which allows for adding more machines to handle traffic spikes instead of just upgrading a single server.
  • High Availability & Fault Tolerance: If one node or computer crashes, others take over to prevent widespread downtime.
  • Optimal Resource Usage: Developers can utilize different operating systems or architectures based on the specific strengths of each.
• Contemporary Examples:
  • Cloud & Web Services: Large-scale systems like Netflix, Amazon, or Google Workspace.
  • Generative AI Models: Systems like ChatGPT process massive streams of token generation by distributing heavy computations across thousands of networked GPUs.
  • Decentralized Applications (dApps): Programs operating on blockchain or P2P networks (like Ethereum) rather than centralized servers.

Decentralized Applications (dApps) and Smart Contracts

• Definition: Software programs that run on a P2P blockchain network rather than a centralized server. They offer enhanced privacy, censorship resistance, and direct control over assets.
• The Architecture of dApps:
  • Smart Contracts: These represent the "backend" of a dApp. They are self-executing programs deployed on blockchains like Ethereum, Solana, or Polygon. They execute automatically once predetermined conditions are met without human intervention.
  • Front End: The user interface looks like a standard site but communicates directly with blockchain smart contracts.
  • No Middlemen: Users interact via self-custodial crypto wallets (e.g., MetaMask), and transactions are settled peer-to-peer.

Middleware in Distributed Systems

Middleware is necessary to provide reusable, standard solutions for common distributed challenges, preventing developers from having to write custom logic for every component.

• Core Functions:
  • Heterogeneity: Allows systems on different hardware, operating systems, or languages to work together.
  • Transparency: Hides the physical distribution of the system so it appears to the user as a single coherent application.
  • Interoperability: Facilitates data exchange between otherwise incompatible platforms.
• Common Types of Middleware:
  • Remote Procedure Calls (RPC): Allows an application to execute a function on a remote server as if it were local.
  • Message-Oriented Middleware (MOM): Uses message queues and brokers for asynchronous data exchange; components don't need to be active simultaneously.
  • Object Request Brokers (ORB): Enables distributed objects to interact across languages and systems (e.g., CORBA).
  • Application Servers & Web Middleware: Manages HTTP requests, business logic, user sessions, and API routing.

Advantages and Disadvantages of Distributed Systems

Advantages

• Resource Sharing: Nodes share data, files, and hardware, reducing overall costs.
• Scalability: Systems handle increased workloads by adding more nodes without major architectural changes.
• Reliability and Fault Tolerance: Failure of a single node does not crash the whole system.
• Performance: Improved processing speeds by distributing workloads across multiple nodes.

Disadvantages

• Complexity: Designing, managing, and maintaining these systems is significantly harder than centralized ones.
• Security Challenges: Multiple nodes increase the risk of unauthorized access and cyberattacks.
• Network Dependency: Performance relies heavily on network speed, latency, and reliability.
• Data Consistency: Synchronizing data across multiple nodes in real-time is difficult.

Architectural Models

An architectural model is the overall design and structure defining how components interact and how development, deployment, and operations take place.

• Client-Server Model: A centralized approach using the request-response model. The client initiates a request, and the server processes it. It often uses TCP/IP and HTTP protocols on the transport layer. Common in web services and database management.
• Microservices Architecture:
  • An application is divided into small, independent services communicating over a network.
  • Instead of one tightly coupled codebase, each service handles a specific function (e.g., authentication vs. payment).
  • Services are loosely coupled and can be written in different programming languages.

Real-World Microservices Case Studies

• Amazon: Originally a monolithic application, Amazon shifted to microservices early on to allow for individual feature updates.
• Banking & FinTech: Uses independent services for accounts, transactions, fraud detection, and support to ensure compliance and reliability.
• Netflix: Adopted microservices after facing service outages in $2007$ while transitioning to movie streaming. This improved performance and reliability.
• Uber: Switched from a monolithic structure to microservices to increase webpage views and search efficiency.
• Healthcare: Separates patient records, scheduling, and billing to improve data management and scalability.

Concurrency vs. Parallelism

Concurrency

• Definition: Handling multiple tasks by sharing a single processing resource without true simultaneous execution.
• Mechanism: Tasks execute in overlapping time periods using interleaving or context switching by the CPU. This creates an "illusion of parallelism."
• Example: A single-core CPU running multiple threads. If Task 1 goes to an I/O stage, Task 2 starts executing, followed by Task 3, before Task 1 returns to finish.

Parallelism

• Definition: Executing multiple tasks simultaneously using multiple processing units.
• Mechanism: Tasks are split into smaller subtasks (e.g., $P1$ to $P5$) that run on separate cores.
• Example: A quad-core CPU running four threads simultaneously. A task that takes $5$ minutes on a single processor can complete in $1$ minute if divided into five parts and run on separate processors.

Comparative Analysis