ITT 440 Network Programming - Chapter 8: Parallel Computing Notes

Introduction to Parallel Computing

  • Clock Rate Evolution:
    • 1988: Computer processors had a clock rate of 40 MHz.
    • 2002: Clock rates increased significantly to 2.0 GHz.
  • Parallel Processing Defined: Processing program instructions by dividing them among multiple processors to reduce the execution time.

Serial vs. Parallel Processing

  • Serial Processing: Instructions are executed sequentially by a single processor.
  • Parallel Processing: The problem is divided into smaller tasks and distributed across multiple processors, enabling concurrent execution.

Clock Speed Plateau

  • Clock Speed Trends:
    • 2005: Flagship Intel desktop processors reached 3.8 GHz.
    • 2016: Clock speeds slightly decreased to 3.2 GHz.
  • Reasons for Plateau:
    • Smaller transistor size and increased transistor count on CPUs lead to current leakage and thermal efficiency issues.
  • The Way Forward: Multicore processors and efficient processing techniques are the solution.

Multicore Processors

  • The number of cores in CPUs continues to increase to enhance parallel processing capabilities.

Processor Architectures

  • Intel x86 CISC Core i5 6500 Skylake Architecture:

  • AMD x86 CISC Ryzen 5 1600.

  • Apple M1 RISC: A power-efficient System on a Chip (SoC) which may represent the future of processor design.

Motivating Parallelism

  • Traditional Challenges:
    • Complexity in specifying and coordinating concurrent tasks.
    • Lack of portable algorithms, standardized environments, and software development toolkits.
  • Arguments for Parallel Computing: Parallel computing can significantly improve computing power.

Computational Power Argument

  • Moore's Law: The number of transistors and floating point operations per second (FLOPS) continues to increase.
  • MOSFET CPU Manufacturing Technology: Advances in MOSFET technology drive improvements in CPU performance.

Memory/Disk Speed Argument

  • Overall Speed of Computation: Depends on both the processor speed and the system memory's ability to feed data.
  • Mismatch: There exists a mismatch between processor speed and data latency.
  • Bandwidth: The bandwidth between DRAM and processors is a critical factor.

Data Communication Argument

  • Internet as a Parallel/Distributed Computer: The vision of using the internet as a large-scale parallel computing platform.
  • SETI Project: An example of utilizing a large number of home computers to analyze signals from outer space.

Scope of Parallel Computing

  • Applications in Engineering and Design: Used for analyzing airfoils, internal combustion engines, high-speed circuits, etc.
  • Scientific Applications: Employed in sequencing the human genome and analyzing biological sequences.
  • Commercial Applications: Utilized in Wall Street for large-scale transactions and data mining.
  • Applications in Computer Systems: Found in computer security and embedded systems.