Lecture 3: Parallel Programming in Python

Lecture 3 introduces parallel programming in Python, focusing on practical steps using threads and processes [1-3].

Key aspects covered in the lecture:

• Python for Parallel Programming:

• Python is a versatile, high-level language with rich libraries, making it widely used in the industry, though its abstraction means it is less optimized for performance and lacks access to bare metal [2, 4].

• Python relies on interpreters like CPython, which uses a Global Interpreter Lock (GIL) that allows only one thread to execute at a time, thus limiting true multithreading [5, 6].

• Parallelism in Python is achieved through multithreading, multi-processing, and heterogeneous computing, which operate outside the GIL [3].

• Threads in Python:

• Threads divide a program's control flow into concurrently running streams, sharing resources but operating independently [7, 8].

• Managing threads requires careful synchronization to avoid conflicts when accessing shared data [8].

• An example is provided to demonstrate creating and starting threads using the threading module, where each thread executes a function concurrently [9, 10].

• A basic problem is presented involving generating and joining random letters, as well as adding random numbers, to be executed using threads [11, 12].

• Sample code is provided to create a thread for each function (joining letters and adding numbers), run them, and time their execution [13, 14].

• Processes in Python:

• Processes are independent programs, each with its own memory space, providing isolation and stability [14, 15].

• Unlike threads, processes do not share memory directly, requiring explicit Inter-Process Communication (IPC) mechanisms for data exchange [15].

• Processes demand more system resources than threads [15].

• An example demonstrates creating and starting processes using the multiprocessing module [16, 17].

• A similar problem to the one using threads is presented, but this time using processes [18].

• Sample code is given to create a process for each function, run them, and measure the execution time [19, 20].

• Threads vs. Processes:

• Threads share memory, are computationally less expensive to start/change, require fewer resources, and need synchronization mechanisms [21].

• Processes do not share memory, are computationally expensive to start/change, require more resources, and do not need memory synchronization [21].

• Performance Evaluation:

• The lecture mentions computing speedup and efficiency using Amdahl’s Law and Gustafson’s Law for both threads and processes [22, 23].