Pool Of Workers and Boss-Worker Model

Queue Structure and System Performance

• The performance of the system using a queue structure is contingent upon the boss thread's ability to insert work requests into the queue without waiting.

  • If the queue is full, the boss must wait, resulting in increased time per order and decreased overall throughput.

• Increasing the number of threads can reduce the likelihood of the queue being full, but excessive thread creation introduces system overheads.

Determining the Number of Workers

• The central question for system performance is determining how many worker threads are adequate.

  • Dynamic Worker Addition: Workers can be added dynamically upon receiving new orders.

  • This method can be inefficient, particularly if there is a prolonged wait for a new worker thread to become available.

  • Pre-created Worker Pool: A more efficient strategy is to establish a pool of workers before orders begin to arrive, minimizing wait times.

Worker Pool Management

• The pool of workers (or threads) eliminates the need to create a new thread each time an order is placed, allowing for immediate response as orders accumulate in the queue.

• Decision on Pool Size:

  • Common approach is to initialize the worker pool size but also provide for dynamic resizing as needed.

  • Unlike purely on-demand models, this technique can adjust the pool size in increments, creating several threads at once when necessary.

• This enhances efficiency in managing threads within the boss-worker pattern.

Summary of the Boss-Worker Model

• Main features of the boss-worker model:

  • A designated boss assigns work to all worker threads.

  • Each worker performs the entire task assigned to them independently.

  • Communication occurs through a shared producer-consumer queue.

  • A worker pool is employed to manage threads flexibly, allowing for size adjustment based on current demand.

Benefits and Drawbacks of the Boss-Worker Approach

• Benefits:

  • Simplicity in design where a single thread (the boss) manages work distribution to all other threads, which perform the same task.

• Drawbacks:

  • Overheads associated with managing the thread pool, including synchronization for the shared buffer, may introduce inefficiencies.

  • Loss of locality in task management, where the boss is unaware of individual worker performances.

  • Workers who complete similar tasks may develop efficiency or have necessary tools at hand, but without proper tracking, the boss cannot optimize task assignment effectively.