Week 1

Parallel Programming

  • Process of breaking down larger problems into smaller ones that can be solved simultaneously

Data parallelism

  • Same set of instructions applied on different inputs

  • Most suitable for GPUs - good for large data sets

Task parallelism

  • Different set of instructions, same or different data


SIMD

  • Instructions sent to different processors - GPU


MIMD

  • CPU good for task parallelism


Shared Address Space

  • Uniform Memory Access

    • Can have multiple cores trying to find data in mm - trying to ask different data to same main memory and create queue

    • Every core in theory should take the same time to reach main memory

    • start having queue

  • Non uniform

    • Have cores have access to certain parts of main memory

    • core associated with part of shared address

    • have distributed shared memory (interconnected)


create a grid of threads

every cudda kernel has access to special variables that can determine global position