Week 1
Parallel Programming
Process of breaking down larger problems into smaller ones that can be solved simultaneously
Data parallelism
Same set of instructions applied on different inputs
Most suitable for GPUs - good for large data sets
Task parallelism
Different set of instructions, same or different data
SIMD
Instructions sent to different processors - GPU
MIMD
CPU good for task parallelism
Shared Address Space
Uniform Memory Access
Can have multiple cores trying to find data in mm - trying to ask different data to same main memory and create queue
Every core in theory should take the same time to reach main memory
start having queue
Non uniform
Have cores have access to certain parts of main memory
core associated with part of shared address
have distributed shared memory (interconnected)
create a grid of threads
every cudda kernel has access to special variables that can determine global position