parallel exam 2

head node

1 per cluster, deals with job submission, cluster management

compute node

1+ per cluster, deals with job execution

job

unit of work in hpc cluster (set of tasks)

task

executable program

job life cycle

create, queue, run, fail/cancel/finish

quest cluster

~1000 Nodes (computers)

28-64 Cores (CPU/processors)

Tech specs vary (several generations)

applications of hpc clusters

In theory: can run any app you want, compute nodes can run any OS you want

In reality: apps have the following characteristics

non-interactive, long-running, compute-intensive, data-intensive

SLURM

Simple Linux Utility for Resource Management

SLURM commands

sftp - move files to/from Quest

sbatch - submit

squeue - monitor

node

computer

socket

CPU chip

core

independent execution chip

map-reduce

Approach for analyzing big data

Massively parallel, fault tolerant, easy to program, must be perfectly/embarrassingly parallel

Hadoop

Most popular Map-Reduce framework

Usage: working with unstructured data not stored in a DB, want to extract one feature

Ex) given movie ratings, find avg rating

automatic parallelism

Compilers recognize and automatically parallelize seq. code

NOT entire programs, but JUST kernels/parts of programs

automatic parallelism problem

Cannot automatically parallelize code if there are pointers b/c could point to overlapping pieces of memory

automatic parallelism solution

Use a different language OR use parallelizing C compiler and “restrict” keyword

func prog benefits

No variables → no race conditions

Lack of function side effects → easier to parallelize

CMU Maple

Functional language for parallel programming

Programmer identifies parallelism

Language maps parallelism to threads based on available HW

Maple vs other things

can outperform existing implementations of parallel implementations (Go)

can compete w/low-level optimized C++ code