cs4000 final exam

Parallel Computing

run mpiexec locally on three cores

mpi exec -np 3

program uptime command

mpiexec uptime

specifying hosts with mpi exec

mpiexec -host sg09,sg08 hostname

running mpiexec with multiple hosts on a file

mpiexec -hostfile file.txt

MPI_Init(Null,Null)

Starts up the mpi environment. Must be called before any other MPI function.

MPI_Comm_size(MPI_COMM_WORLD, &numprocesses)

Initializes the number of processes

MPI_Comm_rank(MPI_COMM_WORLD, &procnum)

used to decide which chunk of work each cpu handles.

MPI_Finalize()

SHuts down the MPI environment. MUst be the last MPI call

MPI_Send(buf, count, type, dest, tag, comm)

Sends count elements of type from buf to process dest. Blocks until the message is safely handed off.

MPI_Recv(buf, count, type, src, tag, comm, &status)

Receives a message from process src into buf. Blocks until message arrives.

MPI_Bcast(buf, count, type, root, comm)

Root process sends buf to all other processes.

MPI_Reduce(&send, &recv, count, type, op, root, comm)

Combines values from all processes using op and stores the result only on root.

MPI_Allreduce(&send, &recv, count, type, op, comm)

Same as MPI_Reduce but the result goes to all processes, not just root. Used when everyone needs to know the combines results.

Race Condition

Accessing data at the same time, results in non-deterministic outcomes

Critical section

A piece of code that must not be executed by more than one process at a time. CAn use omp critical or mutex lock and unlocks.

Deadlock

When two or more processes are waiting for the other process to release a resource, so none of them can ever proceed.

Process Synchronization

Mechanisms to coordinate processes so they dont interfere with each don't. INcludes barriers, semaphores, mutexes, mpi_barrier

Thread Safety

Code is thread safe if it works when called simultaneously by multiple threads.

Amdahls Law

1/(1-p) + p/N

Efficiency

E=Speedup/N

Speedup

T_serial / T_Parallel

Parallel Algorithms

Divides work across multiple processors simultaneously

Public/Private key cryptogtaphy

Asymetric encryption using a key pair.Public key encrypts, private key decrypts.

Client

Any application program, makes a request, awaits a response

Server

always running, awaits a request, computes an answer, issues a response

Ping

Used to determine if a server is alive. sends a packet to a server and awaits a response. Records the change in time and packets lost.

traceroute

shows all the routers a packet goes through to get ot the destination

mtr

fancy parallel traceroute

nslookup

returns the domain name for an ip address or the reverse

DNS

Map a domain name to an IP address

SSH

secure shell, uses tcp connection to pass data

Electronic Mail

Memos sent from one user to another. COntain only a few impoprtant keywords, To, From,

SMTP

Mail transfer, internet standard, specifies tranfer from machine to machine

IP prefix

which network or organization it belongs to

IP suffix

Which computer on that network it is

how necessary is www

it’s unnecessary

IP vs Ports

IP identifies computers ports identify applications on computers

Statically assigned ports vs dynamically assigned ports

Statically assigned ports are reserved for well known programs like email. Dynamically assigned ports are available fro user applications

generate public private key pair

ssh-keygen

Copy key to computer

ssh-copy-id user.computer

Distributed programming

Computing that does not happen on a single machine

Semaphore

A signaling mechanism used to control accesss to a common resource

Barrier

All threads must reach a certain point before continuation

Static scheduling

Iterations divided into fixed sized chunks

Dynamic scheduling

Iterations are broken into chunks and assigned to threads on-demand during runtime. When a thread is finished with a chunk it asks for the next available chunk.

Guided Scheduling

Designed to reduce the overhead of dynamic scheduling. Starts wiht large chunks to reduce overhead then progressively decreases the chunk size as it reaches the end of the loop.