Computer System Architecture - Finals (Module 2: Advanced Architectures)

virtual

The American Heritage Dictionary offers two (2) applicable definitions of _______ that describe the usage of the word in modern computing:

> It is an existence or a result in the essence or effect though not in actual fact, form, or name.

> It is created, simulated, or carried on by means of a computer or computer network.

virtualization

of a computer allows a single computer to appear as a multiplicity of computers, each with its own operating system and hardware resources. An individual computer system is used to simulate multiple computers, all sharing the same CPU and I/O facilities.

The simulated machines are known as virtual computers or virtual machines.

Preemptive multitasking

refers to a task in which the operating system uses some criteria to decide how long to allocate to any one task before giving another task a turn to use the operating system.

preemption

the ability of interrupting a task, switching to another one, and then resuming the first task at a later time.

Multitasking

refers to the ability to run simultaneous processes in a given time unit.

Concurrency

the perception of having tasks that run at the same time.

Parallelism

are tasks that literally run at the same time.

threads

This activity could be done with the use of simultaneous subtasks called _________.

Data Race

While a writer thread modifies the memory, a reader thread might be reading from it. If the writer has not finished its work yet, the reader will acquire corrupted data.

Race Condition

This occurs when two (2) or more threads can access the same shared data and try to change the data at the same time.

Synchronization

It is a way to ensure that resources will be used by only one (1) thread at a time.

It is about marking specific parts of the code as "protected" so that two (2) or more concurrent threads will not execute the code simultaneously, which screws up the shared data

Atomic Operations

A bunch of non-atomic operations can be turned into atomic ones by means of special instructions provided by the operating system.

This way the shared data is always kept in a valid state, no matter how other threads access it.

Immutable Data

Shared data is marked as ___________, which means that nothing can change it.

This implies that threads are only allowed to read from it, eliminating the root cause.

Multicore microprocessor

is a microprocessor that contains multiple processors ("cores") in a single integrated circuit.

Virtually, all microprocessors today in desktops and servers are multicore and are used to run a parallel processing program.

Task-level parallelism (process-level parallelism)

is a method of utilizing multiple processors by running independent programs simultaneously.

The difficulty with parallelism is not the hardware. It is difficult to write software that uses multiple processors to complete a task faster, and the problem gets worse as the number of processors increases.

Flynn's Taxonomy

was proposed in 1966, is one of the categorizations of parallel hardware is still used today.

It was based on the number of instruction streams and the

number of data streams.

Single Instruction stream, Single Data stream (SISD)

Single Instruction: Only one (1) instruction stream is being acted on by the CPU during one (1) clock cycle

Single Data: Only one (1) data stream is being used as input during one (1) clock cycle

- Deterministic execution

Single Instruction stream, Multiple Data streams (SIMD)

Single Instruction: All processing units execute the same instruction at any given clock cycle

Multiple Data: Each processing unit can operate on a different data element

Best suited for specialized problems characterized by a high degree of regularity, such as graphics or image processing.

Synchronous (lockstep) and deterministic execution

Two (2) varieties: processor arrays and vector pipelines

Multiple Instruction streams, Single Data streams (MISD)

Multiple Instruction: Each processing unit operates on the data independently via separate instruction streams

Single Data: A single data stream is fed into multiple processing units

Few (if any) actual examples of this class of parallel computer have ever existed; used in multiple frequency filters operating on a single signal stream or in multiple cryptography algorithm attempts to crack a coded message

Multiple Instruction streams, Multiple Data streams (MIMD)

Multiple Instruction: Every processor may be executing a different instruction stream

Multiple Data: Every processor may be working with a different data stream o Execution can be synchronous or asynchronous, and deterministic or non-deterministic

Currently the most common type of parallel computer (mostmodern supercomputers fall into this category)

Examples: current supercomputers, networked parallel computer clusters and "grids," multiprocessor SMP computers, and multi-core PCs

Hardware multithreading

Allows multiple threads to share the functional units of a single processor in an overlapping fashion to try to utilize the hardware resources efficiently.

Fine-grained multithreading

switches between threads on each instruction, resulting in interleaved execution of multiple threads.

This interleaving is often done in a round-robin fashion, skipping any threads that are stalled at that clock cycle.

Coarse-grained multithreading

was invented as an alternative to fine-grained multithreading.

It switches threads only on expensive stalls, such as last-level cache misses.

Simultaneous multithreading

Is a variation of hardware multithreading that uses the resources of a multiple-issue, dynamically scheduled pipelined processor to exploit thread-level parallelism and, at the same time, exploit instruction-level parallelism.

Hyperthreading

is an Intel-made high-performance computing architecture that simulates some degree of overlap when executing two (2) or more independent sets of instructions.

This is a feature of certain Intel chips that makes one (1) physical CPU appear as two (2) logical CPUs

vector architecture

An older and more elegant interpretation of SIMD is called:

Its basic philosophy is to collect data elements from memory, put them in order into a large set of registers, operate on them sequentially in registers using pipelined execution units, and then write the results back to memory.

scalar architectures

in this context, also pertain to a single vector instruction that specifies a great deal of work—it is equivalent to executing an entire loop. The instruction fetches the needed bandwidth and decodes it afterward.

High-Performance Computing (HPC)

It refers to the usage of aggregated computing power for handling computation and data-intensive tasks that include simulation, modeling, and rendering, in which standard workstations are unable to address

One way of addressing this through clustering.

Cluster

is a group of loosely coupled computers which is configured to work together as a unit.

Unlike the tightly coupled multiprocessing system, each computer in a cluster is a complete unit with its own CPU, memory, and I/O facility.

Each computer in the cluster is called a node.

Different nodes of a cluster may be located in the same physical cabinet or located miles apart, provided there is a way to interconnect the high-speed messaging link and, if applicable, the shared disk links.

There are two (2) primary models used for clustering: shared-nothing model and shared disk model.

Characteristics of Clustering

1. To increase the computing power by combining the power of the individual systems

2. To create fault-tolerant systems

3. To create high-availability systems

4. For load-balancing systems with large workloads

supercomputing

The field of high-performance computing, sometimes called ___________.

arose in an attempt to meet the challenge of solving difficult problems that require massive amounts of computing power.

Beowulf Clusters

These are simple, highly configurable clusters designed to provide high performance at a low cost.

These consist of multiple computers connected by a dedicated, private Ethernet which serves as the link between the computers in the cluster.

Each node contains a CPU, memory, an Ethernet connection, and, sometimes, hard disks, and other peripherals.

Linux is generally the OS of choice for this because of its flexibility

Commodity-off-the-shelf (COTS)

components are simply inexpensive computers connected to form a Beowulf cluster.

Blade (Blade Servers)

are computers mounted on a board similar to a motherboard that can be plugged into connectors on a rack.

Grid Computing

Gelernter (as cited in Patterson, 2017) proved in his research that it was possible to produce supercomputer performance for processing large problems.

The processing algorithms allow the data to be broken into tiny chunks for analysis.

On a smaller scale, grid computing is moving into several large financial enterprises to provide more processing capability for their employees.

Cloud Computing

it can be viewed as a simple but potentially powerful conceptual expansion of client-server computing.

Its basic premise is that many functions of an organization's data center can be moved to services on the Internet.

In its simplest form, __________ provides off-site storage facilities for organizations.

Software as a Service (SaaS)

provides software applications that run on a server, delivering the results to the display on a client.

Infrastructure as a Service (IaaS)

offers cloud-based hardware emulation in the form of virtual machines, networking, and the like.

Advantages of Cloud Computing

The client's data center is simplified, and the cost is reduced.

Cloud services provide strong support for user collaboration since multiple users can easily access the same software, databases, tools, and data files from the cloud.

Properly designed (with proper client applications) cloud services can be accessed on a wide variety of client equipment—fixed or mobile and thick or thin—from anywhere the Internet is available.

A cloud-based system is inherently scalable.

A cloud-based system can continue providing services and recovery during an emergency.

Cloud-based services can be useful for short-term projects with intensive computing needs.

The use of IaaS allows developers to make risky changes to their virtual machine environment without threatening the safety of the production equipment.

Cloud Computing Risks

The quality of security at the cloud service is critically important. Even the smallest data leak or data theft can compromise a client organization's future.

Cloud server outages or a loss of connectivity at any point in the link between the client and the cloudservice can impede the ability of users to work.

The client organization is dependent on the long-term commitment and viability of the cloud service.

Graphics Processing Unit (GPU) Architecture

This is a processor optimized for 2D and 3D graphics, video, visual computing, and display.

This term was coined in 2000 when the graphics device had become a processor from a standard Video Graphics Array (VGA) controller

GPU computing

is the term used for computing via a parallel programming language and an application programming interface (API), without using the traditional graphics API and graphics pipeline model.

General-Purpose computation on GPU (GPGPU)

involves programming the GPU using a graphics API and graphics pipeline to perform nongraphic tasks

Compute Unified Device Architecture (CUDA)

is a scalable parallel programming model and software platform for the GPU and other parallel processors that allows programmers to bypass GPU's graphics API and graphics interfaces.

Direct3D

This is one of the Microsoft DirectX multimedia programming interfaces.

OpenGL

This open standard was originally proposed and defined by Silicon Graphics Incorporated

OpenCL

This is made by Khronos Group and is used for GPU Memory buffering.

Vulkan

This is a cross-platform computing API used by AMD