Memory Hierarchy
The CPU registers are similar to SRAM, but are generally referred toas a separate type of memory
Registers store info using a simple flip flop made from a small number of transistors located directly on the CPU
Because of their small size and proximity to the CPU, registers can be accesed with virtually zeo delay. However, they are optimized for speed rather than reliable storage over long periods of time
Minimal retnetion, eorr checking, etc
lastl, we have secondary stroage ( or permamnent storage), which can be subdivded into two levels: local secondary stroage and remote secondary stroage
bOTH TYPES OF PERMANENT STORAGE ARE NON-VOLATILE,
These are the lowest levels in the memory hierarhcy, meaning they are the slowest, least expensive, and most plentiful types of memory
Local Secondary Storage - refers to permanenet stroage devices that are directly connected to the computer system, such as HDDs and SSDs
Because these de ices are directly connted to the machine, they have less latency and greater reliability than remote secondary storage, which relies on network accesss
Remote Seondary Storage - regers to permanent storage devices that are not directly connected to the computer, and instead rely on access to a remote storage device via a network.
Cloud storage (google drive for example) is an example
Because it relies on network connection, it is slower and less reliable than local stroage. However, it often comes with other benefits such as cost effectiveness, scalability, and remote access
Lastly, let’s take a quick look at two specific types of local secondary storage; namely, hard disk drives and SDDs
HDDs store bites by magnetizing grains on a spinning disk.
A read/write head parses over the disk as it spins. To read from the disk, it detects the magnetic polarization of that particular grain on the platter. TTo write to the disk, it changes that grain’s magnetization
by moving back and forth along its axis, the arm of the read/write head can position itself over any part o the platter - this is called a seek
Consequently, accessing info on an HDD involves three delays:
Seek time: the time it takes the arm to position itself in the correct location
Rotational latency: the time it takes the disk to spin around to the correct location
Trasnfer time: the time required to actually read or write to that location
Rather than using spinning plates, SDDs store info via trapped electrical signals
This is accomplished via NAND flash memory - a floating-gate transistor that traps electrons.
If the floating gate contains electrons, it represents a 1; if the floating gate is empty, it represents a 0
The structure resembles a NOT AND (i.e. NAND) logic gate (but does not function like one)
Because SSDs do not rely on moving parts, they are significantly faster than hDDs. There is no seek time or rotational latency
They also consume less power and are lessp ron to mechanical failure, since there is no need to keep a disk spinning at high speed.
However. Sdds wear out more quickly due the way they write info, and it cna be much more difficult to recover data from them when compared to an HDD
We want to ensure our programs work efficiently, so we optimize the use of our faster memory - accessing it as much as possible, so as to avoid the longer access times from RAM and disk memory
Therefore, it would be helpful to know which memory we are most likely to access, so that we can load it into faster storage
Registers - 0 cycles
Cache - 4 to 75 cycles
RAM - Hundreds of cycles
Disk - Tens of millions of cycles
Counter programs exhibit good locality - both temporal locality and spatial locality
Temporal - the tendency for programs to access the same memory address repeatedly. That is, if a program accesses one memory address, it is likely to access that address again in the near future
Spatial - The tendency for programs to access memory locations that are near to one another. That is, if a program accesses one memory address, that program is likely to reference nearby memory addresses in the near future
The principle of locality allows computerrs to optimize their memory accesses in the haradware, the operating system, and applications
In hardware, we can leverage locality by holding the most recently-referenced blocks of data in smaller, faster memory (i.e. cache) to speed up subsequent accesses
In the operating system, we can leverage locality by using RAM as a cache - storing the most recently-referened items in RAM to avoid lengthier disk access.
In applications, we can leverage locality by writing them in such a way so as to maximize it (both temporally and spatially) to better synergize with the hardware and operating system.
Row-wise iteration - we access each element of a row before moving on to the next row
Column-wise iteration - we access each element of a column before moving to the next column
Conceptually, we understand a matrix as a two dimensional array with rows and columns
access to an element requires u st o specific which row, and then which column
Therefore, to access the center of a 3×3 matrix, we would access the element at row = 1, column = 1.
Although we conceptualize a matrix as a grid of rows and columns, this is not how matrixes are stored in memory - in memory, the matrix is stored as a contiguous series of elements
Some languages use row major order, and others use column major order (row wise iteration and column wise iteration respectively).
C++ and other C langugaes store matrix values according to row major order - consequently, this program will access each data value in the order it is stored. This is an example of good spatial locality
A C++ program utiliziing column wise iteration means we access each element of a row before moving to the next row.
Unlike the previous pr\ogram, this is an example of bad spatial locality, since we are jumping along the array.
Stride - the pattern by which a program accesses elements in memory
For example, our row wise iteration over a matrix in row major order has a stride-1 regerence pattern, since we reference ach eleemnt in order
How many places (k) we jump in memory is referred to as a stride-k reference pattern
For example, our column wise iteration over a 3×3 matrix in row-major order has a stride 3 reference pattern, because there are 3 columns
Endianness
Systems that go from most significant byte to least significant bye are referred toas big endian systems
Systems that go from least significant byte to most significant byte are referred to as little endian systems
As long as all parts of teh system/network are using the same endianness, the hoice between big and little is unimportnat