Comp org 5.1 Memory hierarchy

Memory Hierarchy

A structure composed of multiple memory levels with increasing size and decreasing speed, providing the illusion of a large, fast memory system

Principle of Locality

The idea that programs access a small portion of their address space at any time, enabling performance gains through memory hierarchy

Temporal Locality

Recently accessed data will likely be accessed again soon

Spatial Locality

Data with address proximity to recently accessed data will likely be accessed soon

Block (Line)

The unit of copying within the memory hierarchy, often containing multiple words

Upper Level (of Memory)

Faster, smaller level closer to the CPU that uses more expensive technology

Hit

When requested data is found in the upper level of the memory hierarchy

Hit Ratio

hits ÷ total accesses in the cache or memory level

Miss

When requested data is not present in the upper memory level, requiring a transfer from a lower level

Miss Penalty

Time required to retrieve a block from the lower level of the memory hierarchy

SRAM

A memory type built from 6–8 transistors per bit that does not need refreshing and provides fast access times

Specific to SRAM

Fixed access time to any piece of information

Number of transistors per bit of SRAM

6-8 to ensure information is not disturbed

Standby Mode

Needs only minimal power to retain charge - specific to SRAM

DRAM

Memory built from one capacitor and one transistor per bit that requires periodic refresh due to charge leakage

Reason DRAM Data fades

Capacitors leak charge over time - periodic refresh is the solution

Number of transistors per bit of DRAM

One transistor and one capacitor per bit - high density and cost efficiency

Specific to reading data from DRAM

destructive to capacitor charge

Word-Line

Row line in a DRAM memory array used to access a full row of cells

Bit-Line

Column line in a DRAM array used to transfer bit data during reads and writes

Sense Amplifier

Component in DRAM that detects and amplifies small charge differences to determine bit values, and rewrites data (refresh) after reads

Burst Mode

A DRAM feature that supplies successive words from an opened row with reduced latency

DDR DRAM (Double Data Rate)

DRAM technology that transfers data on both clock edges, doubling bandwidth

QDR DRAM (Quad Data Rate)

DRAM with separate DDR input and output channels for increased bandwidth

NOR Flash

Flash type with random read/write access, used for instruction memory in embedded systems

NAND Flash

Flash type with higher density and block-level access, cheaper per GB, used in USB drives and storage devices

Wear Levelling

A technique that remaps flash memory writes to reduce block wear and extend lifespan

Solid-State Drive (SSD)

A secondary storage device built from flash memory: fast, nonvolatile, and transistor-based with no mechanical parts

What is flash memory

non volatile semiconductor memory

What is the same thing is non volatile memory

Secondary memory in the memory hierarchy - (SSD)

The only memory that doesn’t turn off

Solid State Drive - (SSD)

SSD Access time

Fast Access to all locations regardless of addresses

Cache

A fast memory level between the CPU and main memory that stores frequently accessed data to reduce access time

Casche Hierarchies

Used in modern computers to intelligently store and managed frequently accessed data

main memory for mac computers

flash type

L1 Cache (Level 1)

The fastest cache level, located closest to the CPU and often split into separate instruction and data caches

What removes possibility of MEM structural Hazard between IF&MEM

Split nature of L1 Casche

L2 Cache (Level 2)

A larger, slower cache that focuses on reducing miss rate and sits between L1 and main memory

Typical size of L1 cashe

small

L3 Cache (Level 3)

A large cache shared by multiple CPU cores, reducing L2 miss penalties and bridging the gap to main memory

Direct-Mapped Cache

A cache structure in which each memory block maps to exactly one cache location based on address modulo cache size.

What memory level is not typically found in home computers and servers instead

L3 Cashe

How big is L3 Cashe

2-8 MiB

How many bytes is 1 MiB (Mebibyte)

2^(20) bytes

Direct-Mapped Cache - Block numbers

Powers of two

Direct-Mapped Cache - What part of the addresses it contains

Low-order address bits

Formula for direct map block calculation

(Block Address) modulo (#Blocks in Cashe)

Which associativity has the highest miss rate

Direct map cashe

Tag

The high-order bits of an address stored in a cache entry to identify which memory block the entry corresponds to

Valid Bit

A field that indicates whether a cache block contains valid (1) or invalid (0) data

Fully Associative Cache

A cache in which any memory block can be placed in any cache location

Set-Associative Cache

A cache where each memory block can be placed in a fixed number of locations within a set (e.g., 2-way, 4-way)

Eight-Way Set Associative Cache

A cache divided so each set contains eight possible locations, effectively close to fully associative for small caches

Cache Set

A group of cache lines where a memory block can reside in a set-associative cache, determined by index bits

Replacement Policy

The rule used to decide which block to evict when placing new data into a set (e.g., least recently used)

Oldest Values in Casche

Most common replacement policy used in memory design

Index Field

Middle portion of an address used to select a specific cache set or block entry

Byte Offset

Lower address bits indicating the exact byte within a block (not used to index cache sets)

Address Tag Comparison

The process of comparing stored tag bits with address tag bits to determine a hit or miss in the cache

Specific to only L1 Cashe

Slit nature: composed of two independent cashes (I-Cashe & D-Cashe)

I-Cashe

Located in the L1 Level and contains Instruction memory

D-Cashe

Located in the L1 Level and contains the Data memory

Main Design focus of Memory (Cashe in particular)

Minimize hit time and reduce miss rate