Cache Memory Principles and Design

Comprehensive vocabulary flashcards covering computer memory hierarchy, cache levels, mapping techniques, replacement policies, coherence protocols, and design elements.

Cache Memory

A small, fast type of volatile computer memory that provides high-speed data access to the processor and stores frequently used data and instructions to reduce retrieval time from slower main memory (RAM).

Temporal Locality

The principle stating that recently accessed data is likely to be accessed again in the near future.

Spatial Locality

The principle stating that data stored near recently accessed memory addresses are likely to be used soon.

L1 Cache (Level 1)

The fastest and smallest cache memory located directly on the processor chip (typically $16\,\text{KB}$ to $128\,\text{KB}$) that stores critical data and instructions currently needed by the processor.

L2 Cache (Level 2)

An intermediate cache storage between L1 cache and main memory (typically $128\,\text{KB}$ to several $\text{MB}$) that acts as a backup to store frequently used, less critical data.

L3 Cache (Level 3)

A last-level cache located on the processor chip and shared among multiple cores in multi-core processors (typically several $\text{MB}$ to tens of $\text{MB}$) to reduce accesses to RAM.

L4 Cache

A large cache (often tens of $\text{MB}$) located outside the processor or on a separate chip, found in high-performance computing, server-grade processors, and certain APUs.

Cache Hit

The event where data requested by the CPU is found in the cache and retrieved immediately.

Cache Miss

The event where data requested by the CPU is not found in the cache and must be fetched from lower cache levels or main memory.

Direct Mapping

A cache mapping technique where each main memory block can only be placed into one specific cache line, indexed using Tag, Index, and Offset fields.

Fully Associative Mapping

A cache mapping technique where any memory block can be stored in any cache location, requiring the cache controller to search the entire cache using the Tag field.

Set Associative Mapping

A hybrid mapping technique where the cache is divided into sets (such as 2-way, 4-way, or 8-way), and each memory block maps to a specific set but can occupy any line within that set.

Least Recently Used (LRU)

A replacement policy that evicts the least recently accessed data block first, making it well-suited for workloads exhibiting temporal locality.

First-In, First-Out (FIFO)

A replacement policy that treats the cache like a queue and evicts the earliest stored block when space is needed, regardless of recent usage.

Least Frequently Used (LFU)

A replacement policy that maintains a counter for each block and evicts the one that has been accessed the lowest number of times.

Random Replacement

A replacement policy that evicts a randomly chosen cache block without tracking usage history or frequency.

Adaptive Policies

Dynamic cache replacement strategies that use machine learning or heuristics to switch between policies like LRU, LFU, or FIFO based on workload patterns.

Cache Coherence

A mechanism that ensures changes to data in one processor's cache are consistently reflected across all other caches in a multi-core or multiprocessor system.

MESI Protocol

A cache coherence protocol that tracks data states across multiple caches using four states: Modified, Exclusive, Shared, and Invalid.

Cache Line (Block) Size

The basic unit of data transfer between cache and main memory, typically ranging from $32\,\text{bytes}$ to $128\,\text{bytes}$.

Write-through

A cache write policy where modifications to data are written simultaneously to both the cache and main memory.

Write-back

A cache write policy where modifications are written only to the cache and the main memory is updated later when the cache line is evicted.

Miss Penalty

The time penalty incurred when a cache miss occurs and data must be retrieved from the slower main memory.

Compulsory Misses

Cache misses that occur on the very first access to a memory block.

Capacity Misses

Cache misses that occur when the cache is not large enough to hold all the blocks needed during execution.

Conflict Misses

Cache misses caused when multiple memory blocks map to the exact same cache slots, common in direct-mapped and low-associativity caches.

Cache Controller

The hardware component responsible for managing the cache, including data fetching, miss handling, and coherence management.