VV

Cache Memory Part 1

Memory Hierarchy

  • Cache Memory: Represents a level in the memory hierarchy between the processor and main memory.
  • Cache Controller: Hardware controls cache access and memory requests.

General Cache Concepts

  • Cache Characteristics:
    • Small, fast memory typically using SRAM.
    • Stores recently accessed data from main memory to improve performance by decreasing access time.
  • Volatile Storage:
    • Loses data when power is off.
    • Not large enough for all data needs; backed by primary storage (disk).

Cache Definitions

  • Cache: Small fast storage that improves average memory access time.
  • Locality Exploitation:
    • Spatial and temporal locality principles guide cache design and operation.
      • Registers and caches act like caches for different memory access levels.

Cache Levels

  • Uses multilevel structures (e.g., L1, L2, L3 caches) where:
    • L1 - Fastest but smallest (e.g., 16-32KB).
    • L2 - Larger but slower (e.g., 256KB - 4MB).
    • L3 - Even larger but slower than L2.

Basic Cache Mechanics

  • Cache Line: Smallest unit transferred between cache and memory.
  • Block Size Consideration:
    • Block size should be a multiple of the memory word size for exploiting spatial locality.
    • E.g., a block of 128 bits = 4 words of 32 bits each.

Cache Hits and Misses

  • Cache Miss: Occurs when requested data is not in cache, requiring access to lower memory levels.
  • Hit Rate Calculation:
    • Hit Rate = (Number of hits) / (Number of memory accesses).
  • Miss Rate Calculation:
    • Miss Rate = (Number of misses) / (Number of memory accesses).
  • AMAT (Average Memory Access Time):
    AMAT = Hit Rate imes Hit Time + Miss Rate imes Miss Time

Cache Structure and Operation

  • Cache Entries: Contain a valid bit, tag, and data.
  • Address Mapping in Direct Mapped Cache:
    • Each memory block maps to a single cache line.
    • Map using: (Block Address) memory modulo (Number of Cache Blocks).

Handling Cache Reads and Writes

  • Read Policy:
    • Block address is fetched and compared for a hit or miss.
  • Write Policies:
    • Write-Through: Updates both cache and memory.
    • Write-Back: Updates only cache; writes back to memory on eviction.

Cache Replacement Policies

  • Replacement Strategies:
    • Least Recently Used (LRU):
    • Track usage to replace the least recently accessed block.
    • Random:
    • Select a block randomly to replace; simpler but less effective in locality consideration.
  • Block Replacement on Misses:
    • Direct-mapped caches have a fixed replacement, while associative caches can choose from multiple blocks.

Associative Cache Types

  • Fully Associative: Any block can go into any cache entry.
  • Set-Associative: Each block can be placed in any location within a limited number of sets.
    • E.g., Two-way associative allows a block to be placed in two locations.

Conclusion: Memory Hierarchy Review Questions

  1. Where can a block be placed in the hierarchy? - Block placement types (direct mapped, fully associative, set associative).
  2. How is a block identified? - Using tags and valid bits.
  3. Which block is replaced on a miss? - Heuristic approaches such as LRU, FIFO, or random policies determine replacement.
  4. What happens on a write? - Differentiation between write-through and write-back, along with handling for write misses.