Cache Memory Part 1

Memory Hierarchy

• Cache Memory: Represents a level in the memory hierarchy between the processor and main memory.
• Cache Controller: Hardware controls cache access and memory requests.

General Cache Concepts

• Cache Characteristics:
  • Small, fast memory typically using SRAM.
  • Stores recently accessed data from main memory to improve performance by decreasing access time.
• Volatile Storage:
  • Loses data when power is off.
  • Not large enough for all data needs; backed by primary storage (disk).

Cache Definitions

• Cache: Small fast storage that improves average memory access time.
• Locality Exploitation:
  • Spatial and temporal locality principles guide cache design and operation.
    • Registers and caches act like caches for different memory access levels.

Cache Levels

• Uses multilevel structures (e.g., L1, L2, L3 caches) where:
  • L1 - Fastest but smallest (e.g., 16-32KB).
  • L2 - Larger but slower (e.g., 256KB - 4MB).
  • L3 - Even larger but slower than L2.

Basic Cache Mechanics

• Cache Line: Smallest unit transferred between cache and memory.
• Block Size Consideration:
  • Block size should be a multiple of the memory word size for exploiting spatial locality.
  • E.g., a block of 128 bits = 4 words of 32 bits each.

Cache Hits and Misses

• Cache Miss: Occurs when requested data is not in cache, requiring access to lower memory levels.
• Hit Rate Calculation:
  • Hit Rate = (Number of hits) / (Number of memory accesses).
• Miss Rate Calculation:
  • Miss Rate = (Number of misses) / (Number of memory accesses).
• AMAT (Average Memory Access Time):
  AMAT = Hit Rate imes Hit Time + Miss Rate imes Miss Time

Cache Structure and Operation

• Cache Entries: Contain a valid bit, tag, and data.
• Address Mapping in Direct Mapped Cache:
  • Each memory block maps to a single cache line.
  • Map using: (Block Address) memory modulo (Number of Cache Blocks).

Handling Cache Reads and Writes

• Read Policy:
  • Block address is fetched and compared for a hit or miss.
• Write Policies:
  • Write-Through: Updates both cache and memory.
  • Write-Back: Updates only cache; writes back to memory on eviction.

Cache Replacement Policies

• Replacement Strategies:
  • Least Recently Used (LRU):
  • Track usage to replace the least recently accessed block.
  • Random:
  • Select a block randomly to replace; simpler but less effective in locality consideration.
• Block Replacement on Misses:
  • Direct-mapped caches have a fixed replacement, while associative caches can choose from multiple blocks.

Associative Cache Types

• Fully Associative: Any block can go into any cache entry.
• Set-Associative: Each block can be placed in any location within a limited number of sets.
  • E.g., Two-way associative allows a block to be placed in two locations.

Conclusion: Memory Hierarchy Review Questions

1. Where can a block be placed in the hierarchy? - Block placement types (direct mapped, fully associative, set associative).
2. How is a block identified? - Using tags and valid bits.
3. Which block is replaced on a miss? - Heuristic approaches such as LRU, FIFO, or random policies determine replacement.
4. What happens on a write? - Differentiation between write-through and write-back, along with handling for write misses.