LLM Architecture: The Basics

What is the term for the maximum amount of input text an LLM can process in a single request?

Context Window.

Context Explosion

Exceeding an LLM's context window leads to _, which causes slow responses, high costs, and poor accuracy.

A performance failure where an LLM's reasoning quality drops as context gets too long, causing it to 'forget' early information.

Term: Attention Decay

Truncation.

What LLM failure mode occurs when data exceeding the context window is cut off, causing the model to miss early parts of a conversation?

Retrieval-Augmented Generation.

What does RAG stand for in the context of LLMs?

The metadata is too complex and high-cardinality to be summarized easily.

Why does traditional RAG often 'break' when applied to massive visual datasets?

A design where an LLM receives a lightweight summary or pointer, while the full dataset remains in system memory.

Term: Pass-by-Reference Architecture

Pass-by-value sends raw data to the LLM, while pass-by-reference sends only a lightweight pointer or summary.

What is the key difference between a pass-by-value and a pass-by-reference architecture for LLMs?

Artifact Pattern

The _ is a mechanism where tools return a summary for the LLM and a detailed 'artifact' for the system to store.

Nodes represent images or objects, while Edges represent relationships like similarity or containment.

In a graph-based knowledge system for visual data, what do 'Nodes' and 'Edges' represent?

An architectural shift that treats data as a network of nodes and edges, allowing an LLM to navigate complex data logically.

Term: Graph-based Knowledge System

To selectively show an LLM only the most relevant metadata based on a user's specific question, reducing the information processed.

What is the goal of Query-Aware Sampling?

Agentic Workflows

Designing AI 'agents' that can perform multi-step reasoning and handle their own failure modes is known as creating _.

Data containing millions of unique values, like labels or GPS coordinates, that cannot be easily chunked or embedded.

Term: High-Cardinality Metadata

Noisy Data.

What kind of data, such as duplicates, blurry images, or inconsistent annotations, is known to 'sabotage' AI projects?

To turn raw, 'invisible' files into machine-readable data layers so engineers can focus on model performance.

What is the purpose of creating a Structured Infrastructure for AI data engineering?