Comprehensive Artificial Intelligence, Machine Learning, and LLM Systems Study Notes

Foundations of Artificial Intelligence, Machine Learning, and Deep Learning

• Artificial Intelligence (AI):

  • Encompasses computational systems engineered to execute tasks that traditionally necessitate human-level intelligence.
  • Core capabilities include computer vision, image recognition, natural language processing, and speech translation.
  • Serves as an expansive, overarching domain containing specialized analytical subfields.

• Machine Learning (ML):

  • A core technical subset of AI centered on data-driven statistical algorithms.
  • Replaces explicit procedural code with pattern recognition derived from massive collections of empirical data.
  • Standard real-world application: Email spam filtering, where models analyze historical corpora of genuine and unsolicited communications to recognize textual patterns distinguishing spam from authentic messages.

• Deep Learning (DL):

  • A specialized subdiscipline of Machine Learning structured around Artificial Neural Networks (ANNs).
  • Specifically engineered to emulate biological computational structures present within the human brain.
  • Utilizes stacked, multi-layered network architectures to solve complex pattern recognition challenges across unstructured data such as audio, imagery, and text.

Mathematical Framework of Model Learning and Neural Network Mechanics

• The Learning Process in Humans vs. AI:

  • Humans infer general truths through first-hand experiences, direct interactions, and sensory observations. For instance, observing that healthy, fit individuals consistently consume high quantities of fruit leads to the deductive decision that fruit consumption supports personal health.
  • AI systems learn strictly through pattern extraction across massive datasets comprising millions of data points (e.g., public internet repositories, the entirety of Wikipedia, and digitized books).
  • Image Classification Scenario: In a training set of $1{,}000{,}000$ labeled photos containing happy and sad faces, mathematical algorithms identify distinct visual features (e.g., concave upward curves corresponding to smiles versus watery ocular features indicating sorrow).

• Formal Definition of an AI/ML Model:

  • Fundamentally exists as a parameterized mathematical function $f(x)$.
  • Given an input vector $x$, the function computes and outputs a target prediction $y$:         $y = f(x)$
  • Training constitutes optimizing the internal parameters of $f(x)$ across observed data, whereas inference denotes evaluating $f(x)$ on novel, previously unencountered inputs.

• Artificial Neural Network Structural Topology:

  • Composed of artificial nodes (neurons) distributed across sequential layers linked by weighted computational directed connections.
  • Input Layer: Receives raw multi-parameter external feature values ($x_1, x_2, x_3$).
  • Hidden Layers: Intermediate computational strata that transform feature representations; deep neural networks integrate extensive stacks of these layers.
  • Output Layer: Generates the terminal prediction, classification, or continuous score.

• Weights and Bias System:

  • Connections represent scalar weights ($W$) reflecting the relative importance of specific input signals.
  • Academic Grading System Analogy: Calculating student Grade Point Average (GPA) based on three evaluated inputs—mid-semester exams ($x_1$), end-semester exams ($x_2$), and class tests ($x_3$).
  • Initial arbitrary parameter assignment: Professors might assign weights of $0.4$ ($40\%$) to mid-semester exams, $0.4$ ($40\%$) to end-semester exams, and $0.2$ ($20\%$) to class tests.
  • Optimization adjustment: If evaluation outcomes prove suboptimal, professors adjust the weights (e.g., raising class tests to $0.3$ or $30\%$ while decreasing mid-semester exam weight) to enhance evaluation precision.

• Neuron Mathematical Activation:

  • A single node processes input vectors by computing an inner dot product with associated connection weights, incorporating an additive scalar bias $b$, and routing the sum through a non-linear activation function $f$:         $\text{Output} = f(x_1 W_1 + x_2 W_2 + x_3 W_3 + b)$
  • Activation functions dictate non-linear thresholding and determine whether individual nodes propagate signals onward.

• Forward Propagation, Loss Computation, and Backpropagation:

  • Forward Propagation: Input values pass left-to-right through successive network layers, evaluating node operations until computing an output prediction.
  • Loss Calculation: The output is mathematically benchmarked against the ground truth target value via a loss function, quantifying prediction error.
  • Backward Propagation (Backpropagation): The calculated loss signal propagates backwards (right-to-left) through the network, utilizing gradient optimization to update and calibrate weight matrices throughout the architecture.
  • Scale: Classifying $1{,}000{,}000$ animal photographs (e.g., cat vs. non-cat binary classification) requires $1{,}000{,}000$ forward passes followed by $1{,}000{,}000$ backward passes to systematically minimize error across iterations.

Large Language Models (LLMs) and Tokenization Mechanics

• Core Concepts of Large Language Models:

  • Prominent models and model families include GPT-5, GPT-6, Gemini, Claude, and DeepSeek.
  • Defined as deep neural networks trained upon massive textual corpora encompassing the entirety of Wikipedia, millions of published volumes, and internet scrapes.
  • Perform natural language understanding, generation, translation, extraction, and automated summarization.

• Probabilistic Next-Token Prediction:

  • The foundational mechanism of generative language generation is predicting the next sequential token.
  • Query Execution: When queried with "What is the capital of India?", the model predicts the token sequence completing the phrase ("New Delhi").
  • Statistical Operations: The model computes a probability distribution across its vocabulary. Contextual phrases like "capital of India" yield high probabilities for "New Delhi" and near-zero probabilities for semantically unrelated terms such as "Apple" or "Mango".

• Tokenization Process and Mechanics:

  • Computational hardware cannot directly interpret raw natural language text (such as English, Hindi, Punjabi, or Japanese); all characters and strings must be encoded into numerical values.
  • A token constitutes the fundamental atomic unit of textual processing in an LLM, approximating $\frac{3}{4}$ of a standard English word.
  • OpenAI Tokenizer Demonstration: The sentence "I like generative AI" is fragmented into five discrete tokens:
    1. I
    2. like (incorporating leading whitespace)
    3. Gener
    4. ativ
    5. AI
  • Linguistic Suffix Segmentation: Words like "generative" are split into subword roots (Gener) and standard suffixes (ativ). Suffixes such as -ative (shared by "creative", "reactive", "talkative") and -ful (shared by "playful", "joyful") are preserved across distinct tokens to maintain semantic meaning within the model's lexicon.
  • Vocabulary and Token IDs: Every token maps to an integer index (Token ID) within the model's static dictionary. For example, I may map to Token ID $40$, and like may map to Token ID $1299$. When modifying the trailing clause to "I like machine learning", the initial tokens retain their static Token IDs ($40$ and $1299$).

Context Windows and Context Dynamics

• Context Window Definition:

  • Functions as the temporary random-access memory (RAM) or dynamic working memory allocated to an LLM for an active, single conversation thread.
  • Stores and tracks the user's primary prompt, model responses, appended interaction history, and attached external artifacts (such as uploaded PDFs, documentation, or imagery).

• Historical and Modern Capacity Limits:

  • Early foundational models like GPT-1 possessed a context capacity of $512$ tokens.
  • Modern frontier architectures, such as GPT-6 and Claude, support context allocations reaching approximately $1{,}000{,}000$ tokens.

• Operational Constraints and Context Degradation:

  • Context memory is strictly non-permanent; it clears across isolated conversation threads.
  • Context Saturation: Exceeding or overcrowding the context window degrades generation quality exponentially.
  • Position Bias ("Lost in the Middle"): Language models exhibit heightened focus on information positioned at the extreme beginning and end of the context prompt, while intermediate sections experience cognitive fuzziness. Critical directives and crucial data points should be positioned at the absolute start or end of input sequences.
  • Mitigation Strategies: Complex enterprise challenges should be partitioned into discrete sub-tasks tackled across isolated threads, or long threads should be periodically condensed via context summarization.

Vectors, Embeddings, and the Vectorization Process

• Vector Construction:

  • In mathematical terms, a vector represents a quantity characterized by directional orientation and magnitude, structured within computing as a one-dimensional array of real numbers.
  • Token IDs are projected into continuous $n$-dimensional vector spaces. A token such as cat maps to a multi-dimensional array (e.g., [0.05, 0.3, 0.5, ...] with thousands of continuous dimensions) encoding latent attributes such as animacy, physical traits (whiskers, color), or domestication.

• Vectorization and Semantic Geometry:

  • Vectorization represents the algorithmic pipeline translating discrete token indices into continuous multidimensional embeddings.
  • Semantic Clustering: Projecting vectors into simplified two-dimensional spaces reveals conceptual groupings:
    • The vector for cat locates in close spatial proximity to lion relative to the vector for dog.
    • tiger maps adjacent to lion and cat.
    • pet maps closer to dog and cat than to wild fauna such as lion or tiger.

• Distinction Between Vectors and Embeddings:

  • "Vector" serves as a general data structure format (a numerical 1D array) utilized across physics, linear algebra, and software systems.
  • "Embedding" specifically denotes a vector whose mathematical coordinate space captures the contextual, syntactic, and semantic relationships of tokens within natural language processing.

The Attention Mechanism and Transformer Architecture

• The Contextual Disambiguation Problem:

  • Static lexical representations fail when processing polysemous terminology:
    • Example A: "I code in Python"
    • Example B: "Python bit me"
    • Example C: Context referencing the British comedy troupe "Monty Python"
  • The identical token Python denotes a high-level programming language in Example A, a biological serpent in Example B, and a performance comedy group in Example C. Static word vectors cannot distinguish these contexts without dynamic mechanisms.

• Attention Mechanics and Attention Scores:

  • Introduced in the seminal 2017 research paper entitled Attention is All You Need.
  • Dynamically calculates mathematical correlations between every token and all neighboring tokens within an input sequence.
  • Generates internal Query ($Q$) and Key ($K$) vectors for sequence tokens, using operations such as dot products to derive normalized Attention Scores.
  • Disambiguation: In "Python bit me", the word bit achieves a high attention score relative to Python, directing the model toward the biological snake interpretation. In "I code in Python", the token code yields the highest attention score, signaling the software engineering interpretation.

• Transformer Structural Blueprint:

  • Forms the underlying architecture for all modern foundation models, including ChatGPT, Claude, Gemini, and DeepSeek.
  • Analogy: The Transformer represents the mechanical engine or operating system, whereas an LLM represents the full automobile or consumer smartphone with integrated user applications, external tools, and retrieval systems.
  • Layer Architecture: Each Transformer layer comprises two sequential modules:
    1. Attention Block: Executes Self-Attention and Multi-Head Attention to determine relational context.
    2. Feed-Forward Neural Network (FFNN): Applies non-linear transformations to project attended representations into higher dimensions.
  • Layer Stacking: Information outputs propagate through identical stacked layers, repeating the attention-FFNN sequence across hundreds of iterations ($n$ layers).

• Hierarchical Conceptual Extraction:

  • Consider the sentence: "Bank approved the loan because customer had a good credit score."
  • Base Layers: Resolve localized lexical syntax and direct relationships (associating loan with approved and credit score with good).
  • Deep Layers: Resolve abstract semantic causation, determining that the presence of an elevated credit score served as the causative factor behind the bank's approval action.

• Computational Complexity and State Space Alternatives:

  • Standard Transformer attention requires every input token to compute relational scores with every other token, generating an $n \times n$ matrix where $n$ denotes sequence length.
  • Results in quadratic computational complexity:         $O(n^2)$
  • This quadratic bottleneck dramatically escalates compute and memory costs as context lengths expand.
  • Architectural Alternatives: State Space Models (SSMs), exemplified by architectures like Mamba, scale linearly to bypass quadratic attention costs.

Learning Paradigms: Fine-Tuning and Self-Supervised Learning

• Domain Adaptation via Fine-Tuning:

  • Pre-trained base foundation models possess generic natural language fluency but lack depth in specialized enterprise domains (such as healthcare, quantitative finance, legal litigation, and customer support).
  • Process: Takes an existing pre-trained foundation model and conducts secondary gradient descent training on curated, high-quality, domain-specific datasets.
  • Economic Value: Circumvents the exorbitant financial and computational expenditure required to train foundation models from raw initiation, while aligning model vocabulary, output styling, and precision with industry terminology.

• Supervised Learning vs. Self-Supervised Learning:

  • Supervised Learning: Requires explicit input-output pairs $(x, y)$, such as an email body ($x$) paired with a human-assigned classification label ($y \in \{\text{spam}, \text{not spam}\}$). The model predicts the target, calculates loss, and backpropagates errors. Manual annotation is labor-intensive, slow, and expensive at scale.
  • Self-Supervised Learning: The predominant training methodology for modern foundation models. Ingests enormous quantities of unannotated, unlabeled web corpora.
  • Internal Target Generation: The training framework automatically conceals or masks sequence tokens, requiring the model to predict the omitted elements from surrounding context.
  • Linguistic Masking Example: In the sequence "Don't judge a book by its [mask]", predicting the target token cover produces zero loss, whereas predicting an erroneous token such as watch incurs a high penalty.
  • Computer Vision Masking Example: Masking a portion of an image and requiring the model to reconstruct the missing visual region (e.g., penalizing incorrect classifications like predicting a bicycle where background foliage was masked).

Reinforcement Learning and Human Alignment (RLHF)

• Classical Reinforcement Learning (RL):

  • Operates through an autonomous entity (the agent) interacting with an environment to maximize cumulative scalar rewards.
  • Behavioral Conditioning Analogy: Comparable to training a domestic dog (e.g., Dobby); executing desired commands yields positive rewards (treats), whereas adverse behaviors trigger negative penalties.
  • Chess Simulation: An RL agent playing chess receives positive reward values for capturing opposing pieces or achieving checkmate, and negative penalties for forfeiting material or losing matches, refining its strategic policy over millions of games.

• Reinforcement Learning with Human Feedback (RLHF):

  • Aligns pre-trained probabilistic text generators with human ethical standards, formatting preferences, and factual expectations.
  • Mechanism: An LLM generates two distinct candidate outputs (Response 1 featuring illustrative text and diagrams versus Response 2 featuring plain text) in response to a user prompt.
  • Human evaluators indicate their preferred output variant. Selecting Response 1 assigns a positive scalar reward to that response's policy parameters, while penalizing the unselected alternative (Response 2).
  • RLHF forms the backbone of conversational alignment and safety tuning across contemporary consumer LLMs and robotics navigation systems.

Retrieval-Augmented Generation (RAG) and Semantic Search Systems

• The Contextual Data Boundary Problem:

  • Base foundation models remain blind to proprietary, secure, private, or post-training enterprise records.
  • Example: A proprietary drone engineering enterprise (e.g., DJI) maintaining internal product manuals, hardware schematics, and private bug documentation cannot use an off-the-shelf base LLM to resolve customer queries regarding unreleased or internal components.

• RAG Architecture Pipeline:

  • Chunking Phase: Proprietary documents (PDFs, system documentation, manuals, imagery) are split into smaller segments (chunks).
  • Embedding Phase: Document chunks are converted into dense vector embeddings using specialized embedding models.
  • Storage Phase: Chunks and their associated vector embeddings are indexed within a dedicated Vector Database (VectorDB).
  • Retrieval Phase: When an end-user submits a query, the query is converted into an embedding, and the VectorDB retrieves the most semantically relevant text chunks.
  • Generation Phase: The retrieved context chunks are combined with the user's initial prompt and injected into the LLM, enabling grounded, factually verifiable responses.

• Semantic Search vs. Traditional Keyword Search:

  • Traditional Database Search: Executes literal character/string matching against indexed database entries.
  • E-commerce Scenario: A consumer queries an e-commerce platform (like Amazon or Flipkart) for "jogging shoes". A traditional database searches for exact matches containing jogging or shoes. It overlooks a relevant product catalog entry titled "Nike Light Weight Footwear" due to the complete absence of matching keywords.
  • Semantic Search: Analyzes geometric distance within dense vector space. Recognizes that "Nike Light Weight Footwear" occupies immediate geometric proximity to "jogging shoes", retrieving the item based on conceptual intent.

• Vector Databases and Indexing Infrastructure:

  • Industry implementations include Pinecone, Chroma, and FAISS.
  • Vector engines utilize specialized indexing structures, such as Hierarchical Navigable Small World (HNSW) graphs, to achieve sub-linear approximate nearest neighbor (ANN) retrieval across millions of dense vectors.
  • Data Security Benefits: Medical institutions maintain strict privacy compliance (HIPAA, patient confidentiality) by applying fine-grained access control policies inside the retrieval pipeline, preventing unauthorized context exposure while answering patient inquiries.

Prompting Strategies and the Model Context Protocol (MCP)

• Prompting Paradigms:

  • Zero-Shot Prompting: Submitting an instructional query to an LLM without furnishing prior demonstrations or sample outputs.
  • Few-Shot Prompting (In-Context / Example-Based Prompting): Providing a handful of explicit input-output examples directly inside the input prompt to establish formatting and contextual patterns.
    • Recommendation Scenario: A user requests media suggestions by conditioning the prompt: "I watched The Three-Body Problem, Dark, and Rick and Morty; suggest similar TV shows." The model identifies the latent science-fiction pattern to recommend genre-aligned titles.
  • Many-Shot Prompting: Supplying dozens or hundreds of comprehensive structural examples within massive context windows to guide generation style.

• Model Context Protocol (MCP):

  • An open standard defining how foundation models interface with disparate local/remote computational tools, external enterprise APIs, code repositories, and relational databases.
  • Structural Topology: Features three architectural layers:
    1. LLM: The central reasoning engine.
    2. MCP Client: The intermediary interface intercepting instructions and handling communications.
    3. MCP Server: Standardized wrappers surrounding third-party resources (e.g., GitHub, Freshworks, Zendesk, Shopify, internal SQL datastores).

• MCP Enterprise Workflow Scenarios:

  • Software Engineering Integration: An LLM connected via MCP accesses a corporate GitHub repository to read code, make commits, and open pull requests autonomously.
  • Customer Support Workflow:
    • A customer support manager asks: "What is the shipping status for customer Rohit, and has his support ticket been resolved?"
    • The LLM determines it lacks direct operational access to order tracking and customer support ticketing databases.
    • The LLM directs the MCP Client to query external endpoints.
    • The MCP Client communicates with the Shopify MCP Server to retrieve order and shipping logs, and queries the Freshworks MCP Server to inspect support ticket status.
    • The unified data returns to the LLM, which synthesizes the external records into a comprehensive final response.

Context Engineering and Multimodal AI Systems

• Context Engineering:

  • The practice of designing, structuring, filtering, and curating all informational inputs delivered into an LLM's context window alongside the raw user prompt.
  • Core components of an engineered context payload include:
    • Retrieved contextual chunks sourced from external Vector Databases (RAG pipelines).
    • Formatting paradigms supplied via few-shot demonstrations.
    • Dynamic data retrieved from external systems via MCP clients and servers.
    • User Profiles and Preferences: Explicit preferences, such as programming language choice (e.g., default output in Java or Python) or localized spoken languages (e.g., generating responses in French).
    • Conversation History: Historic dialogue turns from the active thread.
    • System Instructions: Developer-level guardrails enforcing corporate governance, safety constraints, tone, and operational boundaries.
    • Context Summarization: Condensing five to ten historical dialogue turns into five to ten succinct lines of summary text to preserve context window limits.

• Multimodal AI Systems:

  • Early conversational deployments (such as the original release of ChatGPT in 2022) were unimodal, processing and generating text inputs exclusively.
  • Multimodal Architectures: Advanced systems engineered to concurrently process, interpret, cross-reference, and generate multiple distinct data formats, including text, static imagery (diagrams, memes), synthesized audio, and dynamic video.
  • Economic Trade-Offs: Unimodal, text-only models operate faster and incur lower computational costs for purely text- or math-heavy tasks; multimodal models should be reserved for workloads requiring visual interpretation, audio transcription, or media synthesis.

Agentic AI and Multi-Step Reasoning Paradigms

• AI Agents and Agentic Systems:

  • Standard conversational LLMs operate passively, responding to user prompts with generated text without taking direct actions.
  • AI Agents function as goal-oriented execution systems granted tool-use capabilities, operational autonomy, environmental feedback loops, and external resource access.
  • The LLM acts as the core central processing unit (the "brain") of an agentic workflow, decomposing broad directives into structured sub-tasks and orchestrating actions across auxiliary sub-agents.
  • Agentic DevOps Scenario: An AI software engineering agent takes user-defined feature goals, uses MCP to inspect repositories, writes software code, pushes commits, submits pull requests on GitHub, updates Jira task boards, and sends operational status reports to team members on Slack.

• Reasoning Paradigms:

  • Chain of Thought (CoT): The model works through intermediate reasoning and analytical steps prior to outputting a final answer. Deconstructing problems step-by-step improves multi-step planning, mathematical calculations, and dependency tracking.
  • Tree of Thought (ToT): Models reasoning through branching decision trees. The system evaluates hypothetical solution branches; if a branch encounters an analytical dead-end, the model backtracks to pursue viable alternative pathways.
  • Graph of Thought (GoT): Models non-linear reasoning through directed graph networks, enabling multi-path aggregation and cross-branch synthesis.
  • Large Reasoning Models (LRMs): Advanced models (e.g., OpenAI o-series, DeepSeek-R1) that dynamically scale their reasoning steps based on input complexity—allocating minimal reasoning effort to straightforward queries and expanding compute across complex math and logic tasks.

Small Language Models (SLMs), Model Distillation, and Quantization

• Small Language Models (SLMs) vs. Foundation LLMs:

  • Foundation LLMs: Broad generalists containing between $3{,}000{,}000{,}000$ and over $100{,}000{,}000{,}000$ parameters (with frontier architectures reaching into the trillions of parameters).
  • Small Language Models: Highly focused, domain-specific models containing fewer than $10{,}000{,}000{,}000$ parameters (typically spanning $1{,}000{,}000$, $100{,}000{,}000$, $400{,}000{,}000$, or up to $7{,}000{,}000{,}000$ to $8{,}000{,}000{,}000$ parameters).
  • Advantages: Feature lower memory and inference compute requirements, cost-effective deployment, localized execution on edge devices, and strict data privacy preservation (ideal for on-premises deployment in hospitals, financial institutions, and legal firms).

• Knowledge Distillation:

  • A model compression technique structured around a Teacher-Student training framework.
  • A massive, pre-trained foundation model acts as the "Teacher", while a compact Small Language Model acts as the "Student".
  • Training Objective: When an input $x$ is fed into both networks, the Teacher generates an output prediction $y$. Instead of training the Student model on raw ground-truth hard labels, it is trained to predict the Teacher's rich probability distribution (its "soft targets").
  • Enables the compact model to replicate the reasoning, language patterns, and stylistic quality of the larger model while operating at a fraction of the compute footprint.

• Quantization Principles and Memory Optimization:

  • Neural network weights are typically stored as continuous floating-point numbers in $32$-bit (FP32), $16$-bit (FP16), $8$-bit (INT8), or $4$-bit (INT4) numerical precisions.
  • Quantization reduces the bit-width precision of these numerical matrices to lower memory consumption and accelerate compute throughput.
  • Theoretical Memory Savings:
    • Downscaling weights from $32$-bit to $8$-bit yields an approximate memory reduction of:             $75\%$
    • Downscaling weights from $32$-bit to $4$-bit yields an approximate memory reduction exceeding:             $87\%$

• Quantization Methodologies:

  • Post-Training Quantization (PTQ): Compresses the weights of a fully trained model down to lower precision after training completes. Baseline training costs remain unchanged, but deployment and inference compute requirements drop significantly.
  • Quantization-Aware Training (QAT): Simulates low-precision rounding errors during the forward and backward passes of the model's primary training process.
  • Trade-offs: QAT increases upfront training complexity and computational expense, but preserves higher accuracy and output stability at low bit depths (such as $4$-bit) relative to post-training quantization.

Course Curriculum and Program Details

• Curriculum Structure:

  • Covers Machine Learning, Deep Learning, Generative AI, and Agentic AI over a total duration of $4.5$ months.
  • Pedagogical Methodology: Emphasizes a "learning-by-doing" approach focused on code execution, programming assignments, and industry-grade minor and major portfolio projects built for production deployment.

• Batch Schedule and Enrollment:

  • Batch Identifier: Prime 3.0.
  • Start Date: September 17, 2026 (09/17/2026).
  • Special Student Discount Date: September 15.