Building Generative AI Agents: Extensive Study Notes

Introduction to AI Agents

• Andrew Ng's Influence: A prominent figure in AI, academic, and entrepreneur who recognized AI's potential in the 1990s while at Bell Labs. He holds a master's from MIT and a Ph.D. from UC Berkeley (thesis on reinforcement learning). He taught the popular CS229 course at Stanford and held leadership roles at Baidu and Google Brain. He cofounded Coursera (valued at nearly $6,000,000,000 in 2021 with 148,000,000 users), DeepLearning.AI, and Landing AI.
• The Trend of AI Agents: Andrew Ng describes agent workflows as an "exciting trend" that could lead toward AGI (Artificial General Intelligence), viewing it as a journey rather than a destination.
• Bill Gates' Vision: Gates argues that agents will be the next major platform, succeeding Android, iOS, and Windows. He notes that current software is "pretty dumb," whereas AI agents will deeply understand a user's personal and professional life.
  • Travel Example: Instead of manually searching, a travel agent bot would know budget, interests, past destinations, and adventure propensity to recommend and book hotels, restaurants, and itineraries.
• McKinsey Analysis: McKinsey posits that agents unlock value by automating a "long tail" of complex use cases involving variable inputs/outputs that were historically too costly to automate.
• Industry Shift: Sonya Huang (Sequoia Capital) states the industry is moving from "copilots to agents."

Defining AI Agents and Their Components

• Core Definition: Harrison Chase (Cofounder of LangChain) defines agents as systems where an LLM decides the "control flow" of an application.
  • Agentic vs. Traditional RAG: In a traditional Retrieval-Augmented Generation (RAG) chain, steps are fixed (search query $\rightarrow$ retrieve $\rightarrow$ answer). In an agentic system, the LLM decides if it needs to search once, multiple times, or not at all before responding.
• Key Components:
  • Reflection: A system's ability to inspect and adjust its own cognitive processes. The Reflexion framework uses verbal self-reflections stored in memory to improve performance in environments like AlfWorld and tasks like code generation.
  • Tools: Capabilities allowing agents to interact with external APIs, software, or data (e.g., Salesforce's Einstein GPT or AWS Solution Architect Agent). Agents must navigate UIs by interpreting HTML or pixels.
  • Memory:
    • Short-term: Temporary retention for immediate tasks (e.g., a conversation log).
    • Long-term: Stored via vector databases for extended retrieval.
    • Episodic Memory: Specific events/past occurrences.
    • Semantic Memory: General knowledge and facts about the world.
    • Procedural Memory: Learned skills and processes.
  • Planning: LLMs break down complex goals (e.g., organizing a virtual event) into manageable subtasks. Frameworks include TPTU (Task Planning and Tool Usage).
  • Multi-agent Collaboration: Multiple LLMs specializing in distinct roles (e.g., Content Creator, Strategist, Analyst) work together. MIT research suggests this improves reasoning and factual accuracy.
  • Autonomy: The ability for agents to execute tasks without constant human intervention (e.g., autonomous vehicles interpreting sensor data).

UI, UX, and Development Approaches

• Impact of Design: Forrester Research indicates a well-designed UI can increase conversion rates by $200\%$ and better UX by $400\%$.
• Interactive Design: Shifting away from "human-in-the-loop" copilots toward proactive agents that operate in the background and reach out with updates (e.g., smart home energy notifications).
• Traditional vs. AI Development:
  • Traditional: Deterministic, linear progression (requirements $\rightarrow$ design $\rightarrow$ coding $\rightarrow$ testing $\rightarrow$ deployment).
  • AI Agent Development: Probabilistic/stochastic. Testing involves pairwise comparisons (e.g., Langsmith, Lmsys) rather than simple "test that $2=2$" unit tests.
• Flavors of Agents:
  • Embodied Agents: Interact with physical worlds or 3D simulations (robotics, NPCs in video games).
  • Software Agents: Digital environments (office workflows, data management).

Evolution and History

• Early Foundations: The Logic Theorist (1955) and General Problem Solver (1957) by Allen Newell and Herbert A. Simon.
• Generative Breakthrough: Launch of OpenAI's ChatGPT (November 2022).
• Emerging Platforms:
  • BabyAGI (Yohei Nakajima) and AutoGPT (Toran Bruce Richards) provided early autonomous attempts but faced brittleness.
  • LangGraph, AutoGen, and CrewAI now lead the open-source evolution.
• SaaS Market Transition: Currently valued at approximately $261.15\text{ billion}$. Shift from seat-based subscriptions to outcome-based pricing (productivity/cost saving measurements).

Real-World Use Cases and Implementations

• Sierra: Founded by Bret Taylor (ex-Salesforce co-CEO) and Clay Bavor. Focuses on high-security enterprise customer experience agents using multiple models (sometimes 7+) including a "supervisor" model.
• Enso: Founded by Mickey Haslavsky for SMBs. Uses API integrations and RPA to automate complex tasks like podcast generation (topic research $\rightarrow$ script $\rightarrow$ voice $\rightarrow$ music $\rightarrow$ video).
• Asana: Launched "AI Teammates" utilizing the Asana Work Graph to automate internal workflows like help ticket routing.
• IBM: Sandi Besen notes a shift toward embedding agents into operational business processes (e.g., flight rebooking, policy conflict analysis).

Generative AI Foundations

• LLMs: Pretrained on massive datasets (Wikipedia, Reddit).
• Embeddings: Dense vector representations of words/tokens (fixed-length arrays of numbers).
• Scaling Laws: Performance improves with increased training data and model size.
• Transformers: Introduced in 2017 ("Attention Is All You Need"). Uses self-attention to weigh token importance.
  • Autoregressive: Predicts next token (OpenAI GPT).
  • Autoencoding: Bidirectional context (Google BERT).
• Transfer Learning: Pretraining on diverse corpora followed by Fine-tuning on task-specific datasets.
• Parameters: Variables (weights and biases) adjusted during training. GPT-3 had $1.75 \times 10^{11}$ parameters; GPT-4 is rumored to exceed $10^{12}$.
• Alignment: Ensuring outputs meet user requirements via RLHF (Reinforcement Learning from Human Feedback) or DPO (Direct Preference Optimization).
• Multimodal LLMs: Process text, images, audio, and video (e.g., OpenAI Sora, DeepMind Flamingo).

Proprietary vs. Open Source Models

• Proprietary: OpenAI (GPT), Anthropic (Claude), Google (Gemini). Benefits: Financial resources, scalability. Risks: Vendor lock-in, privacy concerns (Sam Altman firing in late 2023 caused customer worry).
• Open Source/SLMs: Mistral (Arthur Mensch). Focused on efficiency and transparency.
  • SLMs (Small Language Models): Better for on-device use (e.g., Apple smartphones) to reduce latency and improve privacy.
• Synthetic Data: Artificially created data to solve the problem of exhausting web data and reducing licensing costs.

Prompt Engineering Techniques

• Clarity/Details: Specific instructions (e.g., "Explain gravity for a high school student").
• Persona: Assigning roles (e.g., "Respond as a seasoned business consultant").
• Delimiters: Using triple quotes to demarcate text sections.
• Chain of Thought: Prompting the model to "think step by step" to improve logical reasoning.
• Recursive Summarization: Summarizing long documents by summarizing sections then summarizing those summaries.

Types of AI Agents (Conceptual Categories)

• Simple Reflex Agents: If-then rules based only on current percepts (e.g., Thermostat starts AC if temp $> 45^\circ\text{C}$).
• Model-Based Reflex Agents: Use internal models to track unobservable aspects of the environment.
• Goal-Based Agents: Driven by specific objectives; use search and planning algorithms.
• Utility-Based Agents: Use a utility function to evaluate the "desirability" of states (e.g., financial trading optimizations).
• Learning Agents: Improve over time via four components: Learning element, Critic, Performance element, and Problem generator.
• Hierarchical Agents: Multi-tiered systems where high-level agents set goals and low-level agents execute specific tasks.

OpenAI Systems and Assistants API

• GPTs: Custom versions of ChatGPT available in the GPT Store. Can use Knowledge (uploaded files like PDFs) and Actions (API connections to Slack/Zapier).
• Tokens: Models process text as tokens ($1000\text{ tokens} \approx 750\text{ words}$).
• Pricing (GPT-4o): Prompt: 5\text{ per }1,000,000\text{ tokens}$; Response: 15\text{ per }1,000,000\text{ tokens}$.
• Assistants API Components:
  • Assistant: Configured with instructions and tools.
  • Thread: Stores message history for state/memory.
  • Run: The execution of the assistant on a thread.
  • Tool Pricing: Code Interpreter (0.03\text{ per session}$); File Search (0.10\text{ per GB per day}$).
• The o1 Model: Uses reinforcement learning to "think before answering." Performance statistics:
  • AIME (2024): o1 achieved $83.3\%$ vs. GPT-4o's $13.4\%$.
  • CodeForces: Ranked in the $89\text{th}$ percentile.
  • GPQA Diamond: Performance on par with human Ph.D. level in physics/chemistry.

Development Tools and Frameworks

• Environments:
  • Jupyter Notebook: Web-based interactive coding.
  • VS Code: Full IDE with extensions for AI development.
  • Google Colab: Cloud-hosted, offering free access to GPUs/TPUs (Pro/Pro+ tiers cost $9.99\text{-}$49.99/month).
• UI Libraries: Streamlit and Gradio for creating quick web demos. Jupyter Widgets for slider/button interactivity.
• Local LLMs: Ollama allows running models (Llama 3, Mistral) locally on Windows/Linux/Mac. Benefits: Privacy, no internet dependence, no usage fees.

Major Frameworks for AI Agents

1. CrewAI

• Focus: Role-playing and collaborative teams. Built on LangChain.
• Agents: Defined by role, goal, and backstory.
• Tasks: Specific assignments with expected_output.
• Processes:
  • Sequential: One after another.
  • Hierarchical: Uses a manager_llm for delegation/oversight.
• Memory Integration: Includes Short-Term, Long-Term, Entity, and Contextual (keeps history across different conversations).
• Tools: Includes ScrapeWebsiteTool, SerperDevTool (requires API from serper.dev), and searches for CSV/PDF.

2. AutoGen (Microsoft)

• Focus: Conversational multi-agent systems.
• ConversableAgent: The base class for agents communicating with each other.
• Reflection Agent Workflow: Example of a TweetWriter whose output is reviewed by a ContentOptimizer, SEOReviewer, and LegalReviewer before returning a final version.
• GroupChat: Multiple agents (User Proxy, Tech Support, Product Expert) solving complex support tickets.
• AutoGen Studio: Low-code UI for defining skills, models, and workflows.

3. LangChain

• Focus: Orchestration. Chains prompts and components.
• LCEL (LangChain Expression Language): Uses the pipe operator (|) for chaining (e.g., prompt | model | parser).
• Output Parsers: Converts LLM text into structures like JSON (using Pydantic BaseModel).
• Text Splitters: RecursiveCharacterTextSplitter and HTMLHeaderTextSplitter (keeps h1, h2, h3 context) ensure text fits context windows.
• ReAct Agent: Synergizing Reasoning and Acting. Uses logic traces (thoughts) to solve tasks via tools.

4. LangGraph

• Focus: Stateful, cyclic workflows.
• Structure: Uses Graphs consisting of Nodes (Python functions) and Edges (conditional or fixed transitions).
• Persistence: Uses checkpointers (SQLite/MongoDB) to save state after every super-step. Allows for manual intervention (Human-in-the-loop).
• LangSmith: Platform for tracing, debugging, and monitoring latency/cost of agent calls.
• LangGraph Studio: Desktop application for visual prototyping of graph workflows.

5. Haystack (Deepset)

• Focus: RAG pipelines and enterprise NLP.
• Components: InMemoryDocumentStore, InMemoryBM25Retriever.
• OpenAIFunctionCaller: Automatically detects when to call functions (e.g., fetching weather info) based on user queries.

Key Takeaways and Comparison

• Industry Adoption:
  • ServiceNow Xanadu: Automates Wi-Fi troubleshooting and IT service tickets.
  • Salesforce Agentforce: Autonomous agents for CRM/Marketing; goal of $1,000,000,000$ agents by end of fiscal year 2026.
  • Workday Recruiter Agent: Automates talent outreach and interview scheduling.
  • Oracle: $50+$ role-based agents for ERP/HCM systems.
• Innovation Pace: Jensen Huang (Nvidia) describes the current era as "Moore's Law squared."