2) Amazon Bedrock (GenAI)

Fundamentals of Generative AI and Foundation Models

• Generative AI Definition: Generative AI (Gen-AI) is a specialized subset of Deep Learning engineered to generate original data that closely mirrors the patterns and characteristics of its training dataset.

• Supported Data Modalities: Generative models process and produce content across diverse modalities, including:

  • Text

  • Images

  • Audio

  • Executable code

  • Video

• Generative Workflow Structure:

  • Training Phase: Unlabeled raw data is ingested to train a core model.

  • Inference Phase: Users provide specific prompts (e.g., requesting a "cartoon dog"), prompting the generative model to evaluate learned patterns and output corresponding new data.

• Foundation Models (FMs):

  • Generative AI capabilities rely on Foundation Models trained on massive, diverse datasets.

  • Training Investment: Developing a base foundation model can require financial investments ranging into tens of millions of dollars.

  • Core Architectural Pipeline: Unlabeled Data $\rightarrow$ Pre-training Phase $\rightarrow$ Base Foundation Model $\rightarrow$ Fine-tuning/Adaptation $\rightarrow$ Deployment across broad task domains.

  • Supported Core Tasks: Text generation, text summarization, information extraction, image generation, chatbot conversational systems, and question-answering engines.

• Major Providers and Licensing Models:

  • Key Industry Providers: OpenAI, Meta (Facebook), Amazon, Google, and Anthropic.

  • Commercial Closed Models: Models requiring proprietary commercial licenses (e.g., OpenAI's GPT-4o behind ChatGPT, Anthropic Claude).

  • Open-Source / Open-Weight Models: Models distributed freely for deployment and adaptation (e.g., Meta's Llama series, Google BERT).

Generative Language Models and Token Selection

• Large Language Models (LLMs):

  • A class of foundation models designed specifically to understand, summarize, and generate coherent human-like text.

  • Corpus and Scale: Trained on vast textual corpora (books, articles, websites, documentation) and usually contain billions of parameters (e.g., GPT-4).

  • Primary Tasks: Language translation, document summarization, automated question answering, and dynamic content creation.

• Prompt-Driven Interaction:

  • Interaction occurs by feeding a text prompt to the LLM.

  • The model evaluates its internalized contextual representations to construct output text token by token.

• Non-Deterministic Property:

  • Generative language models are inherently non-deterministic: submitting the exact same prompt multiple times or across different sessions can yield distinct output text variants.

• Probabilistic Next-Token Generation:

  • LLMs generate text by outputting a ranked list of potential candidate words alongside their calculated probability scores.

  • Selection algorithms sample from this candidate list based on probability distributions (incorporating controlled randomness).

• Step-by-Step Probability Example:

  • Step 1 Context: "After the rain, the streets were [Candidate Next Token]"

    • wet: $0.40$

    • flooded: $0.25$

    • slippery: $0.15$

    • empty: $0.10$

    • muddy: $0.05$

    • clean: $0.03$

    • blocked: $0.02$

  • Step 2 Context (after selecting "flooded"): "After the rain, the streets were flooded [Candidate Next Token]"

    • and: $0.30$

    • with: $0.20$

    • but: $0.15$

    • from: $0.12$

    • until: $0.10$

    • because: $0.08$

    • .: $0.05$

Generative Multimodal and Diffusion Models

• Text-to-Image Generation:

  • Generates visual scenes directly from text prompts.

  • Example Prompt: "Generate a blue sky with white clouds and the word 'Hello' written in the sky."

• Image-to-Image Transformation:

  • Takes an existing image as input alongside instructions to modify its aesthetic or domain.

  • Example Prompt: "Transform this image in Japanese anime style."

• Image-to-Text Analysis (Multimodal Interpretation):

  • Evaluates visual input to produce natural language descriptions.

  • Example Input: Image containing an orange and a red apple.

  • Prompt: "Describe how many apples you see in the picture."

  • Generated Response: "The picture shows one apple. The other fruit is an orange."

• Image Generation via Diffusion Models:

  • Architecture: Models such as Stable Diffusion utilize forward and reverse diffusion processes.

  • Forward Diffusion (Training): Sequentially adds Gaussian noise to a training image until it becomes pure random noise.

  • Reverse Diffusion (Inference): The model is trained to denoise random noise step-by-step, guided by text embeddings (e.g., "a cat with a computer"), generating a crisp image.

Amazon Bedrock Architecture and Core Capabilities

• Fully-Managed AWS Service:

  • Enables building and scaling Generative AI applications without infrastructure management overhead.

• Data Control and Privacy:

  • Users maintain complete ownership and control over data used for inference or model customization.

  • Customer data is completely isolated and is never used to train the underlying base foundation models.

• Unified API Access:

  • Provides a standardized, single API interface to call, swap, and integrate diverse foundation models seamlessly into enterprise codebases.

• Core Out-of-the-Box Features:

  • Built-in Retrieval-Augmented Generation (RAG) through Knowledge Bases.

  • Autonomous multi-step task execution through Bedrock Agents.

  • Comprehensive enterprise security, compliance, data privacy, governance, and Guardrails for Responsible AI.

Amazon Bedrock Foundation Models and Comparison

• Model Selection Criteria:

  • Choosing an FM depends on model types, performance requirements, capabilities, constraints, regulatory compliance, level of customization, parameter size, inference options, licensing, context window capacity, and acceptable latency limits.

• Amazon Titan Model Family:

  • AWS-native, high-performing foundation models covering text, image, and multimodal tasks via managed APIs.

  • Fully customizable using proprietary data.

  • Smaller Titan variants offer enhanced cost efficiency for focused workloads.

• Detailed Model Specification Matrix:

  • Amazon Titan (Titan Text Express):

    • Context Window: $8\text{K tokens}$ ($8,000$

    • Key Features: High-performance text processing, supports $+100$ languages.

    • Primary Use Cases: Content creation, text classification, education tools.

    • Pricing: Input: \\$0.0008 per $1\text{K tokens}$; Output: \\$0.0016 per $1\text{K tokens}$.

  • Meta Llama (Llama-2 70b-chat):

    • Context Window: $4\text{K tokens}$ ($4,000$

    • Key Features: Large-scale tasks, complex multi-turn dialogue, optimized for English.

    • Primary Use Cases: Conversational text generation, customer support bots.

    • Pricing: Input: \\$0.0019 per $1\text{K tokens}$; Output: \\$0.0025 per $1\text{K tokens}$.

  • Anthropic Claude (Claude 2.1):

    • Context Window: $200\text{K tokens}$ ($200,000$

    • Key Features: High-capacity text generation, complex multi-language processing.

    • Primary Use Cases: Deep document comparison, forecasting, detailed legal/financial analysis.

    • Pricing: Input: \\$0.008 per $1\text{K tokens}$; Output: \\$0.024 per $1\text{K tokens}$.

  • Stability AI Stable Diffusion (SDXL 1.0):

    • Context Window: $77\text{ Tokens/Prompt}$

    • Key Features: Text-to-image generation.

    • Primary Use Cases: Commercial image creation for advertising, design, and digital media.

    • Pricing: \\$0.04 to \\$0.08 per generated image.

Model Customization: Fine-Tuning Techniques

• Core Concept of Fine-Tuning:

  • Fine-tuning creates a dedicated, private copy of a base Foundation Model and adjusts its internal weights using user-provided dataset samples.

• Data and Storage Requirements:

  • Training data must adhere strictly to specified JSON schema formats.

  • All training datasets must be stored in Amazon S3 buckets.

  • Note: Fine-tuning availability varies depending on the specific FM chosen.

• Supervised Fine-Tuning (SFT):

  • Targeted learning to boost model performance on specific domains or task types.

  • Uses explicit labeled datasets consisting of input prompt and expected completion target pairs.

  • JSON Example Structure: json { "prompt": "Who is Stéphane Maarek?", "completion": "Stéphane Maarek is an AWS instructor who dedicates his time to make the best AWS courses so that his students can pass all AWS certification exams with flying color!" }         

• Reinforcement Fine-Tuning (RFT):

  • Uses feedback-based iterative learning to align model outputs with desired behaviors.

  • Workflow Pipeline:

    1. User supplies a set of training input prompts.

    2. The base model generates multiple candidate responses.

    3. A Reward Function evaluates candidate responses and assigns quantitative quality scores.

    4. The model iteratively updates parameters to maximize generated output scores over time.

  • Reward Function Implementation:

    • Objective Tasks: Automated via AWS Lambda functions executing deterministic Python scoring logic.

    • Subjective Tasks: Uses a separate high-capability "Judge Model" prompted with explicit scoring rubrics.

  • RFT Execution Scenario (Customer Support Bot):

    • Prompt: "My app is running very slowly."

    • Response 1: "Restart the app" $\\rightarrow$ Score: $5.0$ (Helpful but superficial; does not diagnose underlying root causes).

    • Response 2: "That sounds frustrating. I can help you figure this out. Before we troubleshoot, can you tell me when it started and what you were doing at the time?" $\\rightarrow$ Score: $9.0$ (Empathetic, structured diagnostic approach that acknowledges frustration and gathers signals).

    • Response 3: "Please open a support ticket and attach logs A, B, and C." $\\rightarrow$ Score: $2.0$ (Unhelpful, creates friction by pushing work back to the user, disrupting conversational flow).

• Supervised Fine-Tuning vs. Reinforcement Fine-Tuning:

  • Supervised Fine-Tuning: Both input prompts and expected ground-truth completion targets are explicitly provided in advance.

  • Reinforcement Fine-Tuning: Input prompts are provided, but candidate outputs are generated dynamically and evaluated via calculated reward scores ($2.0$, $5.0$, $9.0$) to guide parameter updates.

• Fine-Tuning Costs and Operational Requirements:

  • Requires substantial compute budgets and specialized ML engineering skills for dataset preparation, execution, and validation.

  • Supervised fine-tuning is less compute-intensive than full domain re-training.

  • Custom models incur separate runtime hosting fees billed either via On-Demand token rates or Provisioned Throughput.

• Primary Fine-Tuning Use Cases:

  • Custom brand persona, corporate tone, or specific agent behavior (e.g., specialized ad copy creation).

  • Incorporating recent or proprietary domain-specific knowledge not included in base model pre-training.

  • Training on exclusive internal records (email history, enterprise support logs).

  • Specialized downstream tasks like specialized text classification or accuracy verification.

Model Distillation and Efficiency

• Model Distillation Definition:

  • A compression technique where knowledge from a large, highly accurate "Teacher Model" is transferred into a much smaller, faster "Student Model".

• Key Benefits:

  • Reduces operational inference costs by up to $75\%$ compared to the original teacher model.

  • Substantially increases inference speed and lowers latency.

  • Accepts a minor tradeoff in overall model accuracy in exchange for high efficiency and lower compute requirements.

• Distillation Process:

  • Sample input dataset prompts are processed through the Teacher Model.

  • The Student Model is trained to mimic the outputs and reasoning behavior of the Teacher Model across these inputs.

Amazon Bedrock Model Evaluation and Benchmarking

• Automatic Model Evaluation:

  • Enables automated quality assessment without manual human intervention.

  • Supported Built-In Tasks: Text summarization, question answering, text classification, open-ended generation.

  • Datasets: Uses custom enterprise prompt datasets or built-in curated benchmark collections.

  • Evaluation Engine: Utilizes statistical algorithms or automated Judge Models to evaluate model outputs against reference ground-truth answers.

• Benchmark Datasets:

  • Curated datasets tailored to measure accuracy, speed, computational efficiency, scalability, and demographic bias.

  • Allows immediate detection of algorithmic bias or unfair performance discrepancies across groups.

• Human Model Evaluation:

  • Uses human reviewer teams (internal company employees or Subject-Matter Experts) to grade outputs.

  • Evaluators submit scores via metrics like thumbs up/down, comparative preference rankings, or numerical scales.

• Automated Evaluation Metrics:

  • ROUGE (Recall-Oriented Understudy for Gisting Evaluation):

    • Measures textual overlap for summarization and translation.

    • ROUGE-N: Measures matching $n$-gram overlaps between reference and generated text.

    • ROUGE-L: Evaluates the Longest Common Subsequence (LCS) shared between generated and reference texts.

  • BLEU (Bilingual Evaluation Understudy):

    • Evaluates translation quality by combining precision across multiple $n$-grams ($1, 2, 3, 4$ ) while penalizing excessive brevity.

  • BERTScore:

    • Evaluates semantic similarity using contextualized embeddings from pre-trained BERT models.

    • Calculates the cosine similarity between contextual vectors, capturing deep semantic equivalence beyond literal word matches.

  • Perplexity:

    • Measures how effectively a language model predicts the next token in a sequence.

    • Key Rule: Lower perplexity values indicate superior language modeling performance.

• Enterprise Business Metrics for Model Assessment:

  • User Satisfaction: Direct feedback ratings collected from end users during live platform interaction.

  • Average Revenue Per User (ARPU): Revenue increases directly attributable to Gen-AI features.

  • Cross-Domain Performance: Measuring model generalizability across diverse business domains.

  • Conversion Rate: Tracking user completion of intended business actions (e.g., dynamic product descriptions driving higher retail checkout rates).

  • Operational Efficiency: Resource usage efficiency, execution latency, and compute cost metrics.

Retrieval-Augmented Generation (RAG) and Knowledge Bases

• RAG Concept:

  • Architecture that allows Foundation Models to reference dynamic external database content outside their training data before generating a response.

  • Eliminates the need to re-train or fine-tune models to incorporate new knowledge.

• Bedrock RAG Execution Pipeline:

  1. Query Input: User submits a prompt (e.g., "Who's the product manager for John?").

  2. Vector Retrieval: Bedrock queries a vector database containing vector embeddings of internal documents.

  3. Context Extraction: Relevant text chunks are retrieved (e.g., "John Product Info: Product Manager: Jessie Smith").

  4. Prompt Augmentation: System combines original user prompt + retrieved context into an Augmented Prompt.

  5. Grounded Generation: FM processes the augmented prompt to output an accurate response ("Jessie Smith is the Product Manager for John.").

• RAG Vector Database Ingestion Pipeline:

  • Raw Documents in S3 $\rightarrow$ Document Chunking $\rightarrow$ Embedding Model Processing (Titan Embeddings) $\rightarrow$ Vector Database Storage.

• AWS Vector Database Options:

  • Amazon OpenSearch Service (Serverless & Managed Cluster): High-scale search engine delivering fast $k$-nearest neighbor ($k\text{NN}$) similarity searches across millions of vector embeddings.

  • Amazon Aurora PostgreSQL: Relational database supporting vector embeddings via the pgvector extension.

  • Amazon Neptune Analytics: High-performance graph database optimized for graph-based RAG (GraphRAG).

  • Amazon S3 Vectors: Cost-effective, durable object storage with sub-second vector search performance.

• Supported Data Source Connectors:

  • Amazon S3

  • Atlassian Confluence

  • Microsoft SharePoint

  • Salesforce

  • Web Pages (crawled websites and social feeds)

• Target RAG Use Cases:

  • Customer Service Chatbots: Grounded answers using product specifications, troubleshooting manuals, and FAQs.

  • Legal Analysis: Querying statutes, regulatory codes, legal precedents, and expert opinions.

  • Healthcare Portals: Retrieval across medical research, clinical guidelines, and treatment protocols.

Key Generative AI Technical Concepts

• Tokenization:

  • Process of breaking raw input text down into discrete numerical units called tokens.

  • Word-based tokenization: Splits text strictly on word boundaries.

  • Subword tokenization: Breaks complex or long words down into smaller subword chunks (e.g., converting "learning" into subword units).

• Context Window:

  • The maximum number of total tokens (input prompt + generated output) an LLM can evaluate in a single interaction.

  • Trade-off: Larger context windows enable processing entire documents and maintaining long-term conversational coherence, but demand significantly higher memory and computational resources.

• Embeddings:

  • Dense mathematical vectors (arrays of floating-point numbers) representing text, images, or audio.

  • High-dimensional vector spaces capture semantic meaning, syntactic structure, and sentiment.

  • Semantic Similarity: Words with similar meanings (e.g., dog, puppy, cat) map to nearby vector coordinates, whereas unrelated words (e.g., houses) reside far away in vector space.

Amazon Bedrock Guardrails and Agents

• Amazon Bedrock Guardrails:

  • Programmable safety and governance layer managing interactions between users and FMs.

  • Capabilities:

    • Blocks undesirable or harmful content categories.

    • Prevents restricted discussion topics (e.g., rejecting culinary recipe prompts on a financial banking bot).

    • Detects and redacts Personally Identifiable Information (PII) to ensure privacy compliance.

    • Reduces hallucinations and enforces output accuracy boundaries.

  • Supports tracking policy violations across applications.

• Amazon Bedrock Agents:

  • Fully-managed orchestration components that plan and execute multi-step business workflows autonomously.

  • Task Coordination: Uses Chain of Thought reasoning to break down user requests, execute steps in logical order, and pass state between operations.

  • Action Groups: Map agent decisions to external execution tools using OpenAPI schemas and AWS Lambda functions.

  • System Integration: Connects dynamically with enterprise APIs, databases, external software, and Knowledge Bases.

Monitoring, Governance, and CloudWatch Integration

• Model Invocation Logging:

  • Captures full input prompts and generated responses (including text, image data, and vector embeddings) and routes logs to Amazon CloudWatch Logs and Amazon S3.

  • Enables deep log query analysis via CloudWatch Logs Insights.

• CloudWatch Metrics:

  • Automated emission of performance and operational metrics from Bedrock to CloudWatch.

  • Key Metric: ContentFilteredCount — tracks how frequently Guardrail policies block inappropriate user inputs or generated outputs.

  • Integrates with CloudWatch Alarms to send immediate administrative notifications when policy violation thresholds are exceeded.

Amazon Bedrock Pricing Models and Optimization

• Bedrock Pricing Structures:

  • On-Demand: Flexible pay-as-you-go billing without long-term usage commitments.

    • Text Models: Billed per $1\text{K}$ processed input tokens and $1\text{K}$ generated output tokens.

    • Embedding Models: Billed per $1\text{K}$ processed input tokens.

    • Image Models: Billed per individual generated image.

  • Batch Inference: Processes large datasets asynchronously (storing outputs as S3 files), providing cost discounts of up to $50\%$ compared to On-Demand.

  • Provisioned Throughput: Dedicated model units purchased for fixed term commitments ($1$ month or $6$ months).

    • Guarantees committed throughput (max input/output tokens per minute).

    • Mandatory requirement for hosting fine-tuned custom models.

• Cost Optimization Drivers:

  • Token Volume: Input and output token counts are the primary drivers of execution cost.

  • Model Size: Selecting smaller, task-specific models substantially lowers runtime expenses.

  • Hyperparameters: Adjusting sampling parameters like Temperature, Top K, or Top P has zero direct impact on Bedrock API pricing.

Model Improvement Cost Hierarchy

• Improvement Techniques (Ordered from Lowest Cost/Complexity to Highest):

  1. Prompt Engineering: Adjusts context and instruction framing without model training or additional compute costs.

  2. Retrieval-Augmented Generation (RAG): Supplies real-time external knowledge without altering base model parameters or requiring training compute.

  3. Instruction-Based Fine-Tuning: Trains the model on focused input-output target pairs; requires moderate compute investment.

  4. Domain Adaptation Fine-Tuning: Deeply re-trains base model weights on extensive, highly specialized domain datasets; requires intensive computational resources and high financial budgets.

Amazon Nova Foundation Model Suite

• Amazon Nova Overview:

  • AWS-native family of state-of-the-art foundation models designed to offer high speed, enterprise governance, and market-leading cost efficiency.

• Nova Model Lineup:

  • Understanding / Reason Models:

    • Nova Premier: Most capable multimodal model, optimized for complex reasoning tasks and acting as a teacher model for custom distillation.

    • Nova Pro: Highly capable multimodal model delivering an optimal balance of accuracy, speed, and cost across enterprise tasks.

    • Nova Lite / Nova 2 Lite: Fast, low-cost multimodal models optimized for rapid text, document, image, and video processing.

    • Nova Micro: Text-only model delivering minimal latency at the lowest price point.

  • Creative Models:

    • Nova Canvas: State-of-the-art image generation model.

    • Nova Reel: State-of-the-art video generation model.

  • Speech & Multimodal Models:

    • Nova Sonic / Nova 2 Sonic: Real-time, bidirectional speech-to-speech conversational model.

    • Nova 2 Multimodal Embeddings: Advanced embedding model designed for agentic RAG and high-accuracy semantic search applications.

    • Nova 2 Omni: All-in-one multimodal reasoning and image generation model supporting context windows up to $1\text{M tokens}$ ($1,000,000$ tokens).