Generative AI Notes

What is AI?

  • Artificial Intelligence (AI) refers to intelligent machines capable of performing tasks that typically require human intelligence.

  • AI encompasses the theory and development of computer systems designed to perform tasks requiring human intelligence:

    • Recognizing speech

    • Making decisions

    • Identifying patterns

  • AI systems are designed to respond to specific inputs.

  • These systems learn from data to make decisions or predictions.

Traditional AI

  • Traditional AI involves systems programmed with predefined strategies, like a computer chess player selecting from known strategies.

  • Examples:

    • Voice assistants like Siri or Alexa

    • Recommendation engines (Netflix, Amazon)

    • Google's search algorithm

  • These AIs follow specific rules and excel at particular tasks but do not create anything new.

Generative AI

  • Generative AI can create new content such as:

    • Text

    • Images

    • Audio

    • Videos

    • 3D models

  • Generative AIs analyze existing data and user input to generate content.

  • Providing input to Generative AIs is known as AI Prompt Writing or AI Prompt Engineering.

  • Analogy: An AI 'friend' who can generate a space adventure story from the starting line, 'Once upon a time, in a galaxy far away…'

  • Generative AI can produce text, images, music, and computer code.

  • Generative AI models are trained on data sets to learn underlying patterns, enabling them to generate new, similar data.

Pattern Recognition vs. Pattern Creation

  • Traditional AI: excels at pattern recognition.

  • Generative AI: excels at pattern creation.

  • Traditional AI analyzes data to describe what it sees.

  • Generative AI uses data to create something entirely new.

Generative Models

  • Generative models use machine learning to find patterns in data and generate new data.

  • Analogy: A child learning to draw animals and later creating a new animal based on learned features.

AI Layers

  • The hierarchy includes:

    • Artificial Intelligence (AI)

    • Machine Learning (ML)

    • Deep Learning

    • Generative AI

    • Large Language Models (LLM)

    • Generative Pre-Trained Transformers (GPT)

    • GPT-4

    • ChatGPT

Generative vs. Discriminative Models

  • Generative models: understand how data is generated and learn the distribution of the data itself.

    • Example: Understanding what makes a cat look like a cat to generate new cat images.

  • Discriminative models: distinguish between different types of data without necessarily understanding how the data is generated.

    • Example: Telling the difference between cats and dogs without generating new images.

Types of Generative Models

  • Generative Adversarial Networks (GANs):

    • Two neural networks (generator and discriminator) trained together.

    • The generator produces data, while the discriminator distinguishes between real and generated data.

    • Used in image generation tasks such as creating realistic human faces.

  • Variational Autoencoders (VAEs):

    • Autoencoders that produce a compressed representation of input data, then decode it to generate new data.

    • Used in image denoising or generating new images with similar characteristics to the input data.

  • Autoregressive Models

    • Future values are predicted using past data from the same series.

    • Employs lagged data of the relevant variable to generate predictions.

  • Transformer Models

    • Neural network architecture using self-attention to determine the significance of words in a phrase.

    • Achieves state-of-the-art performance in natural language processing tasks like machine translation and sentiment analysis.

AI Prompt Writing/Engineering

  • AI Prompt Writing (or Engineering) involves creating input (usually text) to instruct Generative AI to produce the desired response.

  • Prompting is how we ask an AI to do something.

  • Prompts should be tailored to the type of response desired and the specific Generative AI used.

  • Types of prompts:

    • Instructions

    • Questions

    • Data

    • Examples

General Rules for Prompt Writing

  • Start simple and build on it.

  • Use a Call to Action: start prompts with action words (e.g., "Write", "Create", "Summarize").

  • Add specific and relevant context.

  • Add clear expectations for the content (e.g., length, what to include).

Prompt Example and Response

  • Prompt: