Generative AI in Business - Week 8 Notes

Learning Objectives

• Define what Generative AI is.
• Explain how generative AI works.
• Differentiate between language models and diffusion models.

What is Generative AI?

• Definition: Generative Artificial Intelligence (GenAI) refers to AI that creates new content, which can include text, images, code, music, etc.
• Basis: It creates content based on patterns it has learned from large datasets.

Examples of Generative AI:

• Text Generators:

  • ChatGPT
  • Copilot
  • Claude
  • Gemini

• Image Generators:

  • DALL·E
  • MidJourney
  • Stable Diffusion
  • Adobe Firefly

• Code Generators:

  • GitHub Copilot
  • Codeium

• Voice Generators:

  • ElevenLabs
  • Speechify

• Writing Assistants:

  • Grammarly

• Research Assistants:

  • NotebookLM

How Generative AI Works

1. Training: GenAI models are trained on very large datasets, known as big data.

   • Datasets Include:
     • Text: Books, articles, websites, conversations.
     • Images: Photographs, paintings, illustrations.
     • Audio: Speech recordings, music tracks.
     • Video: Movies, YouTube clips, surveillance footage.

2. Learning: The model learns patterns, structures, and relationships in the data.

3. Content Generation: It creates new content based on the learned patterns.

Example Output Process:

• The AI processes input data through various stages leading to the generation of new content.

Model Types

1. Language Models

• Functionality:
  • Example Prompt: "The cat is on __?"
  • Predict the next word or phrase based on previous context.
  • Examples: Generates human-like text for chatbots, translation, writing aids.

2. Diffusion Models

• Functionality:
  • Create images from random noise, employing a step-by-step refinement process.
  • Applications include art, marketing, and design.
  • Examples: Stable Diffusion, Mid Journey, DALL·E.

Deep Learning Foundations

• Utilizes neural networks with many layers to detect complex patterns in data.
• Powers generation of text, image, and audio content.

Transformers: The AI Game Changer

• Mechanism:
  • Processes entire inputs at once rather than sequentially.
  • Employs self-attention to identify key information within the data.
  • Foundation for advanced models such as GPT (Generative Pre-trained Transformer).

How ChatGPT Works

1. Input Processing:

   • User prompts are input as text (e.g., "Once upon a time, in a distant kingdom").
   • The prompt is tokenized into individual components (e.g., "Once", "upon", "a", etc.).

2. Embedding Process:

   • Tokens are converted to numerical representations.

3. Attention Mechanism:

   • Weights are assigned to tokens according to their importance (e.g., location descriptors and narrative indicators).

4. Prediction:

   • The model predicts the next token in the sequence.

5. Repetition:

   • This process is repeated until the content generation is completed.
   • Example Generated Text: "Once upon a time, in a distant kingdom, there lived a brave knight who sought to slay a fearsome dragon."