Artificial Intelligence Notes

What is AI?

  • Rule-based expert systems: Systems that use defined rules to make decisions.
  • Machine-learning-based systems: Systems that learn from data to improve over time.
  • Neuro-symbolic methods: Combines expert knowledge and flexibility of machine learning.
  • Currently, AI is often associated predominantly with Machine Learning.

Brief History of AI

  • Alan Turing: Pioneer in AI thought, proposed the Turing Test.
  • Dartmouth Workshop: The formative conference where AI was first coined.
  • Perceptron: Early neural network model developed in the 1950s.
  • Expert Systems: Software that mimics human decision-making capabilities.
  • Simple statistical ML: Early machine learning approaches that used basic statistics.
  • Deep Learning: Advanced machine learning utilizing neural networks to process data at high levels of abstraction.

Modern AI vs. Traditional AI

  • Expert System Features:
    • Defined inputs (X) and functions (f).
    • Operation similar to a vending machine.
  • Modern ML/AI Features:
    • Utilizes vast data sets on inputs and outputs (X and Y).
    • Transfer of knowledge to a model that learns the function (f) rather than having it explicitly defined.
    • Functions become a complex, often opaque process (black box).

Features of Modern AI

  • Deep Learning Models: Primary driving force behind modern AI.
    • Functions as a large, complex black box.
  • Versatility of Use: Applications span numerous fields, from healthcare to finance.
  • Accuracy: Generally achieves high prediction accuracy but often lacks interpretability.
  • Example Issues: Classifying images as dogs or cookies can be challenging due to the black-box nature of deep learning.

Machine Learning Applications

Supervised Learning:
  • Input Examples:
    • Picture (Are there human faces? 0 or 1)
    • Loan application (Will they repay the loan? 0 or 1)
    • Ad with user data (Will the user click? 0 or 1)
  • Output Examples:
    • Speech recognition (transcription)
    • Language translation (English to French)
    • Preventive maintenance (predictive analysis for machinery)
    • Self-driving cars (navigation and obstacle avoidance).

Generative AI

  • Definitions: Models that can generate content (text, images, video, or sound) based on input prompts.
  • Examples:
    • Text: ChatGPT (OpenAI), Gemini (Google), LLaMA (Meta)
    • Images: Dall-E (OpenAI), Mid Journey, Stable Diffusion
    • Videos: Sora (OpenAI), Runway AI
    • Voices: ElevenLabs.

Training LLMs (Large Language Models)

  • Training Process: Models are trained on large text datasets (books, articles, websites).
  • Tokens: Text is broken down into numeric representations (example: “Apple” may be represented as 10001).
  • Inference: Models predict the next token in a sequence based on learned patterns, leading to more accurate and context-aware text generation.

Limitations of Tokenization

  • LLMs function based on tokens rather than individual characters.
  • Some tokens, especially those from programming, can lead to inaccuracies if the model hasn't been specifically trained on them.

Prompt Engineering

  • Purpose: The art of crafting prompts to elicit improved responses from LLMs.
  • Techniques:
    • Chain of Thoughts (CoT): Directing step-by-step thought processes within prompts.
    • Few-shot Learning: Providing examples in prompts for context and guidance.

Image Generation Models

  • Proprietary Models: Dall-E 3, Mid Journey, which generally perform well out of the box.
  • Open Source Models: Stable Diffusion, which can be customized and adapted for particular uses.
    • Example: Combining Stable Diffusion with control networks for generating specific images.

Video Generation

  • Challenges: Video generation is computationally intense with issues of consistency and control over output.
  • Example Tool: OpenAI's Sora generates high-fidelity videos from text prompts.

Voice Generation/Cloning

  • Resources: Websites like ElevenLabs allow text input for voice synthesis, including cloning features.

Collaboration with AI

How LLMs May Augment Human Capabilities
  • Modes of Collaboration:
    • Ghostwriting: AI generates full text for review or publication.
    • Soundboard: AI critiques, aids brainstorming, or enhances language style.
  • Experiment Insights: Using LLMs as ghostwriters may diminish creative potential due to anchoring effects; however, using them as soundboards increases performance.

Risks and Pitfalls of AI

  • Data Reliance: Exclusively relying on data can lead to significant blind spots and biases.
  • Hallucinations: LLMs can generate plausible but false content due to lack of understanding or meta-knowledge.

Responsible AI Use

  • Important to evaluate tasks suitable for AI and the collaboration modalities between users and AI systems.
  • Understanding and mitigating the societal implications and biases associated with AI, particularly in decision-making processes, is critical.

Bias in Image Generation Models

  • Bias arises from training data leading to misrepresentation and inaccuracies.
  • Strategies for addressing bias include data-debiasing methods or using explicit bias mitigation prompts during generation.

Conclusion: Societal Implications of AI

  • Generative AI's societal impacts include ethical considerations, misinformation, financial loss, and the dynamics of human-like interactions with AI, necessitating vigilant and responsible use.