Apr 9

Introduction to AI Decision-Making Systems

  • Discussed the implementation of AI decision-making systems in banking.
  • Focus on the case of Sasha, who was denied a mortgage by an AI system.
  • Highlighted the complexity and multiple factors behind AI decisions.

Types of Explanations

  • People Explanations: Reasons why individuals or groups made specific decisions.
  • Machine Explanations: Clarifications regarding how AI reached a particular decision.
      - Global Machine Explanations: General rules applied by the AI in decision-making.
        - Example: "Anyone with a 4% or higher likelihood of defaulting on a mortgage loan gets denied."
      - Local Machine Explanations: Reasons for specific decisions regarding individual applicants.
        - Questions might include: "Why did Sasha change jobs frequently?" or "Why did she post on a specific social media thread?"

Complex Patterns in AI Decision-Making

  • AIs process large amounts of data to recognize intricate patterns.
      - Example: Recognition of social media interaction influencing mortgage approval.
  • Discussed how one small factor among many can alter the decision significantly.
  • The AI might analyze complex relationships among numerous data points (e.g., 100,000 points) that humans find hard to comprehend.
  • Illustrated with an example of dog recognition AI that processes images at the pixel level.

Comparing Human and Machine Explanations

  • Human Explanations: Made in understandable terms based on social and ethical reasoning.
  • Machine Explanations: Often highly complex and quantitative, challenging for humans to grasp.
  • Importance of clarity: AIs might provide decisions that align with mathematical principles beyond human understanding.

Demand for Explanations by Users

  • Users (like Sasha) require clarity on decisions made by AI.
  • A distinction between the types of explanations desired by individuals.
      - People Explanations: Typically sought for understanding and emotional closure.
      - Machine Explanations: Can provide insight into decision-making processes involving complex data.

Cost of Explanations in AI

  • Trade-off: There is a technical cost associated with creating explainable AI systems which often reduce accuracy.
      - The more variables included, the harder it might be to make complete sense of AI's decision-making process.
  • Example Given: If a black box AI achieves a 99.8% accuracy rate with no explanation, flipping to explainable AI might decrease that accuracy.
  • System usability: The implications of requiring explanations must be balanced against maintaining accurate decision-making.

Importance of Explanations in AI

  • Reasons for requiring explanations include:
      1. Respect: Not providing an explanation is disrespectful to individuals and their worth.
      2. Potential Actions: Explanations allow individuals to take corrective actions based on the AI’s decisions (e.g., improving credit score).
      3. Understanding the Rules: Enables users to comprehend whether the AI's rules are fair and justifiable, ensuring no bias or discrimination.

Use Cases for Black Box AI vs Explainable AI

  • Black Box AI Usage Scenarios:
      - Decisions not directly impacting individuals: Examples include AI predicting delivery times or optimizing farm outputs.
      - When informed consent is obtained from those impacted, and they understand the risks of black box decisions.

  • Explainable AI Necessity:
       - Situations where individuals directly face consequences (e.g., loans, job applications, criminal sentencing).
       - When AI-related decisions can lead to loss, harm, or discrimination.
       - Cases where humans require clarity on AI outputs that conflict with expert opinions.
       - Cases requiring justification of treatment based on AI decisions, ensuring ethical guidelines are followed.

Evaluation of Good Explanations

  • Consider key attributes of a "good explanation":
      1. Demonstration of Respect: Acknowledging the individual's worth and stimulating human dignity.
      2. Facilitation of Informed Decisions: Allowing users (e.g., Sasha) to understand how they might alter future outcomes.
      3. Assessment of Ethical Justification: Enabling users or stakeholders to evaluate the justifiability of the rules utilized by the AI.