Apr 9
Introduction to AI Decision-Making Systems
- Discussed the implementation of AI decision-making systems in banking.
- Focus on the case of Sasha, who was denied a mortgage by an AI system.
- Highlighted the complexity and multiple factors behind AI decisions.
Types of Explanations
- People Explanations: Reasons why individuals or groups made specific decisions.
- Machine Explanations: Clarifications regarding how AI reached a particular decision.
- Global Machine Explanations: General rules applied by the AI in decision-making.
- Example: "Anyone with a 4% or higher likelihood of defaulting on a mortgage loan gets denied."
- Local Machine Explanations: Reasons for specific decisions regarding individual applicants.
- Questions might include: "Why did Sasha change jobs frequently?" or "Why did she post on a specific social media thread?"
Complex Patterns in AI Decision-Making
- AIs process large amounts of data to recognize intricate patterns.
- Example: Recognition of social media interaction influencing mortgage approval. - Discussed how one small factor among many can alter the decision significantly.
- The AI might analyze complex relationships among numerous data points (e.g., 100,000 points) that humans find hard to comprehend.
- Illustrated with an example of dog recognition AI that processes images at the pixel level.
Comparing Human and Machine Explanations
- Human Explanations: Made in understandable terms based on social and ethical reasoning.
- Machine Explanations: Often highly complex and quantitative, challenging for humans to grasp.
- Importance of clarity: AIs might provide decisions that align with mathematical principles beyond human understanding.
Demand for Explanations by Users
- Users (like Sasha) require clarity on decisions made by AI.
- A distinction between the types of explanations desired by individuals.
- People Explanations: Typically sought for understanding and emotional closure.
- Machine Explanations: Can provide insight into decision-making processes involving complex data.
Cost of Explanations in AI
- Trade-off: There is a technical cost associated with creating explainable AI systems which often reduce accuracy.
- The more variables included, the harder it might be to make complete sense of AI's decision-making process. - Example Given: If a black box AI achieves a 99.8% accuracy rate with no explanation, flipping to explainable AI might decrease that accuracy.
- System usability: The implications of requiring explanations must be balanced against maintaining accurate decision-making.
Importance of Explanations in AI
- Reasons for requiring explanations include:
1. Respect: Not providing an explanation is disrespectful to individuals and their worth.
2. Potential Actions: Explanations allow individuals to take corrective actions based on the AI’s decisions (e.g., improving credit score).
3. Understanding the Rules: Enables users to comprehend whether the AI's rules are fair and justifiable, ensuring no bias or discrimination.
Use Cases for Black Box AI vs Explainable AI
Black Box AI Usage Scenarios:
- Decisions not directly impacting individuals: Examples include AI predicting delivery times or optimizing farm outputs.
- When informed consent is obtained from those impacted, and they understand the risks of black box decisions.Explainable AI Necessity:
- Situations where individuals directly face consequences (e.g., loans, job applications, criminal sentencing).
- When AI-related decisions can lead to loss, harm, or discrimination.
- Cases where humans require clarity on AI outputs that conflict with expert opinions.
- Cases requiring justification of treatment based on AI decisions, ensuring ethical guidelines are followed.
Evaluation of Good Explanations
- Consider key attributes of a "good explanation":
1. Demonstration of Respect: Acknowledging the individual's worth and stimulating human dignity.
2. Facilitation of Informed Decisions: Allowing users (e.g., Sasha) to understand how they might alter future outcomes.
3. Assessment of Ethical Justification: Enabling users or stakeholders to evaluate the justifiability of the rules utilized by the AI.