Study Notes on Algorithmic Bias and (In)Justice

Algorithmic Bias and Algorithmic (In)Justice

Introduction

  • Discussion of algorithmic bias and its relationship to algorithmic injustice.

AI and Bias

  • Understanding Algorithm Bias
    • AI models learn from human-generated data.
    • Parameter Weights: AI adjusts its parameter weights as it sorts through the training dataset, forming implicit internal representations of what it learns.
    • Human Cognitive Biases: Humans possess various cognitive biases that are reflected in their thought processes and communication.
    • Inheritance of Bias: Consequently, AI trained on human-generated data inevitably inherits these biases without direct programming from developers.
    • Key Questions:
    • Why does this inheritance of bias occur?
    • Why is it challenging to rectify these biases?

Explicit vs. Implicit Bias

  • Understanding Human Bias:
    • To comprehend AI bias, it is crucial to first understand human bias.
    • Explicit Bias:
    • Definition: Bias that a person is aware of and consciously factors into decisions.
    • Example: Trusting advice from friends over strangers, being aware of this preference.
    • Implicit Bias:
    • Definition: Bias that subconsciously influences thoughts and decisions without awareness.
    • Example: Being nicer to physically attractive individuals without realizing the influence of physical appearance.

Cognitive Biases and Their Sources

  • Social and Cultural Influences:
    • Cognitive biases are shaped by societal and cultural beliefs, including detrimental beliefs such as racism and homophobia.
    • Negative Connotation: The contemporary perception of implicit bias is primarily negative due to its potential impact on behavior.
    • Survival Necessity: Not all implicit biases are harmful; some cognitive biases are essential for survival.

The Limitation of AI in Recognizing Bias

  • Differentiating Implicit Biases:
    • Humans can often recognize and modify ‘good’ versus ‘bad’ implicit biases through reflection and education.
    • AI models, however, cannot differentiate between these biases and lack the ability to adapt or mitigate them as humans can.

From Bias to Injustice

  • Algorithmic Bias and Injustice Connection:
    • Algorithmic bias is defined as bias embedded within AI algorithms, while algorithmic injustice refers to the negative consequences arising from these algorithms.
    • Existence of Bias: The mere presence of bias does not inherently lead to discrimination or injustice.
    • Impact of Biased AI Models: The biased nature of an AI model used for decision-making can lead to unfair outcomes depending on specific biases present in the model.

Addressing Algorithmic Bias

  • Preventive Measures:
    • Identifying and alleviating algorithmic bias can help to prevent associated injustices later on.
    • Inquiry: How does algorithmic bias operate?

Proxy Attributes in AI

  • Identifying Patterns:
    • AI models are trained to discern patterns and correlations within data, each of which is coded as an ‘attribute’ or ‘feature’ in the model.
    • Real vs. Proxy Attributes: Attributes may or may not correspond to recognizable human attributes, depending on the structure of the dataset used for training.

The Function of Proxy Attributes

  • Predictive Inferences:
    • After learning ‘attributes’, AI employs them to make predictive inferences about incoming data in alignment with user queries.
    • Role of Proxy Attributes: These attributes serve as proxies guiding the AI model's predictions.

Contextual Inappropriateness of AI Outputs

  • Source of Fairness Problems:
    • The connection between proxy attributes and model outputs is where algorithmic bias and fairness dilemmas emerge.
    • Contextual Mismatch: If a model makes predictions based on proxy attributes that do not accurately reflect the relevant context, the outputs produced will be contextually unsuitable.
    • Moral Implications: Ethical concerns arise when proxy attributes stem from harmful stereotypes or unjust social systems.
    • Example Provided: The case of COMPAS, its reliance on zip codes related to race highlights these concerns.

A Catch-22 in AI Development

  • Need for Human-like Thinking:
    • Effective AI must think and reason similarly to humans.
    • Inheriting Biases: While we desire AI to adopt human-like thinking, it also inherits human biases, including negative ones.
    • Paradox: In attempting to resolve one issue, we may inadvertently create another.

Future Considerations in AI Decisions

  • Historical Data Limitations:
    • Datasets used to train AI reflect the world as it existed during data collection.
    • Reinforcement of Bias: When AI guides future decisions based on past learnings, it perpetuates existing societal problems.
    • Flow of Bias: Biased past society → biased datasets → biased AI systems → biased future society.

Illustrated Example: Women in Philosophy

  • Underrepresentation Issue:
    • Women have been historically underrepresented in the philosophy field.
    • Hiring Algorithms: AI systems trained on historical philosophy job data may use metrics like the number of publications to filter applications.
    • Historical Bias Cycle: Fewer women results in fewer publications, which further limits hiring opportunities, creating a cyclical problem.

Path Forward – Addressing Algorithmic Bias

  • Need for Solutions:
    • Now that algorithmic bias and its consequences are better understood, attention shifts to potential solutions.
    • Non-technical Remedies:
    • Consensus suggests that solutions are not solely technical.
    • AI exists within a complex socio-technical system, necessitating comprehensive approaches.

Critical Perspective on Solutions

  • Caution against Automation:
    • "Automated and standardized solutions to complex and contingent social issues often contribute more harm than good—they often fail to grasp complex problems and provide a false sense of solution and safety."
    • Historical, Political, and Moral Awareness:
    • Complex social challenges require an understanding of their historical, political, and moral context, along with structural changes.
    • (Citation: Abeba Birhane, “Algorithmic injustice: a relational ethics approach”, p. 1)

Lessons Learned from Analysis

  • Garbage In, Garbage Out:
    • If flawed data is inputted into AI systems, it produces flawed outputs.
  • Impact on Vulnerable Populations:
    • Predictive algorithms do not function uniformly for all; marginalized groups face heightened risk of adverse effects due to underrepresentation in training datasets.
  • Fairness Requires Fair Context:
    • For AI to become fairer and more equitable, societal contexts must first become fairer and more just.