AI Ethics: The Bias Audit Lab

This set of vocabulary flashcards covers the key concepts of AI ethics, focusing on different types of bias and the framework for auditing AI-generated content based on the Bias Audit Lab lecture.

Representational Bias

A bias that occurs when an AI model makes certain groups of people "invisible" or only shows them in limited, stereotypical ways, reflecting a narrow view of who exists in certain roles.

Linguistic and Cultural Bias

A bias that occurs when an AI assumes Western, American, or English-speaking norms are the "universal" standard, treating one culture as the default and others as alternative or wrong.

Historical Bias

Often called "The Mirror Problem," it occurs when an AI is trained on data from an unfair past and mathematically repeats those prejudices in its predictions.

Blast Radius

The real-world stakes and consequences of AI bias, such as hiring algorithms favoring "traditionally male" hobbies or healthcare AI prioritizing patients based on historical spending.

Opportunity Risk

A concept stating that AI bias is not just a simple mistake or typo, but a significant risk to real-world opportunities.

The "Big 3" Bias Upgrades

A collective term for Representational Bias, Linguistic & Cultural Bias, and Historical Bias.

Original Failure Modes

The four predictable failure modes in the Decision Framework: Overconfidence, Hallucination, Outdated, and Goal Misalignment.

The Mirror Problem

A synonym for Historical Bias, referring to how AI repeats the "bad habits" of past training data.

2024 Gemini error

A specific historical instance where over-correcting for bias led to "hallucinated" historical images.

Blind Recall

A learning technique that involves attempting to recall information without notes or an iPad, recommended to be done $3-4x$ with sleep in between.

Representational Bias Example

An AI provides $10$ images of older men with white hair when prompted for a "brilliant scientist."

Linguistic/Cultural Bias Example

An AI defines a "standard healthy breakfast" as eggs, whole-grain toast, and yogurt, ignoring regional standards like congee, soup, or rice.

Historical Bias Example

An AI trained on successful leaders from the last $50$ years learns that being a white male is a requirement for leadership because women and minorities were socially excluded in the past.

Casting Director Station

A lab station in the Red-Team activity focused on auditing for Representational bias.

Local's Audit Station

A lab station in the Red-Team activity focused on auditing for Cultural or Linguistic bias.

Hiring Hall Station

A lab station in the Red-Team activity focused on auditing for Historical bias.

Red-Team Roles

The three specific roles assigned in group setups for the Bias Audit Lab: Reader, Spotter, and Decider.