Algorithmic Bias, Fairness, and Ethical AI Governance in Criminal Justice and Healthcare

What is the purpose of the COMPAS algorithm?

To predict a criminal defendant's risk of committing additional crimes using a 137-item questionnaire.

What did the 2016 ProPublica study find regarding COMPAS?

Black defendants were more likely to be incorrectly flagged as high risk, while white defendants were more likely to be incorrectly flagged as low risk.

What is the core concept of 'value laden technology'?

Technological systems are not neutral; they inherently reflect the moral, social, political, and economic values of their creators.

What is bias in the context of algorithms?

A cognitive shortcut where negative assumptions are made about a group or person based on preconceived ideas.

Define historical bias in algorithms.

Bias that occurs when training data reflects past societal discrimination.

What is representation bias?

When certain groups are unfairly or insufficiently represented in the training data.

What is measurement bias?

When data acts as a proxy for a protected class, such as using zip codes as a proxy for race.

Define aggregation bias.

When aggregated data fails to account for the needs or characteristics of non-majority or marginalized groups.

What is disparate treatment discrimination?

Unfair treatment of a protected class member where the conduct is proven to have a discriminatory purpose.

What is disparate impact discrimination?

Facially neutral conduct that results in an unfair or prejudicial impact on a protected class.

What is the 'Business Necessity Defense'?

A legal defense where discriminatory conduct is permissible if it is proven necessary to accomplish a legitimate purpose.

Define fairness in the context of algorithmic systems.

The impartial treatment of individuals without bias to ensure equitable treatment and equal opportunities.

What is demographic fairness?

A model where positive prediction rates are equal across different demographic categories.

What is equalized odds fairness?

A model that maintains equal true and false positive rates across demographic categories.

What is calibration fairness?

When the actual results across demographic categories match the model's predictions.

What does the 'Impossibility Theorem' suggest about fairness?

It is impossible to satisfy multiple definitions of fairness simultaneously unless base rates are identical across categories.

What is the primary function of ShotSpotter?

A gunshot detection system that uses sensors to alert law enforcement to gunfire without relying on 911 calls.

What is a major criticism of ShotSpotter regarding enforcement?

The placement of more sensors in communities of color leads to circular statistical justification and enforcement bias.

What are the three main components of the AI system described in the TechCorp case study?

Resumes, coding test results, and behavioral assessments.

What is the primary goal of AI governance?

To manage the development and deployment of AI systems to ensure they are explainable, accountable, and free from bias.

Why is explainability important for AI content moderation decisions?

It provides transparency to users regarding why their content was removed, allowing for accountability.

What is a protected class?

A group of people protected by law from discrimination, including race, color, religion, gender, and national origin.

What is the main challenge for designers regarding bias?

Reducing bias in systems where 'when bias goes in, bias comes out'.

What is the central ethical dilemma regarding Apple and the FBI?

Whether a company should build a backdoor into secure technology to assist law enforcement investigations.

What are the three core pillars of business responsibility?

Economic, legal, and ethical responsibilities.

What is the primary focus of economic responsibility in business?

Managing risks, finding opportunities, and balancing reputation against new ventures.

Name four common stakeholder groups a corporation must consider.

Customers, employees, shareholders, and public interest groups.

What is the difference between the 'letter of the law' and the 'spirit of the law'?

The letter refers to strict adherence to written rules, while the spirit refers to the underlying intent or purpose of the law.

What is procedural justice in an ethical framework?

The fairness of the decision-making process and the methods used to determine impacts on others.

What is the primary mandate of the Federal Trade Commission (FTC)?

To prohibit unfair or deceptive practices in trade.

What constitutes an 'unfair' practice according to the FTC?

Actions that unreasonably harm consumers in a way they could not reasonably avoid.

What does the Gramm-Leach-Bliley Act require of financial institutions?

To protect nonpublic personal information (NPI) and adopt reasonable cybersecurity measures.

Under the Gramm-Leach-Bliley Act, when can financial institutions share NPI with third parties?

Only after providing notice and obtaining consumer consent.

What is the primary purpose of HIPAA?

To regulate the handling and sharing of Protected Health Information (PHI) by health care providers and plans.

What is the mandatory timeline for notifying patients of a data breach under HIPAA?

Within 60 days.

Does HIPAA provide a private right of action under federal law?

No, though providers may still be liable under state law.

What is the main goal of the Fair Credit Reporting Act?

To protect information regarding a consumer's creditworthiness and ensure its accuracy.

What must a user of credit information do if an adverse action is taken against a consumer?

They must disclose the action to the consumer.

What is the 'Purpose Limitation' principle under GDPR?

Data must be processed only for the legitimate purposes explicitly specified to the data subject at collection.

What is 'Data Minimization' according to GDPR?

Collecting and processing only the data that is absolutely necessary for the specified purpose.

What does the 'Accountability' principle under GDPR require of a data controller?

The ability to demonstrate compliance with all GDPR principles.

What is the 'Storage Limitation' principle in GDPR?

Personally identifying information may only be stored for as long as necessary for the specified purpose.

What are the three stages of policy drafting for a governance framework?

Pre-deployment, deployment, and post-deployment.

Why is technical expertise alone insufficient for effective governance?

Because governance also requires domain-specific knowledge, ethical frameworks, legal expertise, and consideration of affected communities.

What is the relationship between decision stakes and governance?

High-stakes decisions require high-stakes, rigorous governance.

What is the primary focus of the Illinois Biometric Information Privacy Act?

Regulating the collection and storage of biometric data.

What is the 'Integrity and Confidentiality' principle of GDPR?

Processing must ensure appropriate security, integrity, and confidentiality of personal data.

Beyond policy decisions, what two elements are central to governance?

Power and incentives.