Fairness

What is demographic parity

A classifier

𝑀

M satisfies demographic parity if its output (

𝑌

^

Y

^

) is independent of the sensitive attribute

𝐴

A.

A classifier M satisfies the demographic parity if the prediction or output (Y^) is independent of A (sensitive attribute)

What is accuracy parity ?

A classifier

𝑀

M satisfies accuracy parity if the accuracy of predictions (

𝑌

^

=

𝑌

Y

^

=Y) is the same across all groups.

This means that the correctness of the predictionsshoule be the same for all sensitive groups.

What is the goal of accuracy parity

The goal here is to ensure that error rates are equalized across different groups.

What is true positive parity

True positive parity ensures that the rate of true positives is equal across groups. This means that for individuals who actually have a positive label (e.g., those who will reoffend), the probability of the classifier predicting this correctly should be the same regardless of their sensitive attribute

What is true positive parity also known as

Equal opportunity

What is the false positive parity

• A classifier satisfies false positive parity if the rate of false positives is the same across all groups.

A false positive occurs when the classifier predicts a positive outcome, but the true outcome is negative (e.g., predicting someone will reoffend when they will not).

True Positive Parity and False Positive Parity together is known as

Equalized odds

What is Precision Parity

Precision parity ensures that the likelihood of a positive label being correct is the same across groups.

What is Precision Parity also known as

Positive Predictive Parity

What is Negative Predictive Value

Negative predictive value is similar to precision parity but focuses on negative predictions.

What is the impossibility theorem

Researchers such as Chouldechova (2017) and Kleinberg, Mullainathan, and Raghavan (2017) have demonstrated that it is impossible to satisfy multiple fairness definitions simultaneously in many real-world situations.

Specifically, you cannot achieve False Positive Rate (FPR) parity, False Negative Rate (FNR) parity, and Predictive Parity all at the same time unless:

• The classifier is perfect (i.e., it always predicts correctly).

• The base rates (e.g., actual recidivism rates) for the groups are equal.

To achieve fairness in a machine learning model, we must decide which

type of fairness to prioritize, depending on the context and consequences.

The base rates (recidivism rates) are different for Black and White defendants, ensuring predictive parity will lead to

unequal false positive rates (FPR) and false negative rates (FNR).

What is statistical parity

Statistical parity (or group fairness) means that the demographics of the positive classifications (e.g., those predicted to receive a certain benefit or outcome) are identical to the demographics of the whole population.

True or False: Statistical parity doesn’t ensure individual fairness because it focuses on group-level metrics.

TRUE: Statistical parity doesn’t ensure individual fairness because it focuses on group-level metrics. It is possible to achieve fairness across groups while still discriminating against specific individuals within those groups.

What is fairness through awareness

The idea is that fairness can be achieved by defining a similarity metric that determines how "similar" two individuals are, based on the task at hand.

What is the similarity metric

• Fairness through awareness relies on having a meaningful distance metric that can determine how close two individuals are to each other in terms of their characteristics.

• This metric is critical because it helps determine how the classifier should behave for individuals who are similarly situated.

What is the classifier definition in terms of fairness through awareness

• The classifier is a randomized mapping from individuals to distributions over outcomes.

• This means that even if two individuals are similar, the classifier might give slightly different outcomes, but those outcomes should still be distributed in a way that ensures fairness.

What is the formal representation

For any two individuals, iii and jjj, in the dataset, if their characteristics are similar according to the distance metric, the probability of receiving the same outcome should be high.