Week 11 - Data bias and ethics

What is bias?

Bias is a technical property of models that can often be measured and mitigated.

In ML this refers to systematic errors leading to consistent distortions in the output of an ML model. These biases can lead to unfair or discriminatory outcomes.

What are ethics?

Ethics is about value-driven decision-making, encompassing whether and how ML systems should be used, including (but not limited to) bias considerations.

What are the types of bias in ML?

Sampling bias - sampling in a non-uniform way.

Exlusion bias - exlcuding certain features during data cleaning.

Technical bias - what model, regularisation, hyperparameters, etc. are used impacts model output.

Real-world bias - high-quality data does not exist for all groups and events equally.

Contextual bias - misusing the model by deploying it in a way not intended during training.

How can we sample in an unbiased way?

Random uniform sampling (each data point has an equal chance of being selected).

Stratified sampling (within each group each data point has an equal chance of being selected)

How can we try to prevent bias?

Have an outer feedback loop to ensure the model is compliant with stakeholder ethical concerns.

Why is interpretability important?

In safety-critical domains we need to know why the ML model made the decisions it made. NN is not good for this, Decision trees are more interpretable.