Lecture 22: Interactive Machine learning bias mitigation

Q: What is Interactive Machine Learning (IML)?

An iterative process where human users periodically review and provide feedback to ML models.

Q: Why include human interaction in ML?

To use human intuition and experience for faster, more efficient, and trusted models.

Q: What types of problems especially benefit from human interaction?

Problems related to visual perception or hard-to-quantify decisions.

Q: How can human interaction help with bias?

By identifying sensitive and non-sensitive variables related to biased outcomes.

Q: What is one social benefit of democratizing machine learning?

Ensuring the model promotes the common good through community sentiment.

Q: What is a major limitation of current ML algorithms regarding bias?

They often act as black boxes without incorporating human feedback.

Q: What was the objective of the study in this deck?

Develop a visualization-supported interactive ML platform for bias mitigation.

Q: What type of feedback loop was demonstrated?

Iterative mutual feedback between the ML model and a single human user.

Q: What dataset was used for the experiment?

Home Credit Default Risk dataset.

Q: What was the protected (sensitive) attribute in the experiment?

Client gender.

Q: What did a 6% difference in prediction accuracy between groups indicate?

Presence of bias in the classification model.

Q: How was bias mitigation visualized?

Through scatterplots showing misclassified test samples for each group.

Q: What method did users apply to reduce bias?

Adding new data samples in regions of disparity between groups.

Q: What was the result of interactive data selection?

The bias gap reduced quickly compared to random sample addition.

Q: After two iterations, what was observed in the male group?

Improved prediction accuracy.

Q: What could not achieve the same bias reduction effect?

Randomly adding samples.

Q: What is the main conclusion about interactive ML for bias mitigation?

Visualization and iterative human feedback can effectively reduce bias.

Q: What is a future direction suggested by the study?

Expand user modalities and include community-level interactions