Historical data may contain patterns of discrimination because it reflects decisions and social conditions from the past. When a machine learning model learns from such data, it may reproduce or amplify existing inequalities. Representation bias occurs when some populations are insufficiently represented in training data. Measurement bias occurs when collected features or labels do not accurately represent the underlying concept. Label bias can arise when human judgments are inconsistent or influenced by stereotypes. Evaluation bias occurs when testing methods do not reflect the conditions in which a model will be used. Fairness is complex because different mathematical fairness criteria can conflict, particularly when groups have different base rates. Therefore, selecting an appropriate fairness approach requires understanding the application, legal requirements, social context, and possible consequences. Mitigation methods include improving data representation, reviewing labels, modifying training procedures, evaluating subgroup performance, and monitoring outcomes after deployment.
Bias and Fairness in Machine Learning
Bias and fairness in machine learning concern whether AI systems produce systematically unequal or unjust outcomes for different individuals and groups.
Statistical parity difference = P(Ŷ = 1 | A = 1) - P(Ŷ = 1 | A = 0)
Explanation
Variables
| Symbol | Meaning |
|---|---|
Ŷ | The model's predicted outcome. |
A | A group or protected attribute used for fairness analysis. |
P(Ŷ = 1 | A = 1) | Positive prediction rate for group 1. |
P(Ŷ = 1 | A = 0) | Positive prediction rate for group 0. |
Example or application
An AI system evaluates job applications. If qualified applicants from one demographic group consistently receive fewer positive recommendations, the organization should investigate training data, labels, features, and evaluation results to identify possible sources of unfairness.
Text
Machine learning bias occurs when data, model design, measurement, or deployment conditions systematically influence predictions in ways that may disadvantage certain groups. Fairness is the goal of evaluating and reducing unjust differences in model outcomes. Bias can enter through historical records, unrepresentative datasets, incorrect labels, feature selection, and human decisions. Fairness evaluation examines model performance and decision outcomes across relevant groups rather than relying only on overall accuracy.
Key Points
Bias can originate in data, labels, model design, and deployment.
Historical data may reproduce existing inequalities.
Overall accuracy can hide poor performance for particular groups.
Fairness has multiple definitions and measurement criteria.
Different fairness criteria may conflict.
Fairness should be evaluated throughout the model lifecycle.
Applications
Fair recruitment and employee selection
Responsible credit-risk assessment
Healthcare prediction systems
Educational admission and assessment tools
Public-service decision systems
Learning Outcomes
Define bias and fairness in machine learning.
Identify common sources of algorithmic bias.
Understand the purpose of statistical parity.
Explain why subgroup evaluation is important.
Describe common bias-mitigation approaches.
Practice Questions
What is algorithmic bias?
How can historical training data introduce unfairness?
What does statistical parity measure?
Why can overall accuracy hide unfair model behavior?
Related Concepts
AI Ethics
Training Data Quality
Model Evaluation
Algorithmic Accountability
Fairness Metrics
Common Mistakes
Assuming that removing sensitive attributes automatically removes bias.
Treating equal prediction rates as proof of complete fairness.
Evaluating only overall model accuracy.
Ignoring differences in real-world deployment conditions.
Sources and verification
Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.
- Barocas, S., Hardt, M., and Narayanan, A., Fairness and Machine Learning, 2023.
- Mehrabi et al., A Survey on Bias and Fairness in Machine Learning, ACM Computing Surveys, 2021.
- NIST, Towards a Standard for Identifying and Managing Bias in Artificial Intelligence, 2022.
Suggest a correction or source · More in Artificial Intelligence & Machine Learning