THE CORE AIM OF VOCABULARY MASTERY · AI MODEL BIAS · DATA → MEASUREMENT → MODEL → OUTCOMES → CORRECTION
Did you know an artificial intelligence model can achieve a good average score and still perform badly for particular kinds of inputs or users? AI model bias vocabulary helps us ask why. Terms such as sampling bias, representation bias, label bias, measurement bias, subgroup performance, fairness metric, selection bias and mitigation give us a more precise language than simply calling a system “biased.”
The core aim of vocabulary mastery for AI model bias is to identify systematic distortion, distinguish its possible causes and investigate its effects with evidence. Readers should finish knowing how to check where data came from, who or what is underrepresented, how labels were created, which errors are unevenly distributed and what a responsible correction could actually change.
This guide belongs to the eduKateSG Vocabulary Hub. It sits alongside AI Safety Vocabulary, Data Quality Vocabulary and Responsible AI Vocabulary. Each asks a connected but different question.
The essential idea: Bias is not established merely by disliking a result. The investigation starts by defining the system, its affected population, a meaningful comparison and the mechanism that could produce systematic error or harm.
The 60-second AI model bias vocabulary router
- Collect: sample, population, selection, representation.
- Measure: proxy, label, annotation, measurement error.
- Train: objective, feature, model assumption, historical pattern.
- Evaluate: subgroup, false-positive rate, false-negative rate, disparity.
- Interpret: statistical bias, fairness, context, impact.
- Improve: mitigation, reweighting, review, monitoring.
Ten important AI model bias terms, explained
| Term | Practical meaning | Question worth asking |
|---|---|---|
| Sampling bias | The observations collected differ systematically from the population the model is intended to serve. | Who was missed during collection? |
| Selection bias | The way examples enter the dataset creates systematic differences. | Why were these cases included? |
| Representation bias | Relevant groups, situations or phenomena are insufficiently reflected in the data or representation. | Whose circumstances are poorly covered? |
| Measurement bias | An instrument or procedure systematically mismeasures what it is supposed to capture. | What is the recorded variable really measuring? |
| Label bias | Labels reflect inconsistent practices, assumptions or historical decisions rather than a reliable target. | Who assigned labels and under what instructions? |
| Proxy variable | An observable feature stands in for another concept that is difficult to measure directly. | Is the proxy a trustworthy substitute? |
| Subgroup evaluation | Performance is assessed separately for meaningful groups or conditions. | Do overall averages hide failures? |
| False-positive rate | The share of actual negative cases incorrectly classified as positive. | Who faces unnecessary alerts? |
| False-negative rate | The share of actual positive cases incorrectly classified as negative. | Who misses an important benefit or warning? |
| Mitigation | An action to reduce an identified source of unfairness or error. | Did it improve outcomes without causing new problems? |
A worked example: speech recognition across accents
Imagine a voice-to-text application tested on thousands of recordings. Its published average word-error rate looks encouraging. But most evaluation recordings use one familiar accent, while students speaking other varieties of English experience far more transcription errors. The overall score is not necessarily false. It is incomplete for the intended population.
Here the useful vocabulary is sampling, representation, subgroup error rate and distribution shift. A careful audit asks how the recordings were gathered, whether microphones and recording environments differ, how ground-truth transcripts were reviewed and whether errors are concentrated in particular conditions. It does not leap straight from a disparity to a single unproven cause.
Bias, error and unfairness are related—not interchangeable
Statistical bias
In statistics, bias can describe systematic deviation of an estimator from the quantity it seeks to estimate. That usage is not identical to social bias or unfair treatment. A model may have statistical estimation bias without a particular social fairness concern, and a system can raise serious fairness concerns without that statistical definition being the central issue.
Performance disparity
A performance disparity means a measurement such as accuracy or false-negative rate differs across meaningful groups. It is a signal to investigate. The importance of a disparity depends on context, sample sizes, consequences and the validity of the groups and labels being compared.
Harmful bias
A model can amplify disadvantage through data, objectives, institutional processes or deployment decisions. The effects matter: whose applications get reviewed, who receives additional scrutiny and who has a meaningful route to challenge a result? These are social and operational questions as well as technical ones.
Why historical labels can be misleading
Suppose a model predicts “successful applicants” using records from past decisions. If past decisions were affected by inconsistent screening or unequal access to opportunities, the historical labels may capture those practices. The model can learn to reproduce the pattern even when the dataset has no explicit field labelled “unfairness.”
This is why target, label and ground truth must be treated carefully. Ground truth is the reference used in evaluation, but a label created by people or imperfect instruments is not necessarily infallible.
Overall accuracy can hide the answer parents and students need
Consider two groups in an educational support system. If most cases are easy, overall accuracy may remain high even when the small group who needs intervention receives poor predictions. Teachers should not let an automated score substitute for direct diagnosis of a student’s learning needs. The score may be a starting signal, never the whole learner.
Fairness metrics: which question is being asked?
- Demographic parity: compares positive decision rates between defined groups; it does not guarantee equal underlying need or equal error rates.
- Equal opportunity: commonly focuses on equal true-positive rates among groups.
- Equalized odds: commonly considers both true-positive and false-positive rates.
- Calibration: examines whether predicted probabilities align with observed frequencies, sometimes considered by subgroup.
- Individual considerations: ask whether similarly situated cases receive appropriately consistent treatment.
Different fairness criteria can conflict. No single statistic can settle all legal, ethical or contextual questions. Teams should justify the criteria they select, document trade-offs and involve people who understand the real-world decision.
A practical six-step bias audit for beginners
- Define the intended use, target population and decision being supported.
- Inspect how data and labels were collected, including missing or inconsistent cases.
- Choose relevant subgroups and ensure there is enough data for meaningful comparison.
- Measure both average performance and the error types that matter for each subgroup.
- Investigate possible sources of disparity before proposing a correction.
- Evaluate the effects of the mitigation, record trade-offs and monitor after release.
What mitigation can and cannot do
A mitigation might improve sampling, repair labels, adjust training weights, revise decision thresholds or add human review. But mitigation is not a magic word for “bias solved.” Reweighting data can change one metric while worsening another. A fairer statistical pattern does not automatically fix an unfair institutional decision process.
Common mistakes and repairs
- “The model is biased because I disagree with its answer.” Repair: define a systematic pattern and gather evidence.
- “A balanced dataset guarantees fairness.” Repair: examine labels, features, objectives and deployment.
- “One overall metric tells the full story.” Repair: use disaggregated evaluation where appropriate.
- “Correlation identifies the cause.” Repair: investigate measurement, selection and contextual explanations.
- “A mitigation ended the problem.” Repair: retest trade-offs and monitor performance over time.
Frequently asked questions
What is AI model bias?
It refers broadly to systematic distortions in AI data, measurements, models or decisions; the precise type of bias should be named before conclusions are drawn.
Can AI be biased without anyone intending discrimination?
Yes. Systematic problems can arise from sampling, measurement, labels and institutional processes without deliberate prejudice.
Is a difference between groups always evidence of discrimination?
No. It can be an important signal, but interpretation requires context, relevant comparisons and investigation of causes and consequences.
How do you test for AI bias?
Document the intended use, audit data and labels, examine relevant group-level outcomes, evaluate error types and investigate identified disparities.
What is the difference between data bias and model bias?
Data bias concerns systematic problems in the information used; model behaviour can reflect those problems alongside modelling choices and how the system is deployed.
Useful references and connected reading
The NIST AI Risk Management Framework identifies trustworthiness characteristics including fairness with harmful biases managed. It also distinguishes systemic, computational and statistical, and human-cognitive biases. For further learning, explore the site’s AI Governance Vocabulary.
The AI model bias vocabulary standard
The goal is reached when a learner can describe the population, the dataset, the metric, the observed disparity, plausible causes and appropriate corrective checks. Clear terminology makes the discussion more thoughtful, more measurable and more useful to the people affected.