How AI Works | Classification Thresholds — Base Rates, Precision, Recall and Review Capacity

A classification threshold is the rule that turns an AI model’s score into an action, such as flag for review, ignore, ask for more information or abstain. Lowering the threshold often catches more true cases but also brings more false alarms. Raising it can reduce review volume while missing people who need help. The correct threshold therefore depends on the real prevalence of the target, the consequences of errors, calibrated evidence about model scores and the capacity of people or services that must handle flagged work. A model can stay unchanged while the best operational decision rule changes.

To choose responsibly, start with a held-out confusion table rather than a dramatic model-demo example. This chapter uses a fictional study-helpdesk classifier across one thousand messages, compares two thresholds, calculates precision and recall with explicit denominators, and introduces a human-review capacity of one hundred and fifty cases. Then we change the underlying prevalence, solve the new arithmetic and show where the original choice breaks. Every count is invented for learning; none is a current provider benchmark, an admission rule or a real-world risk estimate.

This chapter belongs under How AI Works, the general artificial-intelligence mechanism. It complements the limits of benchmarks and task evaluations but owns the narrow decision problem of selecting a classification cutoff under base-rate and operational constraints. Model routing, review queues and agent permissions remain different questions. No claim about present artificial superintelligence is implied.

Choose your reading route

• Learning machine learning? Work through the confusion table and recalculate the two thresholds by hand.
• Running a support team? Concentrate on review capacity, ranked selection and the meaning of an unreviewed alert.
• Planning evaluation or policy? Read base rates, loss assumptions, separate test data and change monitoring.
• Teaching Mathematics? Use the worked fractions, changed-prevalence exercise and explanation of denominators.

Contents

• 1. A probability score is not a decision
• 2. Precision, recall and four outcomes
• 3. Why base rates change the answer
• 4. Review capacity is a real constraint
• 5. A fictional one-thousand-request laboratory
• 6. Two thresholds, four cells each
• 7. Solve the capacity-limited option
• 8. Explicit losses and operational choices
• 9. Evaluate thresholds without leakage
• 10. Diagnose drift, calibration and changing mix
• 11. Transfer exercise and changed conditions
• 12. Limits, responsibility and human impact
• Frequently asked questions
• Authoritative reading
• Learning pathways

1. A probability score is not a decision

A model score is an estimate or ranking quantity produced from inputs. It may be a calibrated probability, an uncalibrated margin or a ranking score. Before comparing it with a fixed cutoff, establish what the score means and how its reliability was tested. A value of 0.8 is not automatically an eighty-percent probability unless the model and prediction interface justify that interpretation. Calling every number between zero and one a probability can make downstream planning appear more certain than the evidence allows.

Prediction and decision are two different layers. A classifier may estimate whether a message contains an urgent tutoring issue. A policy then decides whether to ask a parent for clarification, route the message for prompt human review, or handle it in ordinary order. That policy depends on staff availability, priority, risk and the possibility of an inappropriate intervention. The model should not invent permission to send messages or change a student’s record simply because its risk score is high.

The scikit-learn documentation on tuning the decision threshold explicitly separates learning class probabilities or decision scores from choosing an action. It also warns in its discussion of threshold tuning about using the wrong data for optimisation. This distinction is foundational: model quality and operational choice interact, but a different threshold does not retrain the model or alter the world by itself.

Sometimes the right output is not binary. A system can ask a clearer question, withhold an automatic decision or escalate to a qualified human. A threshold that triggers an unreviewed punitive action is fundamentally different from one that triggers a harmless request for more information. State the action being evaluated before naming a good precision or recall target. Otherwise a model can look impressive while the surrounding process makes damaging choices.

2. Precision, recall and four outcomes

A binary confusion matrix compares the system’s flagged or unflagged prediction against a defined reference outcome. A true positive is a genuinely positive case flagged by the system. A false positive is a negative case incorrectly flagged. A false negative is a positive case missed by the system. A true negative is a negative case correctly left unflagged. Each record belongs in exactly one of these four categories under the given evaluation protocol, or must be explicitly handled as unresolved rather than squeezed into a convenient box.

Precision answers: among the cases we flagged, what fraction were truly positive? Its denominator is true positives plus false positives. Recall answers: among all truly positive cases, what fraction did we flag? Its denominator is true positives plus false negatives. The false-positive rate answers: among all truly negative cases, what fraction did we flag? Its denominator is false positives plus true negatives. Each metric measures a different question, so two percentages with similar numbers need not describe similar risks.

Accuracy uses all correctly classified records divided by the total. It can be deceptively high when almost everyone belongs to the negative class. Suppose a rare urgent condition occurs in only ten of a thousand messages. A system that labels every message ordinary achieves 990 ÷ 1,000 = 99% accuracy while catching none of the ten urgent cases. Precision is undefined if it flags no messages; recall is zero. The system’s apparently excellent accuracy would not serve the actual review objective.

Google’s classification metrics guide explains how priority among precision, recall, false-positive rate and accuracy depends on the application and error costs. That statement is a framework for making a decision, not a universal numerical standard. A tutoring helpdesk, a medical screening process and a spam filter should not blindly use the same score cutoff or the same acceptable error balance.

3. Why base rates change the answer

The base rate or prevalence is the proportion of true positive cases in the population being evaluated. An AI model may have been trained on a deliberately balanced dataset containing half positives and half negatives. Its live users may present only two percent positive cases. Even if the model keeps the same true-positive rate and false-positive rate conditional on each class, the proportion of flagged cases that are genuine can change greatly because most of the live population belongs to the negative class.

This is not a statistical trick. Imagine a detector with an eighty-five-percent true-positive rate and fifteen-percent false-positive rate. Among a thousand cases with one hundred true positives and nine hundred true negatives, the expected flags under those exact rates would be eighty-five genuine plus one hundred and thirty-five false: 85 ÷ 220 ≈ 38.6% precision. If true positives fall to twenty and negatives rise to nine hundred and eighty, the expected flags are seventeen genuine plus one hundred and forty-seven false: 17 ÷ 164 ≈ 10.4% precision. The model’s class-conditional rates are unchanged, while the operational meaning of an alert changes drastically.

The counts in this illustration are chosen so the arithmetic is exact: 85% of 100 is 85, and 15% of 900 is 135; similarly, 85% of 20 is 17 and 15% of 980 is 147. Real observed samples will fluctuate. A precision calculation should not be presented as a guarantee for every future batch. The meaningful habit is to state the base rate, confusion counts, sample size and evaluation period rather than transport one precision figure from a laboratory to a different population.

When base rates are uncertain, plan sensitivity analyses. Show how a fixed score threshold would affect alerts under several plausible prevalence scenarios. Consider who is absent from the sample, whether the label itself is stable and what happens to reviewers if alerts double. Do not fabricate a universal optimum. A system with lower measured precision might still be appropriate for non-invasive screening with a well-designed second-stage check, while the same threshold could be inappropriate for automatic sanctions.

4. Review capacity is a real constraint

A flag is not free when someone must read it. Reviewers may need context, consent verification, translation, access to trusted source records and time to communicate. If a threshold produces two hundred and twenty alerts but the team can meaningfully review only one hundred and fifty, the design has seventy unhandled items. Calling the detector successful while leaving those cases in a growing queue confuses mathematical classification with delivered service.

Capacity should be measured in the unit that matters. One reviewer may process an easy informational message quickly but require far longer to investigate a disputed booking change. The mix of tasks determines sustainable throughput. For an elementary workshop, we use equal review cost per flagged message; real systems should estimate service time distributions and safe concurrency, and protect high-consequence cases with explicit priority and age rules. The decision threshold cannot override an independent consent requirement or a legally necessary human determination.

Some teams impose a top-k rule: at most the k highest scored cases are sent to review. This respects a count limit, but it changes the effective threshold from a fixed score to a position in the ranking. Its performance depends on ranking quality in the current batch. The lower-ranked cases do not become negative just because nobody can inspect them. Record the selection rule and evaluate its missed positives, not only the successful outcomes of the items that reached a human.

Also separate review capacity from an authority to act. A tool might flag a probable issue; a reviewer examines evidence; a responsible person approves a change; the source system makes it; and an independent readback confirms the result. The classification result itself is only an input to this chain. That distinction protects users from an otherwise plausible but unauthorised automation that treats a high score as permission.

5. A fictional one-thousand-request laboratory

Our invented Willow Learning Desk wants to identify messages that need priority human attention today. A qualified reviewer has independently established a reference classification for a synthetic set of one thousand messages. One hundred genuinely require priority attention under the fictional contract; nine hundred do not. The positive prevalence is therefore 100 ÷ 1,000 = 10%. The AI model has supplied a score for each message, and the team is testing two possible cutoff rules. No actual student case or real model measurement is involved.

At a high threshold of 0.8, sixty of the one hundred reference positives are flagged, and twenty of the nine hundred reference negatives are incorrectly flagged. That leaves forty false negatives and eight hundred and eighty true negatives. The four categories sum to 60 + 20 + 40 + 880 = 1,000. The high threshold produces eighty total alerts, a load the team can handle within its one-hundred-and-fifty-case daily review limit.

At a lower threshold of 0.5, eighty-five of the reference positives are flagged, but so are one hundred and thirty-five negatives. Fifteen positives are missed and seven hundred and sixty-five negatives are correctly left alone. This also sums to 85 + 135 + 15 + 765 = 1,000. The lower threshold produces two hundred and twenty alerts, exceeding human-review capacity by seventy. It is tempting to call the lower rule better because recall rises, yet leaving unreviewed alerts behind might make the service less reliable.

• High threshold 0.8: 60 true positives, 20 false positives, 40 false negatives, 880 true negatives; 80 alerts.
• Lower threshold 0.5: 85 true positives, 135 false positives, 15 false negatives, 765 true negatives; 220 alerts.
• Fixed reference population: 100 actual positives, 900 actual negatives, 1,000 total messages.
• Review capacity: 150 alerts per day, assuming one comparable review slot per message in this workshop.
• These are wholly fictional threshold outputs, not an empirical claim about any deployed AI system.

The synthetic population was deliberately selected so the threshold change has a clear mechanical effect. Lowering the cutoff moves some records from predicted negative to predicted positive. Fifteen? No: at the high threshold, eighty are flagged; at the low threshold, two hundred and twenty are flagged, so one hundred and forty additional messages cross the decision boundary. Among them twenty-five are truly urgent cases that were previously missed, and one hundred and fifteen are nonurgent cases. Check 25 + 115 = 140. The model has not learned anything new from changing the threshold; the policy has accepted a different exchange of misses and alarms.

6. Two thresholds, four cells each

For the high threshold, precision is 60 ÷ (60 + 20) = 60 ÷ 80 = 75%. Recall is 60 ÷ (60 + 40) = 60 ÷ 100 = 60%. False-positive rate is 20 ÷ (20 + 880) = 20 ÷ 900 ≈ 2.22%. Overall accuracy is (60 + 880) ÷ 1,000 = 94%. This accuracy number is high largely because the nine hundred negatives dominate the evaluation population. It must not be presented alone as proof that the most important people will receive timely attention.

For the lower threshold, precision is 85 ÷ (85 + 135) = 85 ÷ 220 ≈ 38.64%. Recall is 85 ÷ (85 + 15) = 85%. False-positive rate is 135 ÷ (135 + 765) = 135 ÷ 900 = 15%. Accuracy is (85 + 765) ÷ 1,000 = 85%. Notice that lowering the threshold increases recall by twenty-five percentage points but decreases measured accuracy by nine points. Whether the trade-off is worthwhile depends on which errors matter and what the review process can safely complete.

Do not round a denominator before dividing. The 220 alerted low-threshold cases contain 85 true positives and 135 false positives. Dividing 85 by the total thousand would yield 8.5%, which is the portion of the whole population that was both positive and flagged, not precision. Likewise, dividing 135 by 220 would give approximately 61.36%, which is the fraction of alerts that were false, not the false-positive rate among genuine negatives. A useful audit labels every numerator and denominator in words.

The confusion table also helps explain user experience. At the high threshold, forty genuinely urgent messages remain unflagged. At the lower threshold, fifteen remain unflagged, but a surge of false alerts competes for attention. If the team cannot finish its queue, a correctly flagged message may still be served late. The evaluation therefore needs both statistical classification measures and operational measures such as time to review, escalation quality, age of pending cases and evidence of final resolution.

7. Solve the capacity-limited option

Suppose the team chooses the lower 0.5 threshold but can inspect only the 150 highest-scoring records within the set of 220 flagged messages. Our teaching dataset further specifies that the top 150 contain 78 genuine positives and 72 negatives; the 70 remaining alerts contain 7 positives and 63 negatives. Check the partitions: 78 + 7 = 85 positives flagged at 0.5, and 72 + 63 = 135 negatives flagged. The capacity rule changes the final reviewed set; it does not magically change the reference labels.

The capacity-limited pipeline reviews 150 messages. Its reviewed-positive yield is 78 ÷ 150 = 52%. Relative to all one hundred genuinely urgent messages in the day, the recall of actual reviewed cases is 78 ÷ 100 = 78%. The twenty-two unreviewed positives consist of seven in the flagged-but-not-reviewed group plus fifteen below the original 0.5 cutoff. At the level of delivered human review, the matrix becomes 78 true positives reviewed, 72 false positives reviewed, 22 true positives not reviewed and 828 true negatives not reviewed, totaling one thousand.

The new figure of 52% is not the same metric as precision of all predicted positive messages under the 0.5 threshold. It describes the subset selected for actual review. This is why reports should name their stage: model flagged, selected by ranking, accepted by queue, read by a person, and resolved according to policy. A system could report excellent alert recall while failing its service objective because it never processes the last seventy cases. Human capacity and queue discipline are not optional details.

Does that make a top-150 policy optimal? Not necessarily. It depends on how much score ranking separates genuine priorities among flagged records, whether some classes require legally mandated review, whether late arrival creates unfair delay and whether the team can redistribute staff. A score cap should not override safety-critical routing or ignore users whose evidence was insufficient. We use this fictional ranking arrangement to demonstrate calculation, not to certify that top-k is the best deployment policy for a real education service.

8. Explicit losses and operational choices

One way to expose assumptions is to assign a common fictional cost unit. Suppose every genuinely urgent case not reviewed incurs twelve decision-loss points, while every completed human review consumes two workload points. These are not dollar costs or measured harms; they are deliberately constructed units for comparing scenarios. We are also deliberately ignoring differences in case severity and review duration. If the stakeholders reject these assumptions, the correct response is to change the model of cost, not to defend the numerical answer as a universal optimum.

At the high threshold, the system reviews eighty messages and misses forty positives. Its assumed loss is (80 × 2) + (40 × 12) = 160 + 480 = 640 points. At the low threshold without a capacity limit, it would review 220 messages and miss fifteen positives, giving (220 × 2) + (15 × 12) = 440 + 180 = 620 points. The lower-threshold policy appears better under this simplified loss function but violates the 150-case review capacity. Comparing it as though staffing were unlimited would present an impossible operating plan.

For the capacity-limited lower threshold, 150 reviews and 22 missed urgent cases yield (150 × 2) + (22 × 12) = 300 + 264 = 564 loss points. Under the exact fictional assumptions and ranking, this is lower than the high-threshold 640-point option while respecting capacity. The difference is 640 − 564 = 76 points. It would be wrong to generalise that a 0.5 threshold is always superior. Changing reviewer costs, missed-case severity, model ranking or the number of messages can reverse the choice.

Consider why the model’s numbers may be insufficient for a real decision. A false negative involving an immediate safeguarding concern can have qualitatively different consequences from a delayed reply to a routine timetable request. A fair policy may require certain categories to bypass ordinary ranking, subject to independent evidence and human responsibility. Introducing those exceptions changes the task and can require additional data rather than just an adjusted threshold. A single aggregate utility score should never conceal differences among affected people.

If operational capacity is the main constraint, organisations can also reduce avoidable alerts by improving data quality, clarifying the positive label, introducing a cheap validated first-stage rule, inviting user clarification or funding more qualified reviewers. Some changes affect the model, others affect the workflow, and each needs its own test. Raising a threshold to hide a backlog may make an alert chart look smaller without fixing the underlying service. Report both demand and completed work.

9. Evaluate thresholds without leakage

Choose thresholds using validation data that represents the intended deployment conditions, while preserving a separate test set for a final check. If you repeatedly tune the threshold against the final test set, you begin fitting decisions to that sample even though the model parameters remain unchanged. The resulting estimate can be optimistic. Google’s guidance on dividing datasets explains why training, validation and test sets have separate roles and why repeated use can wear out evaluation data.

Check that the underlying reference labels are themselves credible. A precision calculation against inconsistent annotations can produce a clean decimal while measuring the wrong target. The AI training-label-quality chapter illustrates this distinct data problem. Here, by assumption, the thousand synthetic reference classifications are established under an agreed task contract. Real systems should record adjudication, uncertainty, exclusion and label changes rather than treating their test CSV as unquestionable truth.

Use confidence intervals or uncertainty descriptions appropriate to sample size and dependence. One thousand messages may sound large but the positive class here has only one hundred examples. A team with ten positive observations cannot credibly infer a very precise rare-event recall for every future week. Reporting false-positive and false-negative counts, prevalence, time period and affected subgroups helps readers understand what the metric can and cannot support. Do not confuse a chosen threshold with a causal guarantee that the organisation will respond correctly.

Separate calibration testing from threshold optimisation. A model whose 0.8 scores correspond to far fewer than eighty percent actual positives in comparable groups may be poorly calibrated, even if its rank ordering is useful. A thresholding policy may still make a useful selection from uncalibrated scores, but it should not call those scores trustworthy probabilities. Assess calibration on held-out data and in important subgroups, and revise documentation or the model if the claimed probability semantics are unsupported.

10. Diagnose drift, calibration and changing mix

Failure A: alert precision drops sharply after a marketing campaign. The first question is whether prevalence fell, the model degraded or the annotation definition changed. Count positive reference outcomes within the new sample, compare class-conditional true-positive and false-positive rates where labels are available, and check whether score distributions shifted. A falling precision figure alone cannot identify the root cause. Repair the relevant layer and retest against the changed setting.

Failure B: the alert queue stays full, but the dashboard shows high recall. Inspect whether the metric counts flagged requests instead of completed reviews, and whether older cases are starving. Reconcile flagged, admitted, reviewed and resolved counts under stable request IDs. Address throughput or triage constraints and provide separate, honest status to people whose cases remain pending. A high-recall classifier is not a completed service.

Failure C: threshold tuning looks excellent on a reused validation set and then fails on new data. Examine repeated selection against the same examples, duplicated records, shifting population and unrecorded manual exceptions. Protect a fresh test set, version decision rules and predefine evaluation metrics. Compare matched time periods and populations, rather than attributing every result to the latest model version.

Failure D: two demographic or language groups have the same average score but very different false-negative consequences. Aggregate precision cannot establish fairness or acceptability. Report group-level evidence with privacy and statistical caution, consider the base rates and costs within each group, and seek qualified review of impact. Different rates do not automatically imply a single causal explanation or a mechanically obvious threshold; fairness choices are governance decisions requiring affected perspectives.

Failure E: an operator interprets a score of 0.65 as “a sixty-five-percent chance that this exact decision will be correct.” Without calibration, that statement is unsupported. Replace probability language with a truthful model-score explanation, test calibration under the relevant conditions and keep the human review criteria separate. Do not let confidence-looking decimals grant the model business authority.

11. Transfer exercise and changed conditions

Transfer to a fictional community-library classifier that flags overdue borrowing notices needing a human explanation. The library receives five hundred cases: fifty reference positives and four hundred and fifty negatives. A threshold flags forty positives correctly and ninety negatives incorrectly. Ten positives are missed; three hundred and sixty negatives are correctly left alone. Calculate alert volume, precision, recall, false-positive rate and accuracy before thinking about a response plan.

Worked answer: alerts are 40 + 90 = 130. Precision is 40 ÷ 130 ≈ 30.77%; recall is 40 ÷ 50 = 80%; false-positive rate is 90 ÷ 450 = 20%; accuracy is (40 + 360) ÷ 500 = 80%. The model’s accuracy and recall happen to have the same numerical value in this example, but the denominators and meanings differ. If reviewers can handle only one hundred cases per day, thirty alerts cannot be examined immediately; the system needs an explicit ranking, holding and status policy.

Change the library’s setting: a new reminder system reduces actual urgent cases from fifty to ten, while the classifier keeps its eighty-percent recall and twenty-percent false-positive rate under that new population. With five hundred cases, expected true positives are eight, true negatives number four hundred and ninety, and false positives are 0.20 × 490 = 98. There are 8 + 98 = 106 alerts; precision is 8 ÷ 106 ≈ 7.55%. Recall remains 80%. The base-rate change makes many more of the alerts false, even though the model’s class-conditional rates remain unchanged.

Now suppose the service changes from a human invitation to a system that automatically suspends library access based on a flag. The original threshold analysis is not sufficient. The consequence, authority and fairness requirements have changed, and human due process may be necessary. The useful transfer is to name the decision and its harms before selecting the metric. A sensible nonpunitive review cutoff cannot be carried unchanged into a punitive workflow simply because the score numbers are the same.

Independent exercise: alter the earlier one-thousand-message problem so reviewers can manage two hundred messages rather than one hundred and fifty. Which cases would be added if the seventy lowest-ranked flagged messages contain seven genuine priorities and sixty-three negatives? Without their exact rank order, the answer cannot be known for the next fifty slots. This is the boundary of the provided evidence. State what additional ranked-label information is needed rather than invent a precise expected gain.

12. Limits, responsibility and human impact

A decision threshold is not a universal truth about people. It is a policy applied to predictions under assumptions about labels, populations, error costs, reviewer skill and available alternatives. Calibrated probabilities can help decision analysis but may lose calibration after deployment shifts. Classifier output may be inappropriate for some high-consequence decisions without independent evidence, contestability and oversight. A model’s statistical score should never be confused with legal permission, fairness or personal worth.

The simple arithmetic here assumes mutually exclusive binary reference labels and a fixed evaluation population. In practice, cases may be multi-label, unresolved, delayed or censored; the result may depend on time; and the reviewer’s response may change the outcome being measured. Report these limits explicitly. For future studies, consult the NIST AI Risk Management Framework for the broader governance, mapping, measurement and management duties rather than treating threshold optimisation as the whole AI risk process.

A useful AI workflow strengthens rather than hides human judgment. Students can learn about conditional probability through the denominators; engineers can test thresholds on reliable data; managers can plan capacity without disguising queues; and affected people can receive clear explanations about whether an automatic score, a human decision or a verified action produced the outcome. This chapter concerns existing AI classification systems. Claims about hypothetical superintelligence belong to a different evidence boundary.

Frequently asked questions

Does a score of 0.8 always mean an 80% chance?

No. Some scores are margins or rankings. Even a nominal probability estimate must be calibrated and checked for the relevant population and period before it is interpreted literally.

Does lowering the threshold always improve recall?

Under a fixed score ranking and reference set, a lower cutoff can flag more of the positive cases or leave recall unchanged. It also commonly adds false positives. Whether it improves completed service depends on review capacity and actual downstream actions.

Why can precision fall when prevalence falls?

The proportion of true positive cases among the population shrinks, while the number of negatives that can be falsely flagged remains large. With similar class-conditional error rates, more alerts can be false even though the detector has not changed.

Can a top-k cap replace a threshold?

It can limit the number selected for review, but it becomes a ranking policy with its own omissions. Evaluate the actual reviewed subset and retain a separate plan for cases not selected, especially where delayed or mandatory review matters.

Is overall accuracy the best metric?

Not necessarily. A system can be highly accurate by always predicting the common negative class while missing every rare but important positive case. Choose metrics and process outcomes according to the real task.

Can a threshold approve an external action?

A threshold can trigger consideration, but it does not create user consent, business authority or proof that the requested effect occurred. Application-level permissions, human review and authoritative readback must remain separate where needed.

Authoritative reading

• scikit-learn: Tuning the decision threshold for class prediction, official documentation.
• Google ML Crash Course: Accuracy, precision and recall, official learning reference.
• Google ML Crash Course: Dividing datasets, on validation and test separation.
• NIST AI RMF 1.0, on AI risk assessment and governance.
• Google categorical-data issues, on quality and human labels.

Learning pathways

Begin with How AI Works for the full model-and-data chain, then study training-label quality to ensure the reference target is credible. For operational queues and equitable handling, read agent queue fairness and review queues. For broad learning and reasoning transfer, explore How Intelligence Works, the How X Works hub and Gold Standard of Education. A number helps only when you know what it divides.