How does superintelligence work when an AI compares two organisations and announces that the one with more mistakes performed worse? That seems sensible until you discover that one organisation checked nearly three times as many items. Eight mistakes in 3,200 checks are a smaller error rate than six mistakes in 1,200 checks. Comparing totals alone can reverse the conclusion.
The mechanism is exposure-adjusted rate reasoning. A count tells us how many events were observed. A rate or proportion relates those events to the relevant number of opportunities, items, people or time exposed to the event. Advanced AI systems can calculate either, but an accurate comparison requires the correct denominator, consistent definitions and an honest explanation of what the figures do not prove.
This reader-friendly guide follows two fictional school proofreading teams, a pooled calculation and twenty test claims. It explains why a larger count can coexist with a lower rate, how to convert between per-hundred and per-thousand measures and when the denominator itself can be misleading. The results are invented, not a measured evaluation of real proofreading teams or AI models. “How Superintelligence Works” is our educational series title, not a claim that present systems have achieved artificial superintelligence.
1. A count is not a rate
A count is an integer describing how many defined events were recorded. “Six errors were found” gives a count. “Six errors were found among 1,200 checked questions” gives the additional exposure information needed to describe how common those errors were relative to the workload.
Both statements can be correct. But the first may be inadequate for comparing two teams if their work volumes differ greatly. AI summaries must preserve the denominator rather than treating every headline number as directly comparable.
2. The denominator makes a difference
Consider a school checking one hundred answers and finding five errors. That is five errors per hundred checks, or 5%. Another checks one thousand answers and finds ten errors: ten mistakes is a higher count but only 1%.
Without the denominator, an assistant might praise the smaller error count even though its relative error frequency was larger. The denominator defines the number of opportunities for the event.
3. “Per” expresses a relationship
Terms such as per student, per question, per hour and per thousand deliveries express rates or ratios. They are not interchangeable. A number of corrections per checked question is not the same as corrections per school day unless you know the relevant work volume and duration.
When a model sees a rate, it should preserve the unit on both sides: errors per checked question, minutes per lesson or visits per month. An unlabeled value such as 0.005 is ambiguous until the event and exposure are named.
4. What are we counting?
Before computing a rate, decide whether the numerator counts flawed questions, individual editing marks, documents containing at least one flaw or reports submitted for review. One document may have several spelling mistakes.
In this fictional experiment, the event is a checked question containing at least one confirmed substantive error. Each question contributes at most one event to the numerator. This makes the ratio a simple observed proportion of flawed questions within the checked set.
5. The fictional Cedar Proofreading Desk
Imagine two invented teams independently checking different sets of school revision questions under the same written error definition. Team A checks 1,200 questions and finds six flawed questions. Team B checks 3,200 questions and finds eight flawed questions.
We assume both teams’ reviewed questions are uniquely counted, their decisions have been independently confirmed and the two collections do not overlap. These simplifications make the arithmetic transparent but do not establish that the underlying question difficulty or reviewer conditions were identical.
6. Freeze the observations
- Team A: 1,200 checked questions.
- Team A: 6 flawed questions.
- Team A: 1,194 checked questions without a confirmed flaw.
- Team B: 3,200 checked questions.
- Team B: 8 flawed questions.
- Team B: 3,192 checked questions without a confirmed flaw.
- Combined checked questions: 4,400.
- Combined flawed questions: 14.
- All figures are invented and refer to unique checked questions.
- An observed error proportion is not an established causal measure of team competence.
7. Compute Team A’s rate
Divide the number of flawed questions by the number checked: 6 ÷ 1,200 = 0.005. As a percentage, that is 0.5%. Multiplying the proportion by one thousand gives five flawed questions per thousand checked questions.
The different formats express the same observed proportion. An AI that calls 0.005 “five percent” would be wrong by a factor of ten. Units and scale conversions matter even when the division is correct.
8. Compute Team B’s rate
Team B’s proportion is 8 ÷ 3,200 = 0.0025, or 0.25%. Per thousand checked questions, that is 2.5 flawed questions.
The decimal 2.5 is an average standardised rate, not an assertion that half a physical question existed. Standardised rates need not be whole numbers because they scale an observed ratio to a reference exposure.
9. Compare the raw counts
Team B recorded eight flawed questions versus Team A’s six, so B had two more flawed questions in absolute count. This is a true statement about the invented observations.
But B checked two thousand more questions: 3,200 versus 1,200. The larger workload changes how the two error counts should be compared for a question about frequency per checked item.
10. Compare the rates
A’s rate is 0.5%, and B’s rate is 0.25%. Therefore A’s observed error proportion is twice B’s: 0.5 ÷ 0.25 = 2. The difference is 0.25 percentage points.
This is not a claim that Team A is universally less skilled. The toy data do not show how hard each question was, who created the questions, or whether the two teams were assigned comparable material.
11. The phrase “twice as many” can mislead
Team A did not record twice as many errors in total. It recorded fewer. It has twice the proportion of flawed checked questions in this packet. These two comparisons use different denominators.
An assistant should name the comparison explicitly: B found more flawed questions because its inspected collection was larger, while A’s inspected set had the higher observed error proportion.
12. Relative difference versus percentage points
The rates differ by 0.5% − 0.25% = 0.25 percentage points. Relative to B’s rate, A’s rate is 100% higher. Relative to A’s rate, B’s rate is 50% lower.
Those two relative descriptions refer to the same pair of rates but choose different starting points. A careful explanation never says simply “a 50% difference” without identifying which rate was used as the baseline.
13. Combine both teams correctly
Across both teams, there are 6 + 8 = 14 flawed questions among 1,200 + 3,200 = 4,400 checked questions. The combined observed proportion is 14/4,400 ≈ 0.0031818, or about 0.3182%.
Expressed per thousand checked questions, the pooled figure is approximately 3.18. The combined rate lies between the two team rates because it is a weighted average based on their different checked-question volumes.
14. Why averaging two rates can be wrong
The simple mean of 0.5% and 0.25% is 0.375%. That would give equal weight to Team A and Team B, even though B checked many more questions. It is not the correctly pooled per-question rate for the combined 4,400-question collection.
The pooled rate must use the combined numerator and combined denominator, or equivalently an appropriately weighted average. A model that averages percentages without knowing group sizes can misstate an overall rate.
15. The checked comparison ledger
- Team A event count: 6; exposure: 1,200.
- Team A observed proportion: 6/1,200 = 0.005 = 0.5%.
- Team A per-thousand rate: 5.
- Team B event count: 8; exposure: 3,200.
- Team B observed proportion: 8/3,200 = 0.0025 = 0.25%.
- Team B per-thousand rate: 2.5.
- B’s absolute event count exceeds A’s by 2.
- A’s observed per-question rate is twice B’s.
- Combined event count: 14; combined exposure: 4,400.
- Pooled proportion: 14/4,400 ≈ 0.3182%.
- Pooled events per thousand: about 3.18.
- Unweighted mean of the two team percentages: 0.375%, not the pooled rate.
16. The denominator must describe the same opportunity
Suppose Team A reports mistakes per question while Team B reports mistakes per worksheet. A worksheet can contain several questions, so comparing those two ratios directly would be misleading.
Good AI data extraction records numerator definition, denominator definition and observation period separately. A percentage is not self-explanatory; it inherits the meaning of the quantity being counted.
17. When the numerator can exceed the denominator
If a system counts individual spelling errors rather than flawed questions, one question could contain several errors. A rate such as 1.2 errors per question is then possible, whereas a percentage of questions with at least one error cannot exceed 100%.
This demonstrates why an event rate and a binary-event proportion are not necessarily equivalent. Our teaching example uses the latter because each question can contribute at most one positive outcome.
18. Why time can be the exposure
Some activities are measured per hour rather than per item. For example, a system might record interruptions per 100 operating hours. If two devices operate for different durations, raw interruption totals are not directly comparable.
Such rates require a consistent definition of operating time. A device that ran for one hour cannot be compared fairly with a device that ran for a month using only total interruption count.
19. Risk language needs caution
An observed proportion of flawed questions is not automatically the probability that the next independent question will contain a flaw. That predictive interpretation needs assumptions about how future questions resemble the checked sample.
If the inspection covers an unusual set of difficult questions, the observed rate may not generalise to the usual workflow. The assistant should prefer “observed error proportion in this collection” over an unqualified prediction about future risk.
20. Twenty claims for an AI to classify
The fictional desk asks a classifier to label each statement SUPPORTED, CONTRADICTED or NOT ESTABLISHED from the given ledger and its explicitly defined metrics. The final category applies to claims about future tasks or differences in skill that the counts cannot settle.
The test is meant to expose denominator confusion. A numerically precise statement can be contradicted when it attaches the correct value to the wrong group or scale.
- 01. “Team A checked 1,200 questions.” — SUPPORTED.
- 02. “Team B checked 3,200 questions.” — SUPPORTED.
- 03. “A recorded six flawed questions.” — SUPPORTED.
- 04. “B recorded eight flawed questions.” — SUPPORTED.
- 05. “B had more flawed questions in absolute count.” — SUPPORTED.
- 06. “A had the lower observed flawed-question proportion.” — CONTRADICTED.
- 07. “A’s observed flawed-question proportion was 0.5%.” — SUPPORTED.
- 08. “B’s observed proportion was 0.25%.” — SUPPORTED.
- 09. “A’s proportion was twice B’s proportion.” — SUPPORTED.
- 10. “B’s count was twice A’s count.” — CONTRADICTED.
- 11. “Combined exposure was 4,400 questions.” — SUPPORTED.
- 12. “Combined flawed-question count was fourteen.” — SUPPORTED.
- 13. “The pooled proportion equals the unweighted mean of 0.5% and 0.25%.” — CONTRADICTED.
- 14. “The pooled per-thousand rate was about 3.18.” — SUPPORTED.
- 15. “Team A had five flawed questions per thousand checks.” — SUPPORTED.
- 16. “Team B had 2.5 flawed questions per thousand checks.” — SUPPORTED.
- 17. “Both teams checked equal numbers of questions.” — CONTRADICTED.
- 18. “Team B is guaranteed to make fewer errors on every future set.” — NOT ESTABLISHED.
- 19. “The observed difference proves a causal difference in staff skill.” — NOT ESTABLISHED.
- 20. “The pooled observed proportion is approximately 0.3182%.” — SUPPORTED.
21. Reconcile the results
Supported claims are 01, 02, 03, 04, 05, 07, 08, 09, 11, 12, 14, 15, 16 and 20: fourteen. Contradicted claims are 06, 10, 13 and 17: four. Not-established claims are 18 and 19: two. Fourteen plus four plus two equals twenty.
The unknown cases show that an observed rate difference does not prove a permanent performance hierarchy or a specific causal mechanism. Counts and denominators let us calculate the observed proportion, not everything we might want to know about the teams.
22. The temptation to call the smaller count safer
A public report may emphasise the group with fewer mistakes. That statement can be factually true and still invite a misleading comparison when workloads differ. If the reporting aim is the frequency of errors per check, use the appropriately adjusted rate.
However, the absolute count can still matter for the total number of items needing correction. A team with more cases may create a larger immediate review workload even while its per-item rate is lower. The right metric depends on the decision.
23. Counts and rates answer different operational questions
An administrator planning staff time might ask how many flawed questions must be corrected: the counts matter. Someone comparing proofreading processes might ask how common flawed questions are relative to output volume: the rates matter.
Neither metric should simply replace the other. The useful answer may report both and say what each reveals. Clear writing prevents an apparently contradictory pair of facts from becoming a confused argument.
24. Aggregation can hide subgroup changes
Suppose one team’s work contains mostly simple questions and the other’s mostly advanced mathematical proofs. Different mixes could explain part of the observed difference in error rates. We cannot check that possibility from the toy ledger.
In a real evaluation, the source may need to report rates by difficulty, topic or reviewer experience. An overall percentage is often only a starting point. Subgroup analysis should be appropriately designed rather than invented after the result.
25. Simpson’s paradox as a warning
When groups contain different mixtures of cases, aggregate patterns can differ from patterns observed within each subgroup. This phenomenon is often discussed as Simpson’s paradox. The correct response is to inspect group composition and causal context, not to assume all aggregated statistics are invalid.
Our fictional packet does not contain the subgroup data needed to demonstrate such a reversal. The concept is a methodological caution, not a hidden finding from these numbers.
26. Avoid false precision
The pooled percentage is a repeating decimal when calculated exactly from 14/4,400. Reporting approximately 0.318% or 3.18 per thousand is reasonable for a classroom explanation. Giving ten decimal places does not create more information.
Report the source counts alongside a rounded rate so readers can check the arithmetic and know the underlying sample size. Precision of presentation is different from certainty about generalisation.
27. A larger denominator is not an excuse for bad work
B’s lower rate does not erase the eight genuine flaws that need correction. The count still represents real items in the fictional record. A relative comparison helps interpret the workload, but it does not imply that any error can be ignored.
A practical quality system tracks corrections and seeks better processes while reporting both total workload and error frequency. The metric should support work, not replace it.
28. A fictional baseline error
Imagine an invented assistant that reads “eight is greater than six” and declares B the lower-quality team. It never checks how many questions each team reviewed. The arithmetic is valid, but the ranking is incomplete for a per-question comparison.
A corrected process identifies the event definition, exposure and rates, then states both the count and adjusted comparison. This is a teaching demonstration rather than a measured improvement in a real model.
29. A transfer exercise with reading progress
One class completes forty reading exercises in ten lessons; another completes sixty in thirty lessons. The totals favour the second class, but exercises per lesson are four versus two. Neither value alone proves which class learned more.
The denominator gives a clearer measure of throughput, yet the educational outcome would still require independent assessment of understanding and task quality.
30. A transfer exercise with school attendance
A school has ten absences among one hundred recorded pupil-days in one activity and fifteen among three hundred pupil-days in another. The raw counts are ten and fifteen, but observed absence proportions are 10% and 5% respectively.
The larger event count again corresponds to the lower exposure-adjusted proportion. The example does not establish why attendance differed. Appropriate risk language would need more context.
31. A transfer exercise with AI reliability
Suppose an AI application logs twenty failed tasks in a thousand attempts and another logs thirty in three thousand attempts. Their observed task failure proportions are 2% and 1%. The second has more failures but a lower rate.
Before concluding it is more reliable for your workload, check whether tasks, success criteria and observation methods are comparable. A numerical proportion cannot repair an inconsistent definition of success.
32. How to build an auditable comparison
Write down the target event, eligible population or exposure, source period, number of events, denominator and units of the resulting rate. Compute group proportions separately. Use the same scaling unit when comparing them. Then show both counts and rates.
If combining groups, add their event counts and exposures before computing the pooled result. Do not average percentages without accounting for different group sizes, unless the objective explicitly assigns equal weight to each group.
33. What to check before publishing an AI report
Ask whether the event count and denominator come from the same period and population. Are there duplicated items? Did one team perform harder tasks? Do multiple incidents occur per opportunity? Is the figure a proportion or an event rate? Were any observations missing?
These questions are not obstacles to communication. They let the writer make a precise statement that does not quietly change the purpose of the comparison.
Frequently asked questions
Is a larger number of errors always a higher error rate? No. The number of opportunities may be much larger.
What is the difference between a count and a rate? A count records events; a rate relates events to a specified exposure or denominator.
Why did eight errors correspond to a lower rate than six? B checked 3,200 questions while A checked 1,200.
What does per thousand mean? The observed ratio scaled to one thousand comparable opportunities.
Can you average two percentages? Sometimes for a defined equal-weight objective, but the pooled per-item proportion needs exposure-weighting.
Does a lower observed rate prove better staff skill? No. Differences in case mix, selection and measurement may matter.
Can a proportion exceed 100%? Not when it counts the share of unique opportunities with a binary outcome; event rates counting multiple events per opportunity can behave differently.
The central lesson
A count tells us what happened in total. A rate tells us how frequently it happened relative to a clearly defined opportunity. Both can be true while suggesting different practical conclusions, because they answer different questions.
A reliable AI assistant does not stop at whichever number is bigger. It identifies the denominator, checks comparability, reports counts and rates and avoids inventing a causal explanation. That is how a simple piece of arithmetic becomes a trustworthy comparison.
Connected lessons and further references
- How Super Intelligence Works — complete curriculum.
- Comparisons and Baselines — what better actually means.
- Quantifiers — groups, numerators and denominators.
- Sampling Bias — who was included in the evidence.
- CDC Field Epidemiology Manual: Describing Epidemiologic Data — appropriate denominators, rates and data interpretation.
- NIST/SEMATECH e-Handbook of Statistical Methods — statistical definitions and comparisons.
- NIST AI Risk Management Framework — risk-aware AI assessment.