Adding a long list of computer numbers is not always the same as adding the mathematical values they represent. Floating-point arithmetic stores a finite set of significands and exponents, so a small addend can disappear when it is combined with a much larger running total. Kahan compensated summation keeps a correction variable that carries a model of recently lost low-order information into the next addition. Mathematics matters because place value, rounding, conditioning and error analysis explain why ordinary summation can drift, why compensation often helps dramatically, and why no short recurrence can make every ill-conditioned sum exact.
The practical search question is not simply “what is this named method?” but “which mathematical structure makes the method work, what does it guarantee, and what evidence would show that it is a poor choice?” This article keeps those questions together so students can connect mathematics education with computing, engineering, data reasoning and careful everyday decisions without turning one technique into a career promise.
Quick navigation: Why it matters · Mechanism · Worked example · Deeper ideas · Investigations · Limits · Learning route · Parent guidance · FAQs · Sources
Why this mathematics matters
Mathematics is important here because it turns an apparently visual or computational trick into a chain of reasons. The representation tells us what information is kept. The equation or invariant explains each legal step. The complexity analysis describes the work. The boundary conditions say when a conclusion should not be trusted. That combination is a practical form of problem-solving literacy.
A strong learner should be able to move in both directions: from a real task to a mathematical model, and from a formal guarantee back to its real meaning. Naming an algorithm is not enough. The useful achievement is to explain why the next operation is valid, check a small instance independently, and notice which assumptions the implementation has added.
Source: William Kahan’s 1965 note on reducing truncation errors.
Primary or implementation reference: Oracle’s reproduction of Goldberg’s floating-point arithmetic tutorial.
How the mechanism works
Start with sum=0 and c=0. For each floating-point input x, compute y=x−c, t=sum+y, c=(t−sum)−y, then sum=t. The expression (t−sum) reconstructs the part of y that the rounded addition appears to have accepted; subtracting y estimates the rounding residue under this sign convention. On the next iteration, x−c feeds that residue back. Parentheses and evaluation order are part of the algorithm. Optimising compilers or fast-math modes that reassociate expressions may erase the correction. Neumaier’s variant adds a separate compensation and can be more robust when the next term is larger in magnitude than the running total.
For round-to-nearest floating-point arithmetic, each elementary operation is commonly modelled as fl(a op b)=(a op b)(1+δ), with |δ| bounded by unit roundoff when no overflow, underflow or exceptional case intervenes. Naive left-to-right summation can accumulate error that grows with the number and order of addends. Compensation does not change the exact mathematical sum; it changes the path by which rounded operations approach it. Its error is often comparable to a small multiple of unit roundoff times the sum of absolute values, rather than a multiple that grows directly with n. Cancellation still makes relative error large when the true result is tiny compared with the magnitudes being combined.
Keep a four-column notebook labelled object, rule, guarantee and caveat. In the first column name the inputs and stored state. In the second write one update precisely. In the third state what remains true after the update. In the fourth record a tie, approximation, domain restriction or hardware assumption. This routine prevents symbols from drifting away from their meaning.
Worked example and core calculations
Use a toy decimal machine that rounds to four significant digits and add [10000, 3.141, −10000]. Naive summation rounds 10000+3.141 back to 10000, then returns 0, although the exact sum is 3.141. Kahan’s recurrence records c=−3.141 after the lost addition. Before the final addition it forms y=−10000−(−3.141)=−9996.859, which rounds to −9997; adding this to 10000 returns 3. The compensated answer is not exact in this deliberately coarse format, but it recovers the leading lost information while the naive order loses it completely. Reversing or sorting the sequence can produce another result, which is why order, format and reference sum belong in the audit.
How to audit the calculation
- Recompute every intermediate value directly from the definition.
- Keep labels, indices, signs, units and normalisation visible.
- Test a case with a known exact answer.
- State tie rules, random seeds, tolerances and boundary conventions.
- Compare with a slower transparent baseline or trusted implementation.
- Separate agreement of numbers from suitability of the real-world model.
Small examples are not childish; they are diagnostic instruments. A five-line trace can reveal a reversed inequality, a missing scale factor or an inconsistent label convention before a large dataset hides the same error behind plausible output.
Four deeper ideas
The compensation is state, not extra precision
It stores a correction in the same format; the improvement comes from reorganising rounding errors, not from secretly creating exact arithmetic.
To transfer this idea, ask what would remain true if labels, scale or input order changed. Then identify the step whose justification would fail if the stated assumption were removed. This turns the paragraph into a reusable reasoning pattern rather than a fact to memorise.
Did you know? Algebraic equivalence can fail computationally
Real-number identities permit reassociation, but floating-point addition is not associative. Compiler settings can therefore change both the value and the validity of the recurrence.
To transfer this idea, ask what would remain true if labels, scale or input order changed. Then identify the step whose justification would fail if the stated assumption were removed. This turns the paragraph into a reusable reasoning pattern rather than a fact to memorise.
Conditioning and algorithmic stability are different questions
A well-designed summation cannot make a near-cancelling problem insensitive to uncertain inputs; it can only reduce error introduced by the computation itself.
To transfer this idea, ask what would remain true if labels, scale or input order changed. Then identify the step whose justification would fail if the stated assumption were removed. This turns the paragraph into a reusable reasoning pattern rather than a fact to memorise.
Independent references reveal hidden loss
A higher-precision, exact-integer, decimal or expansion-based sum on a small dataset can separate mathematical truth from one floating-point pathway.
To transfer this idea, ask what would remain true if labels, scale or input order changed. Then identify the step whose justification would fail if the stated assumption were removed. This turns the paragraph into a reusable reasoning pattern rather than a fact to memorise.
Twelve practical investigations
1. Four-digit toy arithmetic
Repeat the worked three-term sum. Round every operation explicitly.
What the mathematics reveals: Lost low-order information becomes visible. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: A toy format illustrates rather than benchmarks. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
2. Many tiny increments
Add one million small equal terms to a large base. Compare naive and compensated totals.
What the mathematics reveals: Repeated absorption accumulates. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Use a high-precision oracle. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
3. Alternating cancellation
Sum large positive and negative terms. Compare several orders.
What the mathematics reveals: Conditioning can dominate algorithm choice. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Relative error may explode near zero. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
4. Sorted magnitudes
Sum from smallest to largest and reverse. Measure both error and cost.
What the mathematics reveals: Order influences rounding. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Sorting changes streaming behaviour. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
5. Neumaier variant
Use a term larger than the current total. Trace its branch and compensation.
What the mathematics reveals: A variant handles magnitude reversals differently. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: State the exact recurrence. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
6. Pairwise reduction
Build a balanced summation tree. Compare depth and parallelism.
What the mathematics reveals: Balanced errors can scale better. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Tree order remains part of the result. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
7. Parallel blocks
Compensate within blocks then combine. Vary worker count.
What the mathematics reveals: Reproducibility needs a fixed reduction plan. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Do not promise bitwise identity. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
8. Overflow boundary
Add values near the largest finite number. Observe exceptional results.
What the mathematics reveals: Compensation cannot extend exponent range. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Scale or redesign first. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
9. Subnormal range
Add values near underflow. Inspect gradual underflow behaviour.
What the mathematics reveals: Representation limits shape accuracy. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Hardware modes may flush to zero. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
10. Compiler audit
Compile with strict and fast modes. Compare generated results.
What the mathematics reveals: Evaluation semantics matter. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Record flags and architecture. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
11. Exact-integer oracle
Scale suitable decimals to integers. Compare final totals.
What the mathematics reveals: Representation can remove a whole error source. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Scaling must fit the integer range. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
12. Property test
Generate random magnitude mixtures. Compare with high precision.
What the mathematics reveals: Error distributions reveal patterns. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Keep seeds and worst cases. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
Limits and misconceptions
Kahan summation is not exact arithmetic, a proof of correct input data, or a cure for overflow, underflow, NaNs and infinities. It may still struggle when a very large term follows accumulated smaller terms; Neumaier or pairwise methods may behave better. Parallel reductions change order and need a deliberate compensated or pairwise design. Hardware instructions, fused operations, extended registers and compiler reassociation can change details. For nonnegative data the absolute error story differs from a heavily cancelling signed sum. If every cent, count or combinatorial value must be exact, use an appropriate integer, decimal or arbitrary-precision representation rather than treating compensation as a legal or accounting guarantee.
Five misconceptions to challenge
- The correction makes the sum exact: It usually reduces rounding error but remains finite-precision arithmetic.
- Floating-point addition is associative: Changing parentheses or reduction order can change the result.
- A small final residual means the data are accurate: It says nothing about measurement, transcription or model error.
- Compensation always beats pairwise summation: The best method depends on magnitude pattern, parallelism, precision and cost.
- Fast-math optimisation cannot affect a mathematical identity: Reassociation can remove the deliberately ordered correction steps.
Another broad misconception is that advanced mathematics automatically creates intelligence, admission, income or employment. It does not. Deliberate study can strengthen modelling, abstraction, calculation and explanation, while real pathways also depend on interests, communication, domain knowledge, opportunities and circumstances outside one topic.
A learning route for students
Move from a visible trace to a symbolic rule, then to code and critique. For every step below, prepare three pieces of evidence: a correct hand example, a boundary or failure case, and a short explanation using the relevant invariant. Do not advance merely because software returned a number.
Step 1: Explain floating-point representation
Begin with the smallest non-trivial instance that lets you explain floating-point representation. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 2: Trace the compensation recurrence
Begin with the smallest non-trivial instance that lets you trace the compensation recurrence. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 3: Identify rounding absorption
Begin with the smallest non-trivial instance that lets you identify rounding absorption. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 4: Measure absolute and relative error
Begin with the smallest non-trivial instance that lets you measure absolute and relative error. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 5: Separate conditioning from stability
Begin with the smallest non-trivial instance that lets you separate conditioning from stability. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 6: Compare summation orders
Begin with the smallest non-trivial instance that lets you compare summation orders. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 7: Implement neumaier’s variant
Begin with the smallest non-trivial instance that lets you implement Neumaier’s variant. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 8: Design a pairwise reduction
Begin with the smallest non-trivial instance that lets you design a pairwise reduction. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 9: Audit compiler assumptions
Begin with the smallest non-trivial instance that lets you audit compiler assumptions. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 10: Choose exact representation when required
Begin with the smallest non-trivial instance that lets you choose exact representation when required. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
A four-week practice plan
- Week 1 — vocabulary and diagrams: define each object, redraw the mechanism and reproduce the worked example.
- Week 2 — equations and manufactured tests: derive the key relation and test inputs with known answers.
- Week 3 — implementation and evidence: use a trusted library, record parameters and compare with a transparent baseline.
- Week 4 — transfer and critique: apply the method to a new context, report limits and identify when a different method is better.
Guidance for parents and educators
Ask “what stays true?” and “what could make this conclusion fail?” before asking for speed. Invite the student to draw the input and output, estimate the result and locate the first step that disagrees with a reference. These prompts reveal conceptual understanding more clearly than memorised terminology.
Advanced enrichment should sit beside secure school foundations in algebra, functions, geometry, statistics and computational thinking. It need not accelerate a student into a fixed career identity. A healthy goal is curiosity with discipline: the learner becomes comfortable reading unfamiliar notation, checking an example, revising a model and communicating uncertainty.
Evidence prompts for a useful conversation
- Which quantity is observed and which is inferred?
- Which step saves work, and why is it allowed?
- Is the claim exact, approximate, probabilistic or empirical?
- What units and conventions are in use?
- Which input makes the method struggle?
- What simple baseline could check the answer?
- What would count as evidence that the parameters are poor?
Mini-project assessment
A strong mini-project contains the original question, a small reproducible dataset, a hand-worked example, code or a spreadsheet trace, at least one boundary test, a comparison with a baseline, and a paragraph on limits. Assess the reasoning trail as well as the final output. Repairing a model after a surprising result is valuable mathematical progress.
Six evidence checks for a finished project
Check 1: Explain floating-point representation
Ask the student to demonstrate how to explain floating-point representation without relying on a hidden software step. The evidence should include a labelled input, the exact rule used, one intermediate state and a result that can be checked independently. If the answer changes after labels or order change, the student should say whether that reflects the mathematics or only a convention.
Then request one deliberately awkward example. The learner should predict the difficulty before running the method, explain what the output does and does not establish, and name a reasonable alternative or safeguard. This check joins correctness with judgement: a technically valid calculation is useful only when its assumptions and interpretation remain visible.
Check 2: Trace the compensation recurrence
Ask the student to demonstrate how to trace the compensation recurrence without relying on a hidden software step. The evidence should include a labelled input, the exact rule used, one intermediate state and a result that can be checked independently. If the answer changes after labels or order change, the student should say whether that reflects the mathematics or only a convention.
Then request one deliberately awkward example. The learner should predict the difficulty before running the method, explain what the output does and does not establish, and name a reasonable alternative or safeguard. This check joins correctness with judgement: a technically valid calculation is useful only when its assumptions and interpretation remain visible.
Check 3: Identify rounding absorption
Ask the student to demonstrate how to identify rounding absorption without relying on a hidden software step. The evidence should include a labelled input, the exact rule used, one intermediate state and a result that can be checked independently. If the answer changes after labels or order change, the student should say whether that reflects the mathematics or only a convention.
Then request one deliberately awkward example. The learner should predict the difficulty before running the method, explain what the output does and does not establish, and name a reasonable alternative or safeguard. This check joins correctness with judgement: a technically valid calculation is useful only when its assumptions and interpretation remain visible.
Check 4: Measure absolute and relative error
Ask the student to demonstrate how to measure absolute and relative error without relying on a hidden software step. The evidence should include a labelled input, the exact rule used, one intermediate state and a result that can be checked independently. If the answer changes after labels or order change, the student should say whether that reflects the mathematics or only a convention.
Then request one deliberately awkward example. The learner should predict the difficulty before running the method, explain what the output does and does not establish, and name a reasonable alternative or safeguard. This check joins correctness with judgement: a technically valid calculation is useful only when its assumptions and interpretation remain visible.
Check 5: Separate conditioning from stability
Ask the student to demonstrate how to separate conditioning from stability without relying on a hidden software step. The evidence should include a labelled input, the exact rule used, one intermediate state and a result that can be checked independently. If the answer changes after labels or order change, the student should say whether that reflects the mathematics or only a convention.
Then request one deliberately awkward example. The learner should predict the difficulty before running the method, explain what the output does and does not establish, and name a reasonable alternative or safeguard. This check joins correctness with judgement: a technically valid calculation is useful only when its assumptions and interpretation remain visible.
Check 6: Compare summation orders
Ask the student to demonstrate how to compare summation orders without relying on a hidden software step. The evidence should include a labelled input, the exact rule used, one intermediate state and a result that can be checked independently. If the answer changes after labels or order change, the student should say whether that reflects the mathematics or only a convention.
Then request one deliberately awkward example. The learner should predict the difficulty before running the method, explain what the output does and does not establish, and name a reasonable alternative or safeguard. This check joins correctness with judgement: a technically valid calculation is useful only when its assumptions and interpretation remain visible.
Frequently asked questions
What problem does Kahan summation address?
Rounding error accumulated while adding finite-precision floating-point values.
What does the correction variable store?
An estimate of low-order information lost by the previous rounded addition under the declared sign convention.
Is the result exact?
No.
Why do the parentheses matter?
They force the sequence of rounded operations on which the correction depends.
Can a compiler break the method?
Yes, if it reassociates or simplifies expressions under permissive floating-point rules.
What is Neumaier summation?
A compensated variant that handles some large-next-term cases more robustly.
Is pairwise summation useful?
Yes, especially for balanced and parallel reductions.
Does compensation fix bad measurements?
No; it addresses computation error, not data or model error.
When should integers or decimals be used?
When the quantity has an exact discrete or decimal contract and the range is manageable.
What should students know first?
Place value, scientific notation, significant figures, error bounds and loops.
Useful next reading and sources
Source: William Kahan’s 1965 note on reducing truncation errors.
Source: Oracle’s reproduction of Goldberg’s floating-point arithmetic tutorial.
Related eduKateSG reading: Numerical stability, conditioning and cancellation.
Related eduKateSG reading: Welford’s algorithm and online variance.
Related eduKateSG reading: Bareiss elimination and exact arithmetic.
Read sources with a question in mind: which definition, theorem, default or limitation is being supported? An authoritative link is useful evidence, but it does not replace a worked calculation or a clear statement of how the source applies to the example.
A cheerful final thought
Mathematics becomes friendlier when every symbol has a job and every claim has a boundary. Start small, make the state visible, test the awkward case, and celebrate the moment when a pattern becomes explainable. That is the practical benefit of learning mathematics: not certainty about every future, but a growing ability to ask better questions and build trustworthy answers.