Choose your reading route
eduKateSG · Why Mathematics? · Equivalence Testing: TOST, Margins and Evidence of Practical Similarity
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 8–14
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 15–21
Check assumptions
See where design and calibration matter.Route 4 · Chapters 22–28
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 29–42
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why is mathematics important when the question is whether two methods are close enough for a real purpose? A conventional failure to find a difference does not establish similarity. Equivalence testing reverses the burden: it defines an acceptable difference in advance and asks whether the full uncertainty interval fits inside that range.
Did You Know? Two one-sided tests at the 5% level correspond to checking whether a 90% two-sided confidence interval lies entirely inside the equivalence margins, under the same assumptions.
Full chapter index
Chapters 1–7 · Understand the question
Chapters 8–14 · Reproduce the example
Chapters 15–21 · Check assumptions
Chapters 22–28 · Explore applications
Chapters 29–42 · Study and verify
- Communication
- A spreadsheet workflow
- A coding workflow
- Applications and careers
- A student investigation
- Common misconceptions
- Connection to recent mathematics articles
- Guidance for students and parents
- Practice questions
- Frequently asked questions
- A reporting checklist
- The number-line test
- Final equivalence audit
- Useful next reading
Suppose a new assessment method is easier to use. The goal may be to show its average score differs from the established method by no more than two marks in either direction.
A large p-value in a difference test can result from true similarity or from imprecise data. Equivalence requires positive evidence that differences outside the acceptable range are incompatible with the data.
State exactly what difference is being estimated: new minus reference mean, log ratio, paired change or another population quantity. Direction determines every later inequality.
Let the acceptable interval be -2 to +2 marks. These margins must come from subject knowledge, measurement quality and decision consequences before results are inspected.
Suppose the estimated mean difference is 0.5 marks with standard error 0.6 marks. Use alpha=0.05 for each of the two one-sided tests.
| Lower margin | Estimate | Upper margin | SE |
|---|---|---|---|
| -2 | 0.5 | 2 | 0.6 |
The large-sample 90% critical value is about 1.645. The margin of error is 1.645×0.6=0.987 marks.
The 90% interval is 0.5±0.987, giving approximately -0.487 to 1.487 marks.
Both endpoints lie inside -2 and +2, so the data support equivalence at these prespecified margins and assumptions.
| Lower | Estimate | Upper |
|---|---|---|
| -0.487 | 0.5 | 1.487 |
The first null says the true difference is at or below -2. Rejecting it supports that the difference is greater than the lower margin.
The second null says the true difference is at or above +2. Rejecting it supports that the difference is less than the upper margin.
Equivalence needs rejection of both nonequivalence nulls. Passing only one establishes, at most, that one unacceptable direction has been ruled out.
Current FDA guidance describes equivalence comparisons using a criterion, a confidence interval and predetermined limits, and notes the two one-sided-tests basis for average bioequivalence. Read the FDA guidance.
CHAPTER 13 OF 42 · Reproduce the example
13. Narrower margins require more precision
If the acceptable range were -1 to +1, the same interval would not fit. A more demanding practical claim needs more information or less variability.
Equivalence power depends strongly on the margins, expected true difference, variability, design and dropout. Planning at an unrealistically perfect true difference can be fragile.
When the same participant receives both methods, within-person correlation can reduce variance. Period, sequence and carryover effects must still be addressed.
Independent groups avoid some carryover concerns but often need more participants for the same precision. The analysis must match allocation and variance structure.
Pharmacokinetic outcomes are often analysed on a log scale, where additive log differences correspond to ratios after exponentiation. Symmetry on the log scale is multiplicative on the original scale.
Some settings tolerate more difference in one direction than the other. Asymmetric margins are valid when scientifically justified and prespecified.
Missing outcomes can widen intervals and bias the estimated difference. Sensitivity analyses should examine departures from the missing-data assumptions.
Claiming equivalence on several primary outcomes creates multiplicity questions. The required family rule depends on whether all endpoints or any endpoint must pass.
A comparison can appear equivalent because both methods are ineffective or the study cannot distinguish meaningful differences. Design quality and a responsive outcome are essential.
An interval can support equivalence and also exclude zero if it is narrow and shifted. These are different claims and should be reported separately.
Equivalent group means do not guarantee that individual measurements agree. Method-comparison work may need paired differences and limits of agreement.
The conclusion is not that the methods are exactly equal. It is that differences outside a chosen practical range were ruled out with stated uncertainty.
A digital and paper test might be considered equivalent for average scores within two marks, yet still differ for particular subgroups or individual students.
Check the estimand, subtraction direction, margin source, confidence level, design, standard error, missingness, multiplicity and individual-agreement needs.
Reproduce -0.487 to 1.487, tighten the margins, increase the standard error, shift the estimate and compare TOST with a difference test.
Equivalence testing does not prove identical mechanisms, equal safety, equal individual responses or transfer to populations outside the study.
Say “equivalent within prespecified margins of ±2 marks” rather than “the methods are the same.” The margin is part of the conclusion.
Keep estimate, SE, critical value, lower and upper margins, interval endpoints and both one-sided decisions in labelled cells.
Implement TOST with explicit direction and unit tests, then verify the confidence-interval decision against a trusted statistical package.
Quality engineers, clinicians, psychologists, educators and product teams use equivalence tests when practical similarity is the real decision.
Simulate paired scores under true differences of 0, 1 and 2.5 marks, then compare difference-test and equivalence-test conclusions.
Nonsignificance is not equivalence, a 90% interval is not chosen to hide uncertainty, and equivalence margins are not data-driven error bars.
The effect-size guide helps define meaningful magnitude. The confidence-interval guide explains interval coverage.
Students should draw the margins and interval on one number line. Parents can ask who decided what difference in marks would be educationally acceptable.
- Reproduce the 90% interval.
- Test ±1 margins.
- Increase SE to 1.0.
- Reverse the subtraction order.
- Distinguish equivalence from agreement.
### Does p>0.05 prove equivalence? No.
### Why a 90% interval? It matches two one-sided tests at alpha=0.05.
### Are margins universal? No.
### Does equivalence mean identical? No.
Report the estimand and direction, margins and justification, design, estimate, SE, 90% interval, both one-sided tests, missing-data analysis, multiplicity and practical interpretation.
Place the lower and upper equivalence margins on a number line, then place the full confidence interval. The claim succeeds only when the whole interval is inside.
Confirm the margin source, direction, matching confidence level, design-aware SE, complete sensitivity analyses and wording that preserves the chosen practical range.
Continue through the Mathematics Learning Hub. Recalculate one decision boundary, state its assumptions and explain which conclusion it can and cannot support.
Extended practice laboratory
Practice case 1: Define estimand
Setup: Start from the article’s equivalence-TOST mechanism for case 1. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 2: Set lower margin
Setup: Start from the article’s equivalence-TOST mechanism for case 2. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 3: Set upper margin
Setup: Start from the article’s equivalence-TOST mechanism for case 3. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 4: Set alpha
Setup: Start from the article’s equivalence-TOST mechanism for case 4. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 5: Set estimate
Setup: Start from the article’s equivalence-TOST mechanism for case 5. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 6: Set standard error
Setup: Start from the article’s equivalence-TOST mechanism for case 6. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 7: Calculate 90% margin
Setup: Start from the article’s equivalence-TOST mechanism for case 7. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 8: Calculate lower bound
Setup: Start from the article’s equivalence-TOST mechanism for case 8. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 9: Calculate upper bound
Setup: Start from the article’s equivalence-TOST mechanism for case 9. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 10: Run lower one-sided test
Setup: Start from the article’s equivalence-TOST mechanism for case 10. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 11: Run upper one-sided test
Setup: Start from the article’s equivalence-TOST mechanism for case 11. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 12: Compare interval with margins
Setup: Start from the article’s equivalence-TOST mechanism for case 12. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 13: Narrow margins
Setup: Start from the article’s equivalence-TOST mechanism for case 13. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 14: Increase standard error
Setup: Start from the article’s equivalence-TOST mechanism for case 14. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 15: Increase sample size
Setup: Start from the article’s equivalence-TOST mechanism for case 15. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 16: Add paired design
Setup: Start from the article’s equivalence-TOST mechanism for case 16. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 17: Add multiplicity
Setup: Start from the article’s equivalence-TOST mechanism for case 17. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 18: Add missingness
Setup: Start from the article’s equivalence-TOST mechanism for case 18. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 19: Add transformation
Setup: Start from the article’s equivalence-TOST mechanism for case 19. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 20: Check direction
Setup: Start from the article’s equivalence-TOST mechanism for case 20. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 21: Audit margin source
Setup: Start from the article’s equivalence-TOST mechanism for case 21. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 22: Audit power
Setup: Start from the article’s equivalence-TOST mechanism for case 22. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 23: Audit assumptions
Setup: Start from the article’s equivalence-TOST mechanism for case 23. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 24: Name limits
Setup: Start from the article’s equivalence-TOST mechanism for case 24. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 25: Write conclusion
Setup: Start from the article’s equivalence-TOST mechanism for case 25. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.