Choose your reading route
eduKateSG · Why Mathematics? · Combination Tests: Stagewise P-Values, Inverse-Normal Weights and Valid Adaptation
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 8–14
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 15–21
Check assumptions
See where design and calibration matter.Route 4 · Chapters 22–28
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 29–42
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why is mathematics important when an experiment changes after an interim analysis? Combination tests preserve a valid overall test by converting independent stagewise evidence into a prespecified statistic. The design can adapt sample size or selection while keeping the evidential rule fixed.
Did You Know? The second-stage sample size may depend on first-stage results, yet a valid combination test can still control Type I error when stagewise p-values remain properly calibrated and the combination rule was fixed in advance.
Full chapter index
Chapters 1–7 · Understand the question
Chapters 8–14 · Reproduce the example
Chapters 15–21 · Check assumptions
Chapters 22–28 · Explore applications
Chapters 29–42 · Study and verify
- Communication
- A spreadsheet workflow
- A coding workflow
- Applications and careers
- A student investigation
- Common misconceptions
- Connection to recent mathematics articles
- Guidance for students and parents
- Practice questions
- Frequently asked questions
- A reporting checklist
- Why inverse normal works
- Final combination-test audit
- Useful next reading
After stage one, investigators may want to increase sample size, select a promising arm or refine a population. Ordinary pooling after data-driven changes can distort the reference distribution.
Divide the study into disjoint stages and calculate a valid p-value from each stage under the same null hypothesis. Stage separation makes the evidential contributions explicit.
An inverse-normal combination test transforms each one-sided p-value into a normal score and takes a weighted sum. The weights and rejection boundary are prespecified.
For two equally weighted stages, use w1=w2=1/sqrt(2), so w1 squared plus w2 squared equals 1. Weights often reflect planned information fractions rather than observed convenience.
Let the one-sided stage-one p-value be 0.04. The upper-tail normal score is z1=Phi inverse of 0.96, approximately 1.7507.
Let the independent one-sided stage-two p-value be 0.01. Its score is z2=Phi inverse of 0.99, approximately 2.3263.
The equal-weight statistic is (1.7507+2.3263)/sqrt(2)=2.8829 after rounding. Under the null and required calibration, it follows a standard normal distribution.
The one-sided upper-tail probability beyond 2.8829 is about 0.00197. This is stronger than either stage alone because both contribute evidence in the same direction.
| p1 | p2 | z1 | z2 | Combined Z |
|---|---|---|---|---|
| 0.04 | 0.01 | 1.7507 | 2.3263 | 2.8829 |
Stage-two data should be independent of stage-one data conditional on the design where required. Reusing the same outcome observations in both p-values double-counts evidence.
| Combined Z | One-sided p-value |
|---|---|
| 2.8829 | 0.00197 |
A changed endpoint, population or estimand may no longer test the same null. A combination formula cannot repair a scientific question that silently changed.
CHAPTER 11 OF 42 · Reproduce the example
11. Weights are not chosen after the result
Selecting weights to maximise the observed combined z score would introduce another data-driven choice. Prespecified or conditionally valid weighting rules are necessary.
A design can include stage-one stopping boundaries alongside a final combination rule. The joint rule, including whether futility is binding, determines overall error and power.
Stage-two size may increase after weak but promising evidence. The stage-two p-value must remain valid despite the chosen size, and the original combination rule must be respected.
Bauer and Köhne proposed combining p-values from disjoint subsamples for adaptive interim designs and discussed stopping, sample size and bias. Read the PubMed record.
Lehmacher and Wassmer described an inverse-normal method enabling data-driven sample-size reassessment without inflating Type I error under the method’s conditions. Read the PubMed record.
Conditional error is the probability, under the null, that the original design will reject given the interim data. A modified design may be acceptable if its conditional rejection probability does not exceed that budget.
If only the best-looking arm continues, selection must be included in the multiple-testing and combination strategy. A single unadjusted stage-two p-value can ignore the selection advantage.
Enriching for a subgroup after interim results changes the family of claims. Closed testing, intersection hypotheses or other multiplicity methods may be needed.
A more sensitive endpoint chosen after seeing correlations and trends can create selection bias. Scientific relevance and a valid joint testing strategy are both required.
The stage-one cutoff must be locked. Late outcomes, corrections and missing values need rules so the first p-value is reproducible and not selectively revised.
A valid hypothesis test does not automatically give an unbiased effect estimate or nominal ordinary confidence interval. Selection and sample-size changes can affect estimation.
Simulate the entire adaptive algorithm under every relevant null configuration. Record familywise error, power, selection rates, expected sample, early stopping and estimator behaviour.
If 200,000 null simulations estimate a 2.5% error rate, the Monte Carlo standard error is about sqrt(0.025×0.975/200000)=0.000349. Report it with the estimate.
Only an independent group should access unblinded stage-one evidence. The adaptation communicated to sites should reveal as little as possible about interim results.
A two-stage school trial might expand sample after stage one or continue only one teaching approach. Schools, outcomes and cohorts must remain comparable enough for the combined claim.
Check disjoint data, p-value direction, uniform calibration under the null, common hypothesis, squared weights summing to one, selection multiplicity and stage cutoff.
Reproduce z1, z2, 2.8829 and 0.00197 with a trusted normal function. Change weights and p-values, then simulate uniform null p-values to verify the rejection rate.
Combination testing does not justify arbitrary redesign, prove comparability across stages or solve estimation automatically. Validity depends on the stagewise tests and adaptation framework.
Report stagewise p-values, weights, transformation, combined statistic, boundary and every adaptation. Do not present the combined p-value as if it came from a fixed design.
Place each stage’s cutoff, hypothesis, sample, p-value, normal score, weight and contribution in columns. Sum weighted scores only after independent checks of direction and calibration.
Use a trusted inverse-normal function, test extreme p-values safely and simulate uniform stagewise p-values. Build separate modules for selection, adaptation and final combination.
Adaptive-trial statisticians, platform designers, quality engineers and experimenters use combination methods when a planned evidential rule must survive data-informed design changes.
Generate pairs of independent uniform p-values and verify a 2.5% one-sided rejection rate. Then add selection or dependence deliberately and observe how validity changes.
A valid combined test does not make every adaptation valid, independent stages do not mean unrelated scientific populations, and a small combined p-value is not an effect size.
The multiple-testing guide frames selection. The sequential-testing guide explains prespecified stopping evidence.
Students should write the hypothesis above both stagewise p-values before combining them. Parents can ask whether a changed school trial still measures the same outcome for the same intended population.
- Reproduce both normal scores.
- Verify the combined statistic.
- Change the weights.
- Add arm selection.
- Simulate null calibration.
### Can stage two be larger? Yes, under a valid prespecified adaptive framework.
### Can weights be chosen after seeing p-values? Generally no.
### Does the test estimate effect size? No.
### May endpoints change freely? No.
Report the common hypothesis, stage definitions and cutoffs, p-value construction, weights, combination rule, stopping and adaptation, multiplicity, simulations, stagewise and combined results, and compatible estimation.
Under a true null, valid independent one-sided p-values transform into independent standard-normal scores. A weighted sum with squared weights summing to one is again standard normal.
Confirm every stagewise p-value is valid for the same claim, no observation is counted twice, adaptations respect conditional error and the published account makes selection visible.
Continue through the Mathematics Learning Hub. Recalculate the central worked example, name the assumptions that make it valid and explain which conclusion the result can and cannot support.
Extended practice laboratory
Practice case 1: Define stagewise hypotheses
Setup: Start from the article’s inverse-normal combination test mechanism for case 1. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 2: Set stage-one p-value
Setup: Start from the article’s inverse-normal combination test mechanism for case 2. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 3: Set stage-two p-value
Setup: Start from the article’s inverse-normal combination test mechanism for case 3. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 4: Choose weights
Setup: Start from the article’s inverse-normal combination test mechanism for case 4. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 5: Transform first p-value
Setup: Start from the article’s inverse-normal combination test mechanism for case 5. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 6: Transform second p-value
Setup: Start from the article’s inverse-normal combination test mechanism for case 6. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 7: Calculate combined statistic
Setup: Start from the article’s inverse-normal combination test mechanism for case 7. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 8: Calculate combined p-value
Setup: Start from the article’s inverse-normal combination test mechanism for case 8. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 9: Check independence
Setup: Start from the article’s inverse-normal combination test mechanism for case 9. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 10: Change weights
Setup: Start from the article’s inverse-normal combination test mechanism for case 10. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 11: Add early efficacy rule
Setup: Start from the article’s inverse-normal combination test mechanism for case 11. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 12: Add futility rule
Setup: Start from the article’s inverse-normal combination test mechanism for case 12. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 13: Increase stage two
Setup: Start from the article’s inverse-normal combination test mechanism for case 13. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 14: Select one arm
Setup: Start from the article’s inverse-normal combination test mechanism for case 14. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 15: Drop one dose
Setup: Start from the article’s inverse-normal combination test mechanism for case 15. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 16: Change endpoint
Setup: Start from the article’s inverse-normal combination test mechanism for case 16. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 17: Preserve estimand
Setup: Start from the article’s inverse-normal combination test mechanism for case 17. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 18: Apply conditional error
Setup: Start from the article’s inverse-normal combination test mechanism for case 18. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 19: Add multiplicity
Setup: Start from the article’s inverse-normal combination test mechanism for case 19. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 20: Add missingness
Setup: Start from the article’s inverse-normal combination test mechanism for case 20. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 21: Simulate null error
Setup: Start from the article’s inverse-normal combination test mechanism for case 21. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 22: Simulate power
Setup: Start from the article’s inverse-normal combination test mechanism for case 22. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 23: Audit stage separation
Setup: Start from the article’s inverse-normal combination test mechanism for case 23. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 24: Name limits
Setup: Start from the article’s inverse-normal combination test mechanism for case 24. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 25: Write conclusion
Setup: Start from the article’s inverse-normal combination test mechanism for case 25. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the estimand and design, verify the statistical identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.