VIEW THIS AS

Auto mode follows the Route Engine until you choose a viewpoint.

YOU ARE HERE

ROUTE CHECK

CONNECTED TO

WHAT NEXT

Use the canonical route for this room, or HELP if you are unsure.

Why Mathematics? | Stratified Bootstrap: Within-Stratum Resampling and Fixed Composition

Why is mathematics important when a sample deliberately contains known subgroups whose representation should remain stable? The stratified bootstrap resamples independently within each stratum. It preserves the observed stratum counts while allowing cases inside each group to vary, aligning the resampling law with a stratified sampling or analysis design.

Did You Know? If a sample contains 60 students from stratum A and 40 from stratum B, each bootstrap replicate can retain exactly 60 and 40 draws. Ordinary unrestricted resampling would make those counts fluctuate even when the design fixed them.

Full chapter index

Chapters 1–7 · Understand the question
  1. The composition question
  2. The official implementation basis
  3. The within-stratum rule
  4. A worked allocation
  5. A worked overall mean
  6. Fixed counts versus population weights
  7. Why unrestricted resampling differs
Chapters 8–14 · Reproduce the example
  1. Proportional allocation
  2. Treatment arms
  3. Rare strata
  4. Numeric stratification
  5. Variance decomposition
  6. A weighted mean example
  7. A classification example
Chapters 15–21 · Check assumptions
  1. A school example
  2. Post-stratification distinction
  3. Comparison with cluster bootstrap
  4. Comparison with balanced bootstrap
  5. Comparison with class-balanced machine learning
  6. Validation plan
  7. Limits
Chapters 22–28 · Explore applications
  1. A spreadsheet workflow
  2. A coding workflow
  3. Applications and careers
  4. A student investigation
  5. Common misconceptions
  6. Connection to recent mathematics articles
  7. Guidance for students and parents
Chapters 29–34 · Study and verify
  1. Practice questions
  2. Frequently asked questions
  3. A reporting checklist
  4. Composition arithmetic
  5. Final stratified-bootstrap audit
  6. Useful next reading

CHAPTER 1 OF 34 · Understand the question

1. The composition question

Back to contents

Age bands, treatment arms, schools or sampling strata may be fixed by design. Allowing bootstrap composition to wander can add variation that the original design did not generate.

Contents · Next chapter

CHAPTER 2 OF 34 · Understand the question

2. The official implementation basis

Back to contents

R boot states that nonparametric simulations are performed independently within supplied strata. Read the official R documentation.

Contents · Previous chapter · Next chapter

CHAPTER 3 OF 34 · Understand the question

3. The within-stratum rule

Back to contents

For stratum h with n_h observations, draw n_h cases with replacement from that stratum. Concatenate all strata so the replicate total remains n=sum n_h.

Contents · Previous chapter · Next chapter

CHAPTER 4 OF 34 · Understand the question

4. A worked allocation

Back to contents

Suppose A has observations (2,4,6) and B has (10,14). One valid replicate draws A indices (3,3,1) giving (6,6,2), and B indices (2,1) giving (14,10).

A drawB drawCounts
6,6,214,103 and 2
Within-stratum replicate.

Contents · Previous chapter · Next chapter

CHAPTER 5 OF 34 · Understand the question

5. A worked overall mean

Back to contents

The replicate values sum to 6+6+2+14+10=38 across five observations, so the unweighted sample mean is 38/5=7.6.

ValuesSumMean
6,6,2,14,10387.6
Replicate mean.

Contents · Previous chapter · Next chapter

CHAPTER 6 OF 34 · Understand the question

6. Fixed counts versus population weights

Back to contents

Preserving sample counts does not decide how strata represent the target population. A survey estimate may require external population weights even though resampling remains within sample strata.

Contents · Previous chapter · Next chapter

CHAPTER 7 OF 34 · Understand the question

7. Why unrestricted resampling differs

Back to contents

An unrestricted bootstrap treats all rows as exchangeable and lets stratum counts fluctuate multinomially. That can be correct for a simple random sample but wrong for fixed-allocation designs.

Contents · Previous chapter · Next chapter

CHAPTER 8 OF 34 · Reproduce the example

8. Proportional allocation

Back to contents

When sample stratum shares mirror population shares, the overall statistic may use ordinary row weights. Disproportionate allocation needs explicit design weights for population inference.

Contents · Previous chapter · Next chapter

CHAPTER 9 OF 34 · Reproduce the example

9. Treatment arms

Back to contents

Randomised trials often stratify or block randomisation. Resampling within treatment-by-block cells can preserve allocation, but causal inference may also call for randomisation-based methods.

Contents · Previous chapter · Next chapter

CHAPTER 10 OF 34 · Reproduce the example

10. Rare strata

Back to contents

A stratum with one observed unit produces the same unit in every replicate and contributes no within-stratum empirical uncertainty. Pooling strata changes the estimand and must be justified.

Contents · Previous chapter · Next chapter

CHAPTER 11 OF 34 · Reproduce the example

11. Numeric stratification

Back to contents

Software may bin a numeric variable into quantiles. Bin cutpoints, sparse groups and whether cutpoints are fixed or re-estimated all affect the procedure.

Contents · Previous chapter · Next chapter

CHAPTER 12 OF 34 · Reproduce the example

12. Variance decomposition

Back to contents

Total uncertainty combines within-stratum variability under fixed allocation and, when relevant, uncertainty in external stratum weights. A simple stratified bootstrap may estimate only the first component.

Contents · Previous chapter · Next chapter

CHAPTER 13 OF 34 · Reproduce the example

13. A weighted mean example

Back to contents

If population shares are 0.7 for A and 0.3 for B, calculate replicate stratum means first, then 0.7 mean_A-star + 0.3 mean_B-star. Do not substitute sample proportions unless they represent the target.

Contents · Previous chapter · Next chapter

CHAPTER 14 OF 34 · Reproduce the example

14. A classification example

Back to contents

Stratifying bootstrap splits by outcome class can stabilise rare-class model training, but performance evaluation must reflect the intended population prevalence.

Contents · Previous chapter · Next chapter

CHAPTER 15 OF 34 · Check assumptions

15. A school example

Back to contents

If schools sample fixed numbers by grade, grade-stratified resampling preserves those allocations. Dependence within classroom may require another cluster level.

Contents · Previous chapter · Next chapter

CHAPTER 16 OF 34 · Check assumptions

16. Post-stratification distinction

Back to contents

Design strata are used during sampling. Post-strata are formed after sampling for adjustment. Their resampling treatment depends on how weights and cells were estimated.

Contents · Previous chapter · Next chapter

CHAPTER 17 OF 34 · Check assumptions

17. Comparison with cluster bootstrap

Back to contents

Stratified bootstrap resamples cases independently inside strata. Cluster bootstrap resamples whole groups. A classroom can be a cluster while grade is a stratum, so both structures may coexist.

Contents · Previous chapter · Next chapter

CHAPTER 18 OF 34 · Check assumptions

18. Comparison with balanced bootstrap

Back to contents

Balanced bootstrap coordinates inclusion counts across the whole set of replicates. Stratification constrains each replicate within known groups; the invariants differ.

Contents · Previous chapter · Next chapter

CHAPTER 19 OF 34 · Check assumptions

19. Comparison with class-balanced machine learning

Back to contents

Oversampling classes for model fitting changes the training distribution. A stratified inferential bootstrap normally preserves the observed design allocation and separately handles target prevalence.

Contents · Previous chapter · Next chapter

CHAPTER 20 OF 34 · Check assumptions

20. Validation plan

Back to contents

Check stratum labels and counts, verify every replicate allocation, reproduce weighted and unweighted statistics, inspect sparse cells and simulate the actual sampling design.

Contents · Previous chapter · Next chapter

CHAPTER 21 OF 34 · Check assumptions

21. Limits

Back to contents

Stratification cannot recover unobserved groups, fix wrong population weights, preserve within-cluster dependence by itself or justify arbitrary outcome balancing.

Contents · Previous chapter · Next chapter

CHAPTER 22 OF 34 · Explore applications

22. A spreadsheet workflow

Back to contents

Use one table per stratum with original and replicate indices. Confirm count invariants, calculate stratum estimates and combine them with clearly labelled sample or population weights.

Contents · Previous chapter · Next chapter

CHAPTER 23 OF 34 · Explore applications

23. A coding workflow

Back to contents

Split indices by stratum, sample n_h within each group and unit-test A draw (6,6,2), B draw (14,10), total mean 7.6. Save cell counts and seeds.

Contents · Previous chapter · Next chapter

CHAPTER 24 OF 34 · Explore applications

24. Applications and careers

Back to contents

Stratified resampling appears in surveys, trials, education, ecology and machine learning. Careers require sampling theory, weighting and domain-aware subgroup definitions.

Contents · Previous chapter · Next chapter

CHAPTER 25 OF 34 · Explore applications

25. A student investigation

Back to contents

Simulate two strata with unequal variances and disproportionate sampling. Compare unrestricted, stratified unweighted and stratified population-weighted intervals.

Contents · Previous chapter · Next chapter

CHAPTER 26 OF 34 · Explore applications

26. Common misconceptions

Back to contents

A stratum is not automatically a cluster, fixed sample shares are not always population shares, and outcome balancing can change the target distribution.

Contents · Previous chapter · Next chapter

CHAPTER 27 OF 34 · Explore applications

27. Connection to recent mathematics articles

Back to contents

The cluster-bootstrap guide explains whole-group resampling. The survey-sampling guide explains population weights and margins of error.

Contents · Previous chapter · Next chapter

CHAPTER 28 OF 34 · Explore applications

28. Guidance for students and parents

Back to contents

Students should resample two small strata by hand and verify the counts. Parents can ask whether group proportions were fixed by design or estimated from the sample.

Contents · Previous chapter · Next chapter

CHAPTER 29 OF 34 · Study and verify

29. Practice questions

Back to contents

  • Recreate both stratum draws.
  • Calculate 38/5.
  • Apply external population shares.
  • Diagnose a one-unit stratum.
  • Compare strata and clusters.

Contents · Previous chapter · Next chapter

CHAPTER 30 OF 34 · Study and verify

30. Frequently asked questions

Back to contents

### Where are cases resampled? Within their strata.

### Are stratum counts preserved? Usually yes.

### Are sample shares always population shares? No.

### Does stratification preserve clusters? Not by itself.

Contents · Previous chapter · Next chapter

CHAPTER 31 OF 34 · Study and verify

31. A reporting checklist

Back to contents

Report strata, n_h, allocation origin, population weights, B, seed, sparse-cell rule, estimator, count checks, design simulation and any nested clusters.

Contents · Previous chapter · Next chapter

CHAPTER 32 OF 34 · Study and verify

32. Composition arithmetic

Back to contents

The replicate total is sum n_h because each stratum contributes its original sample count. This removes random composition variation while preserving within-cell empirical variation.

Contents · Previous chapter · Next chapter

CHAPTER 33 OF 34 · Study and verify

33. Final stratified-bootstrap audit

Back to contents

Confirm fixed allocations, correct within-cell exchangeability, explicit population weights, sparse-stratum treatment, nested cluster handling and a target estimand consistent with the design.

Contents · Previous chapter · Next chapter

CHAPTER 34 OF 34 · Study and verify

34. Useful next reading

Back to contents

Continue through the Mathematics Learning Hub. Reproduce one cluster, stratum, autoregressive or spatial-block calculation by hand before trusting a large automated run.

Contents · Previous chapter · Mathematics Learning Hub

Extended practice laboratory

Practice case 1: Define strata

Setup: Start from the article’s stratified-bootstrap mechanism for case 1. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 2: Count each stratum

Setup: Start from the article’s stratified-bootstrap mechanism for case 2. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 3: Draw within A

Setup: Start from the article’s stratified-bootstrap mechanism for case 3. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 4: Draw within B

Setup: Start from the article’s stratified-bootstrap mechanism for case 4. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 5: Verify counts

Setup: Start from the article’s stratified-bootstrap mechanism for case 5. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 6: Calculate overall mean

Setup: Start from the article’s stratified-bootstrap mechanism for case 6. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 7: Calculate stratum means

Setup: Start from the article’s stratified-bootstrap mechanism for case 7. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 8: Apply population weights

Setup: Start from the article’s stratified-bootstrap mechanism for case 8. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 9: Change allocation

Setup: Start from the article’s stratified-bootstrap mechanism for case 9. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 10: Use treatment arms

Setup: Start from the article’s stratified-bootstrap mechanism for case 10. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 11: Add rare stratum

Setup: Start from the article’s stratified-bootstrap mechanism for case 11. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 12: Pool sparse cells

Setup: Start from the article’s stratified-bootstrap mechanism for case 12. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 13: Bin numeric variable

Setup: Start from the article’s stratified-bootstrap mechanism for case 13. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 14: Add cluster inside stratum

Setup: Start from the article’s stratified-bootstrap mechanism for case 14. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 15: Compare unrestricted bootstrap

Setup: Start from the article’s stratified-bootstrap mechanism for case 15. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 16: Compare cluster bootstrap

Setup: Start from the article’s stratified-bootstrap mechanism for case 16. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 17: Compare balanced bootstrap

Setup: Start from the article’s stratified-bootstrap mechanism for case 17. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 18: Use classification data

Setup: Start from the article’s stratified-bootstrap mechanism for case 18. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 19: Increase B

Setup: Start from the article’s stratified-bootstrap mechanism for case 19. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 20: Change seed

Setup: Start from the article’s stratified-bootstrap mechanism for case 20. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 21: Audit weights

Setup: Start from the article’s stratified-bootstrap mechanism for case 21. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 22: Audit counts

Setup: Start from the article’s stratified-bootstrap mechanism for case 22. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 23: Validate design

Setup: Start from the article’s stratified-bootstrap mechanism for case 23. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 24: Name limits

Setup: Start from the article’s stratified-bootstrap mechanism for case 24. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.

Practice case 25: Write conclusion

Setup: Start from the article’s stratified-bootstrap mechanism for case 25. Task: Change the named quantity, recompute the dependent result and compare with the original.

Check: Preserve the sampling unit, verify the grouping, recursion or geometry identity, and state one diagnostic before interpretation. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.