Choose your reading route
eduKateSG · Why Mathematics? · Infinitesimal Jackknife: Weight Derivatives and Bagged Prediction Variance
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 8–14
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 15–21
Check assumptions
See where design and calibration matter.Route 4 · Chapters 22–28
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 29–34
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why is mathematics important when deleting a full observation is a coarse way to measure influence? The infinitesimal jackknife asks how an estimator changes under an tiny change in each observation’s weight. For bagged predictors, covariances between bootstrap inclusion counts and predictions turn those derivatives into a variance estimate.
Did You Know? For a bagged prediction, the infinitesimal-jackknife variance can be estimated by summing squared covariances between each training case’s bootstrap count and the replicate prediction.
Full chapter index
Chapters 1–7 · Understand the question
Chapters 8–14 · Reproduce the example
Chapters 15–21 · Check assumptions
Chapters 22–28 · Explore applications
Chapters 29–34 · Study and verify
Ordinary jackknife compares weight one with weight zero. The infinitesimal jackknife studies the derivative at the observed empirical weights, giving a first-order local sensitivity measure.
Wager, Hastie and Efron developed jackknife and infinitesimal-jackknife standard errors for bagged learners and random forests, including finite-B corrections. Read the JMLR paper.
Let N_i-star be the number of appearances of training case i and t-star(x) the replicate prediction. Estimate C_i=Cov-star(N_i-star,t-star(x)).
Suppose four estimated covariances are (0.10,−0.05,0.20,−0.15). Squaring removes direction only after the influence covariance is estimated.
| Covariances |
|---|
| 0.10, -0.05, 0.20, -0.15 |
The infinite-bootstrap IJ variance estimate is 0.10²+(−0.05)²+0.20²+(−0.15)²=0.075. Its standard error estimate is sqrt(0.075)≈0.274.
| Squared sum | Standard error |
|---|---|
| 0.075 | 0.274 |
If including case i more often systematically raises a prediction, its count and prediction covary positively. Negative covariance indicates downward influence.
C_i approximates the effect of infinitesimally increasing empirical mass on case i. Summing squared effects estimates first-order sampling variance.
The theoretical target assumes the bagging average over infinitely many bootstrap replicates. A finite forest adds Monte Carlo noise and bias to covariance estimates.
Finite-B corrections subtract estimated simulation components. Reporting an uncorrected IJ from a small B can exaggerate variance.
The cited work analyses how many replicates are needed and proposes improved estimators. A large forest for prediction may still be too small for stable uncertainty estimation.
IJ variance depends on the query point x. A prediction near sparse or boundary regions can have larger training-sample sensitivity.
The estimate concerns how the fitted predictor changes with a new training sample. It does not automatically include irreducible outcome noise for a future observation.
Forests built with subsamples rather than bootstrap samples need inclusion indicators and formulas matched to that scheme. Count covariance cannot be copied unchanged.
For classification probabilities, bounded predictions can have skewed sampling distributions near zero or one. Normal intervals may leave the valid range.
Plot C_i against training features or residuals. Large positive and negative influences can reveal sparse regions, data errors or genuinely informative cases.
Delete-one refits after removing each case. IJ estimates local weight derivatives, often reusing one ensemble’s replicate counts.
Both reuse bootstrap information. JAB uses replicate subsets omitting a case; IJ uses covariance between replicate counts and outputs.
Permutation importance measures predictive reliance on a feature. IJ influence measures training-sample sensitivity of a particular prediction; the questions differ.
Track variance estimate against B, apply finite-B correction, inspect covariance stability, compare with repeated training datasets and map influential cases.
Simulate data with known prediction variance, vary n and B, compare IJ, jackknife and empirical retraining variance, and test boundary points.
IJ relies on first-order approximations, stable bagging, adequate B and correct inclusion records. It does not establish model accuracy, fairness or causal validity.
Store one row per case with mean count, count-prediction covariance, squared covariance and metadata. Sum squares and chart convergence against B.
Save every replicate prediction and inclusion count, unit-test squared sum 0.075 and square root 0.274, then apply the documented finite-B correction.
IJ methods matter in machine learning uncertainty, bagging and influence diagnostics. Careers require statistics, software engineering and responsible model evaluation.
Simulate regression data, fit forests at increasing B, compare IJ standard errors with repeated-dataset predictions and inspect influential training cases.
IJ uncertainty is not prediction error, negative covariance is not a negative variance contribution, and a large forest does not guarantee stable covariance estimates.
CHAPTER 27 OF 34 · Explore applications
27. Connection to recent mathematics articles
The jackknife-after-bootstrap guide explains omission subsets. The multiplier-bootstrap guide explains random weighted influence sums.
Students should calculate four squared covariances and separate sampling variance from prediction noise. Parents can ask what uncertainty the error bar actually represents.
- Square four covariances.
- Sum to 0.075.
- Calculate 0.274.
- Explain covariance sign.
- Separate uncertainty targets.
### Is each model refitted n times? Not necessarily.
### What data are needed? Replicate predictions and inclusion counts.
### Does IJ estimate test error? No.
### Why correct finite B? Monte Carlo noise biases covariance estimates.
Report learner, resampling scheme, n, B, query points, inclusion records, covariance formula, finite-B correction, convergence, influential cases and validation against retraining.
Influence direction lives in the covariance sign; variance adds squared effects. Squaring too early or discarding count information destroys the derivative interpretation.
Confirm resampling-compatible counts, adequate B, Monte Carlo correction, query-specific reporting, first-order validity and no confusion between model sampling variance and total predictive uncertainty.
Continue through the Mathematics Learning Hub. Recalculate one deletion, pseudovalue or infinitesimal-weight example by hand before trusting software output.
Extended practice laboratory
Practice case 1: Save bootstrap counts
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 1. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 2: Save predictions
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 2. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 3: Calculate mean counts
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 3. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 4: Calculate mean prediction
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 4. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 5: Calculate covariance 1
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 5. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 6: Calculate all covariances
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 6. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 7: Square covariances
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 7. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 8: Sum variance
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 8. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 9: Take square root
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 9. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 10: Increase B
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 10. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 11: Apply finite-B correction
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 11. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 12: Plot convergence
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 12. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 13: Change query point
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 13. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 14: Use boundary point
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 14. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 15: Use regression forest
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 15. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 16: Use classifier
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 16. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 17: Use subsampling forest
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 17. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 18: Map influence
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 18. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 19: Compare delete-one
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 19. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 20: Compare JAB
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 20. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 21: Compare retraining
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 21. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 22: Separate noise variance
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 22. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 23: Audit count scheme
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 23. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 24: Name limits
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 24. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 25: Write conclusion
Setup: Start from the article’s infinitesimal-jackknife mechanism for case 25. Task: Change the named quantity, recompute the dependent result and compare with the original.
Check: Preserve the independent unit, verify the deletion or derivative identity, state one diagnostic, record units and compare one defensible alternative with clear reasoning. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.