Choose your reading route
eduKateSG · Why Mathematics? · Stacking Predictive Distributions for Reliability Models: LOO Densities, Simplex Weights, Log Scores and M-Open Uncertainty
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 6–10
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 11–15
Check assumptions
See where design and calibration matter.Route 4 · Chapters 16–20
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 21–25
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why mathematics is important becomes practical when several reliability models each capture part of the failure process and choosing one wastes useful predictive information. Stacking of predictive distributions chooses nonnegative weights that sum to one, maximising the leave-one-out log score of the combined predictive density. It targets prediction even when none of the candidate models is literally true, the M-open setting.
Suppose stacking assigns weights 0.65, 0.25 and 0.10 to three models. For one held-out reliability record, their predictive densities are 0.004, 0.006 and 0.002. The mixture density is 0.65×0.004+0.25×0.006+0.10×0.002=0.0043, with log score ln(0.0043)≈-5.449. This guide develops pointwise LOO densities, simplex optimisation, censored outcomes, diagnostics, weight stability and deployment limits.
Full chapter index
Chapters 1–5 · Understand the question
Chapters 6–10 · Reproduce the example
Chapters 11–15 · Check assumptions
Chapters 16–20 · Explore applications
Chapters 21–25 · Study and verify
Start from one held-out prediction. Different models can represent complementary features of a reliability process. A Weibull model captures monotone ageing while a lognormal model captures multiplicative variability. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Combining weak candidates does not create missing physics. Define a credible candidate set.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Keep the pointwise LOO matrix aligned by observation identifier before optimising any weight. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use why combine predictive distributions as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Did You Know? A model with weaker individual ELPD can still receive positive stacking weight when it predicts cases missed by stronger models. That is why a careful report includes the construction, not only the final scalar.
Now combine distributions on the simplex. In an M-open setting, the data-generating process is not assumed to be one of the fitted candidates. Every lifetime family is an approximation to field behaviour. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Posterior model probabilities from a closed set answer a different question. State the model-world assumption.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. The mixture belongs inside the logarithm, and all weights must remain on the simplex. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use m-open model uncertainty as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
The predictive unit controls the meaning. For each unit i and model k, estimate p(y_i|y_-i,M_k). Each model predicts a component excluded from its own fit. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Using full-data fitted densities produces optimism. Use PSIS-LOO diagnostics or exact refits.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. A zero weight describes redundancy in a candidate set rather than scientific falsehood. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use leave-one-out predictive densities as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Start from one held-out prediction. A held-out failure is scored by density and a held-out censored record by the probability of its observed censoring event under the defined likelihood. Survival to 500 hours contributes S(500). This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Scoring censoring incorrectly changes weights. Match the observation process.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. PSIS diagnostics and the choice of leave-out unit precede optimisation. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use censored predictive contributions as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Primary research: https://doi.org/10.1214/17-BA1091. Read the authors’ definition and conditions before carrying the formula into a censored reliability model.
Now combine distributions on the simplex. Stacking weights are nonnegative and sum to one. Weights 0.65, 0.25 and 0.10 satisfy the simplex constraints. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Unconstrained coefficients can create negative predictive densities. Verify bounds and the sum numerically.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Validate the combined distribution on later data and with more than one operational metric. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use weights on a simplex as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
The predictive unit controls the meaning. Choose weights to maximise the sum over observations of the log of the weighted predictive density. The optimiser rewards combinations assigning high density to held-out records. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Averaging log scores after model weighting is a different operation. Write the mixture inside the logarithm.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Keep the pointwise LOO matrix aligned by observation identifier before optimising any weight. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use log-score objective as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Did You Know? A model with weaker individual ELPD can still receive positive stacking weight when it predicts cases missed by stronger models. That is why a careful report includes the construction, not only the final scalar.
Start from one held-out prediction. Multiply each density by its weight and add. 0.65×0.004+0.25×0.006+0.10×0.002=0.0043. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Rounding weights early changes small densities. Keep full optimisation precision.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. The mixture belongs inside the logarithm, and all weights must remain on the simplex. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use one-record mixture 0.0043 as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Now combine distributions on the simplex. Take the natural logarithm of the positive mixture density. ln(0.0043)≈-5.44914. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. A larger, less negative log score is better. Label the density unit and log base.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. A zero weight describes redundancy in a candidate set rather than scientific falsehood. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use log score -5.449 as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
| Model | Weight | Held-out density |
|---|---|---|
| A | 0.65 | 0.004 |
| B | 0.25 | 0.006 |
| C | 0.10 | 0.002 |
| Mixture | 0.0043 | |
| Natural log score | -5.449 |
The predictive unit controls the meaning. The optimisation uses an n-by-K matrix of LOO predictive densities or log densities. Rows are machines and columns are candidate models. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Misaligned row order silently corrupts the mixture. Join by stable observation identifiers.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. PSIS diagnostics and the choice of leave-out unit precede optimisation. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use pointwise matrix as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Start from one held-out prediction. Log-sum-exp calculations avoid underflow for tiny predictive densities. Long-lifetime tail densities may be below ordinary floating-point range. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. A printed zero makes the log score negative infinity. Optimise in the log domain.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Validate the combined distribution on later data and with more than one operational metric. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use numerical stabilisation as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Now combine distributions on the simplex. A numerical optimiser searches the simplex for the best held-out log score. Several starts reach the same weights. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Local or boundary solutions need checking. Use multiple starts and verify the objective.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Keep the pointwise LOO matrix aligned by observation identifier before optimising any weight. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use optimisation and constraints as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Did You Know? A model with weaker individual ELPD can still receive positive stacking weight when it predicts cases missed by stronger models. That is why a careful report includes the construction, not only the final scalar.
The predictive unit controls the meaning. A model can receive zero weight when it adds no predictive value beyond the others. Two nearly identical Weibull variants make one redundant. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Zero is not proof the model is scientifically false. Interpret weights as combination coefficients.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. The mixture belongs inside the logarithm, and all weights must remain on the simplex. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use zero weights as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Second authoritative source: https://mc-stan.org/loo/reference/loo_model_weights.html. Compare its prediction target, diagnostics and finite-sample cautions with this worked example.
Start from one held-out prediction. A model with weaker individual ELPD can receive positive weight if it predicts observations missed by stronger models. A shock model helps on rare early failures. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Ranking models separately misses complementarity. Inspect pointwise contributions.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. A zero weight describes redundancy in a candidate set rather than scientific falsehood. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use complementary models as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Now combine distributions on the simplex. Nearly identical predictive distributions can yield unstable individual weights but a stable combined prediction. Two parameterisations predict almost the same survival curve. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Overinterpreting which duplicate receives weight is misleading. Report prediction stability and merge duplicates if justified.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. PSIS diagnostics and the choice of leave-out unit precede optimisation. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use collinearity and non-unique weights as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
The predictive unit controls the meaning. Stacking based on PSIS-LOO inherits Pareto-k and Monte Carlo diagnostics. One influential failure has a high k value. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Optimisation cannot repair an unreliable LOO approximation. Refit, moment-match or use K-fold.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Validate the combined distribution on later data and with more than one operational metric. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use psis diagnostics as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Start from one held-out prediction. Repeated rows from one machine should often be left out together. A degradation trajectory contains ten inspections. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Row-level LOO leaks machine information. Stack at the deployment unit.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Keep the pointwise LOO matrix aligned by observation identifier before optimising any weight. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use grouped predictive units as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Did You Know? A model with weaker individual ELPD can still receive positive stacking weight when it predicts cases missed by stronger models. That is why a careful report includes the construction, not only the final scalar.
Now combine distributions on the simplex. Future prediction may require forward or rolling validation rather than exchangeable LOO. Train on earlier batches and score later batches. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Random leave-one-out can ignore drift. Match resampling to deployment.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. The mixture belongs inside the logarithm, and all weights must remain on the simplex. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use time-ordered validation as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
The predictive unit controls the meaning. Finite data make estimated weights variable. Bootstrap or repeated folds produce a range around 0.65. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Treating weights as exact hides selection uncertainty. Report sensitivity and predictive intervals.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. A zero weight describes redundancy in a candidate set rather than scientific falsehood. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use weight uncertainty as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Start from one held-out prediction. Weights change when a useful or redundant model enters the set. Adding a degradation-informed model absorbs weight from generic families. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Weights are not intrinsic model properties. Record the candidate list.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. PSIS diagnostics and the choice of leave-out unit precede optimisation. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use candidate-set dependence as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Now combine distributions on the simplex. BMA weights models using posterior model probabilities based on marginal likelihood; stacking optimises predictive score. The two methods can give different weights. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Neither should be renamed as the other. Choose from the inferential objective.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Validate the combined distribution on later data and with more than one operational metric. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use stacking versus bma as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
The predictive unit controls the meaning. Pseudo-BMA+ transforms ELPD estimates into bootstrap-stabilised weights; stacking directly optimises the combined density. Pseudo-BMA+ is computationally simpler after pointwise LOO estimates. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Approximate weights may lose complementarity. Compare combined log score.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Keep the pointwise LOO matrix aligned by observation identifier before optimising any weight. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use stacking versus pseudo-bma+ as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Did You Know? A model with weaker individual ELPD can still receive positive stacking weight when it predicts cases missed by stronger models. That is why a careful report includes the construction, not only the final scalar.
Start from one held-out prediction. A higher stacked log score does not ensure every reliability metric improves. Mixture prediction ranks well but is slightly miscalibrated at 1,000 hours. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. One score cannot represent all deployment consequences. Check calibration, discrimination and decision value.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. The mixture belongs inside the logarithm, and all weights must remain on the simplex. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use calibration and discrimination as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Now combine distributions on the simplex. Apply frozen component models and weights to later fleets. Weights from laboratory tests shift in field data. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Development stacking does not prove transportability. Revalidate and document update rules.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. A zero weight describes redundancy in a candidate set rather than scientific falsehood. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use external validation as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
The predictive unit controls the meaning. Stacking joins weighted averages, constraints, logarithms and optimisation. Students can verify the weights sum to one and mixture equals 0.0043. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. Prediction meaning matters more than arithmetic. Draw model densities and their mixture.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. PSIS diagnostics and the choice of leave-out unit precede optimisation. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use parent and student learning route as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Useful next reading: approach to learning mathematics, Bukit Timah Mathematics curriculum overview, cross-subject learning approach, nested likelihood-ratio guide, Mathematics Learning Hub, eduKate Mathematics Learning System. Choose the destination that matches the next question rather than treating links as a checklist.
Start from one held-out prediction. Stacking weights are not probabilities that models are true and do not guarantee causal validity. A weight of 0.65 describes a predictive combination. This step matters because predictive combinations, log scores and open-model uncertainty require the mathematical object, prediction target and reliability question to stay aligned. More models are not automatically better. Read Yao and colleagues and current loo documentation.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Validate the combined distribution on later data and with more than one operational metric. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: verify simplex weights, calculate the 0.0043 mixture density and its log score, then change one component density before predicting the direction. Use common misconceptions and next reading as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Quick answers
What must be fixed before comparing models? Use the same observations, censoring mechanism, time scale, likelihood convention and predictive unit.
Does a preferred value prove a model is true? No. The comparison is conditional on its candidates, data and assumptions.
Which mathematics matters? Algebra organises identities, logarithms stabilise products, probability represents uncertainty, variance measures instability and graphs expose influential cases.
How should a student practise? Preserve intermediate quantities, predict directions before calculating and change one input at a time.
What follows the comparison? Diagnostics, uncertainty analysis, posterior or predictive checks, later-data validation and a separate decision analysis when costs or safety matter.
Extended practice laboratory
Practice case 1: Why combine predictive distributions
Setup: Use this chapter’s illustrative case: A Weibull model captures monotone ageing while a lognormal model captures multiplicative variability.
Task: Define a credible candidate set.
Checking target: Combining weak candidates does not create missing physics.
Practice case 2: M-open model uncertainty
Setup: Use this chapter’s illustrative case: Every lifetime family is an approximation to field behaviour.
Task: State the model-world assumption.
Checking target: Posterior model probabilities from a closed set answer a different question.
Practice case 3: Leave-one-out predictive densities
Setup: Use this chapter’s illustrative case: Each model predicts a component excluded from its own fit.
Task: Use PSIS-LOO diagnostics or exact refits.
Checking target: Using full-data fitted densities produces optimism.
Practice case 4: Censored predictive contributions
Setup: Use this chapter’s illustrative case: Survival to 500 hours contributes S(500).
Task: Match the observation process.
Checking target: Scoring censoring incorrectly changes weights.
Practice case 5: Weights on a simplex
Setup: Use this chapter’s illustrative case: Weights 0.65, 0.25 and 0.10 satisfy the simplex constraints.
Task: Verify bounds and the sum numerically.
Checking target: Unconstrained coefficients can create negative predictive densities.
Practice case 6: Log-score objective
Setup: Use this chapter’s illustrative case: The optimiser rewards combinations assigning high density to held-out records.
Task: Write the mixture inside the logarithm.
Checking target: Averaging log scores after model weighting is a different operation.
Practice case 7: One-record mixture 0.0043
Setup: Use this chapter’s illustrative case: 0.65×0.004+0.25×0.006+0.10×0.002=0.0043.
Task: Keep full optimisation precision.
Checking target: Rounding weights early changes small densities.
Practice case 8: Log score -5.449
Setup: Use this chapter’s illustrative case: ln(0.0043)≈-5.44914.
Task: Label the density unit and log base.
Checking target: A larger, less negative log score is better.
Practice case 9: Pointwise matrix
Setup: Use this chapter’s illustrative case: Rows are machines and columns are candidate models.
Task: Join by stable observation identifiers.
Checking target: Misaligned row order silently corrupts the mixture.
Practice case 10: Numerical stabilisation
Setup: Use this chapter’s illustrative case: Long-lifetime tail densities may be below ordinary floating-point range.
Task: Optimise in the log domain.
Checking target: A printed zero makes the log score negative infinity.