Choose your reading route
eduKateSG · Why Mathematics? · Annealed Importance Sampling for Bayesian Reliability Models: Temperature Paths, Transition Kernels, Importance Weights and Evidence Ratios
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 6–10
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 11–15
Check assumptions
See where design and calibration matter.Route 4 · Chapters 16–20
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 21–25
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why mathematics is important becomes visible when ordinary importance sampling cannot bridge a simple reference distribution and a complex reliability posterior. Annealed importance sampling, introduced by Radford Neal, moves a particle through a sequence of intermediate distributions. Markov transitions help it follow the changing target, while a product of incremental density ratios produces an importance weight that corrects the path.
In a teaching example, one run has incremental factors 1.2, 0.8 and 1.5, giving weight 1.44. Three independent runs produce weights 1.44, 0.96 and 1.20; their arithmetic mean is 1.20, so the illustrative log normalising-constant ratio is log(1.20)=0.18232. The log of the mean must be computed after averaging weights. This guide develops paths, kernels, weights, variance, multimodality, censoring and validation.
Full chapter index
Chapters 1–5 · Understand the question
Chapters 6–10 · Reproduce the example
Chapters 11–15 · Check assumptions
Chapters 16–20 · Explore applications
Chapters 21–25 · Study and verify
Start from the known reference distribution. Start from a distribution with a known normalising constant and end at the reliability posterior target. A proper prior serves as the reference and likelihood times prior as the target. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. An unknown or improper base constant prevents an absolute evidence estimate. State both endpoints.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Save every schedule, transition setting, seed and run-level log weight. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use reference and target distributions as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Did You Know? The arithmetic mean of AIS weights targets a normalising-constant ratio, while averaging log weights targets a different quantity. That is why a careful report includes the construction, not only the final scalar.
Now move along the annealing path. Create distributions f_beta proportional to prior times likelihood to beta. Beta increases from zero to one. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. A schedule that changes too quickly produces volatile weights. Record every beta.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Average ordinary weights before taking a logarithm, using log-sum-exp for stability. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use intermediate annealing path as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Importance weights correct each trajectory. Draw from the base, update the weight and apply transitions as the path advances. A particle moves through failure-rate parameter space. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. A trajectory is not an ordinary posterior chain. Keep path order explicit.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Verify that every transition kernel leaves its intermediate target invariant. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use one ais trajectory as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Start from the known reference distribution. Each factor compares adjacent unnormalised targets at the current state. Likelihood to a beta increment supplies a positive multiplier. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. Evaluating the factor after the wrong transition changes the algorithm. Follow the defined convention exactly.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Inspect extreme-weight concentration, ESS and modal coverage across independent runs. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use incremental weight factors as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Primary research: https://doi.org/10.1023/A:1008923215028. Read the authors’ definition and conditions before carrying the formula into a censored reliability model.
Now move along the annealing path. Multiply factors 1.2, 0.8 and 1.5. The AIS weight is 1.44. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. Adding the factors gives no importance ratio. Reproduce the product and its log.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Repeat with denser schedules and reverse or alternative estimators when practical. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use worked weight 1.44 as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
| Quantity | Value |
|---|---|
| Incremental factors | 1.2, 0.8, 1.5 |
| Run 1 weight | 1.44 |
| Run 2 weight | 0.96 |
| Run 3 weight | 1.20 |
| Mean weight | 1.20 |
| Log mean weight | 0.18232 |
Importance weights correct each trajectory. Average weights across independent AIS runs to estimate the normalising-constant ratio. Weights 1.44, 0.96 and 1.20 average to 1.20. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. A median weight targets a different quantity. Keep every run.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Save every schedule, transition setting, seed and run-level log weight. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use average of independent weights as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Did You Know? The arithmetic mean of AIS weights targets a normalising-constant ratio, while averaging log weights targets a different quantity. That is why a careful report includes the construction, not only the final scalar.
Start from the known reference distribution. Take log only after the arithmetic mean of ordinary weights. log(1.20) is about 0.18232. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. Averaging log weights and exponentiating generally differs by Jensen’s inequality. Use log-sum-exp for stability.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Average ordinary weights before taking a logarithm, using log-sum-exp for stability. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use worked log ratio 0.18232 as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Now move along the annealing path. At each stage, use a kernel that leaves the current intermediate distribution invariant. A Metropolis update targets the current power posterior. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. A convenient move with the wrong invariant density biases AIS. Verify detailed balance or invariance.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Verify that every transition kernel leaves its intermediate target invariant. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use markov transition kernels as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Importance weights correct each trajectory. AIS correctness does not require an exact independent draw at every stage when kernels are valid, though poor movement raises variance. Several short Metropolis steps follow each beta. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. Zero mixing can produce extreme weights. Balance path length and movement.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Inspect extreme-weight concentration, ESS and modal coverage across independent runs. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use transitions need not equilibrate fully as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Start from the known reference distribution. Dense schedules make adjacent targets more similar. Two hundred beta values replace twenty. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. More stages cost computation and do not cure a wrong kernel. Pilot several schedules.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Repeat with denser schedules and reverse or alternative estimators when practical. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use temperature schedule as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Now move along the annealing path. Incremental factors use the same failure-density and censoring-survival likelihood. A censored record contributes survival probability to every powered target. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. Changing censoring rules mid-path breaks the ratio. Unit-test observation contributions.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Save every schedule, transition setting, seed and run-level log weight. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use censored likelihood as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Did You Know? The arithmetic mean of AIS weights targets a normalising-constant ratio, while averaging log weights targets a different quantity. That is why a careful report includes the construction, not only the final scalar.
Importance weights correct each trajectory. Add incremental log factors to avoid overflow and underflow. A long path has a log weight of -730. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. Direct multiplication becomes zero in floating-point arithmetic. Store and centre log weights.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Average ordinary weights before taking a logarithm, using log-sum-exp for stability. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use log-weight accumulation as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Second authoritative source: https://doi.org/10.1023/A:1008923215028. Compare its prediction target, diagnostics and finite-sample cautions with this worked example.
Start from the known reference distribution. A few very large weights can dominate the estimator. One run contributes half the total estimated evidence. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. Many nominal runs can have little effective information. Plot weights and report concentration.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Verify that every transition kernel leaves its intermediate target invariant. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use weight variability as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Now move along the annealing path. Normalised AIS weights give an ESS summary. One over the squared-weight sum equals 18 from 1,000 runs. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. ESS does not detect every path or kernel error. Use it with repeated runs.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Inspect extreme-weight concentration, ESS and modal coverage across independent runs. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use importance effective sample size as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Importance weights correct each trajectory. Under the AIS construction, the average ordinary weight targets the ratio, while its logarithm has finite-sample bias. Repeated small ensembles underestimate log evidence on average. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. Calling log mean weight unbiased is incorrect. Separate ratio and log-scale properties.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Repeat with denser schedules and reverse or alternative estimators when practical. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use unbiased ratio and biased log as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Start from the known reference distribution. Annealing can help reach isolated modes when transitions move between them. Hot early stages explore two failure mechanisms. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. A rapid schedule can commit particles to one mode. Track modal coverage.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Save every schedule, transition setting, seed and run-level log weight. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use multimodal targets as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Did You Know? The arithmetic mean of AIS weights targets a normalising-constant ratio, while averaging log weights targets a different quantity. That is why a careful report includes the construction, not only the final scalar.
Now move along the annealing path. Running a reverse construction from posterior draws provides complementary information and bounds in some contexts. Forward and reverse log estimates disagree. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. The reverse run needs appropriate posterior starting draws. Use disagreement diagnostically.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Average ordinary weights before taking a logarithm, using log-sum-exp for stability. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use forward and reverse ais as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Importance weights correct each trajectory. Independent AIS trajectories can run concurrently. Hundreds of particles use separate workers. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. Parallel speed does not reduce weight degeneracy by itself. Aggregate diagnostics centrally.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Verify that every transition kernel leaves its intermediate target invariant. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use parallel computation as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Start from the known reference distribution. The base prior determines the first distribution and affects evidence. A broad scale prior makes the early path harder. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. AIS does not make prior choice disappear. Repeat plausible priors.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Inspect extreme-weight concentration, ESS and modal coverage across independent runs. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use prior sensitivity as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Now move along the annealing path. Both traverse temperatures, but AIS attaches importance weights for consistent estimation. An optimisation annealer seeks a mode without an evidence weight. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. Sharing a schedule does not make the purposes identical. Name the objective.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Repeat with denser schedules and reverse or alternative estimators when practical. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use relation to simulated annealing as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Importance weights correct each trajectory. SMC carries a population with adaptive weights and resampling; AIS uses independent weighted trajectories without population resampling. Both can share intermediate targets and kernels. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. Algorithm labels should match the implemented mechanics. Compare cost and diagnostics.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Save every schedule, transition setting, seed and run-level log weight. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use relation to sequential monte carlo as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Did You Know? The arithmetic mean of AIS weights targets a normalising-constant ratio, while averaging log weights targets a different quantity. That is why a careful report includes the construction, not only the final scalar.
Start from the known reference distribution. Save schedules, seeds, transition settings, every log weight and software versions. A second implementation reproduces the mean ratio. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. A final scalar alone cannot reveal degeneracy. Archive run-level output.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Average ordinary weights before taking a logarithm, using log-sum-exp for stability. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use reproducibility as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Now move along the annealing path. An accurate evidence ratio does not prove future reliability predictions are calibrated. The selected model fails on hotter operating conditions. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. Evidence and deployment utility remain distinct. Test later data.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Verify that every transition kernel leaves its intermediate target invariant. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use predictive validation as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Importance weights correct each trajectory. AIS connects exponents, Markov chains, products, averages and logarithms. Students can verify 1.44, 1.20 and 0.18232. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. The key is correction: a path can be approximate while weights target the right integral. Compare mean weight with mean log weight.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Inspect extreme-weight concentration, ESS and modal coverage across independent runs. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use parent and student learning route as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Useful next reading: approach to learning mathematics, Bukit Timah Mathematics curriculum overview, cross-subject learning approach, nested likelihood-ratio guide, Mathematics Learning Hub, eduKate Mathematics Learning System. Choose the destination that matches the next question rather than treating links as a checklist.
Start from the known reference distribution. Annealed importance sampling is not ordinary simulated annealing and its output is not an unweighted posterior sample. Valid kernels and correct incremental weights are essential. This step matters because annealed paths, importance correction and reliability evidence require the mathematical object, prediction target and reliability question to stay aligned. Extreme weights signal low information. Read Neal’s original paper and compare SMC samplers.
A reproducible audit should record variables, units, observation boundaries, censoring codes, likelihood factorisation and software settings. Repeat with denser schedules and reverse or alternative estimators when practical. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to support a model comparison.
Try a small learning experiment: multiply three incremental factors, average three independent weights and compare log of the mean with mean log weight. Use common misconceptions and next reading as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.
Quick answers
What must be fixed before comparing models? Use the same observations, censoring mechanism, time scale, likelihood convention and predictive unit.
Does a preferred value prove a model is true? No. The comparison is conditional on its candidates, data and assumptions.
Which mathematics matters? Algebra organises identities, logarithms stabilise products, probability represents uncertainty, variance measures instability and graphs expose influential cases.
How should a student practise? Preserve intermediate quantities, predict directions before calculating and change one input at a time.
What follows the comparison? Diagnostics, uncertainty analysis, posterior or predictive checks, later-data validation and a separate decision analysis when costs or safety matter.
Extended practice laboratory
Practice case 1: Reference and target distributions
Setup: Use this chapter’s illustrative case: A proper prior serves as the reference and likelihood times prior as the target.
Task: State both endpoints.
Checking target: An unknown or improper base constant prevents an absolute evidence estimate.
Practice case 2: Intermediate annealing path
Setup: Use this chapter’s illustrative case: Beta increases from zero to one.
Task: Record every beta.
Checking target: A schedule that changes too quickly produces volatile weights.
Practice case 3: One AIS trajectory
Setup: Use this chapter’s illustrative case: A particle moves through failure-rate parameter space.
Task: Keep path order explicit.
Checking target: A trajectory is not an ordinary posterior chain.
Practice case 4: Incremental weight factors
Setup: Use this chapter’s illustrative case: Likelihood to a beta increment supplies a positive multiplier.
Task: Follow the defined convention exactly.
Checking target: Evaluating the factor after the wrong transition changes the algorithm.
Practice case 5: Worked weight 1.44
Setup: Use this chapter’s illustrative case: The AIS weight is 1.44.
Task: Reproduce the product and its log.
Checking target: Adding the factors gives no importance ratio.
Practice case 6: Average of independent weights
Setup: Use this chapter’s illustrative case: Weights 1.44, 0.96 and 1.20 average to 1.20.
Task: Keep every run.
Checking target: A median weight targets a different quantity.
Practice case 7: Worked log ratio 0.18232
Setup: Use this chapter’s illustrative case: log(1.20) is about 0.18232.
Task: Use log-sum-exp for stability.
Checking target: Averaging log weights and exponentiating generally differs by Jensen’s inequality.
Practice case 8: Markov transition kernels
Setup: Use this chapter’s illustrative case: A Metropolis update targets the current power posterior.
Task: Verify detailed balance or invariance.
Checking target: A convenient move with the wrong invariant density biases AIS.
Practice case 9: Transitions need not equilibrate fully
Setup: Use this chapter’s illustrative case: Several short Metropolis steps follow each beta.
Task: Balance path length and movement.
Checking target: Zero mixing can produce extreme weights.
Practice case 10: Temperature schedule
Setup: Use this chapter’s illustrative case: Two hundred beta values replace twenty.
Task: Pilot several schedules.
Checking target: More stages cost computation and do not cure a wrong kernel.