Choose your reading route
eduKateSG · Why Mathematics? · Sequential Importance Sampling for Reliability: Intermediate Densities, Weights, Resampling and Rare Events
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 6–10
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 11–15
Check assumptions
See where design and calibration matter.Route 4 · Chapters 16–20
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 21–25
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why mathematics is important in rare-event reliability becomes practical when ordinary sampling almost never visits the failure region. Importance sampling draws more often from consequential scenarios and corrects the deliberate sampling bias with likelihood ratios. Sequential importance sampling goes further: it constructs a sequence of intermediate distributions, reweights particles gradually, resamples when weights become too uneven and moves them with Markov kernels.
This guide explains target and proposal densities, weights, effective sample size, resampling, intermediate temperatures and failure-probability estimation. The method can be powerful, but it cannot estimate a failure mode that its proposal never reaches, and unstable weights can make a large nominal sample behave like only a handful of useful scenarios.
Full chapter index
Chapters 1–5 · Understand the question
Chapters 6–10 · Reproduce the example
Chapters 11–15 · Check assumptions
Chapters 16–20 · Explore applications
Chapters 21–25 · Study and verify
Reliability failure probability is the expectation of a failure indicator under the true input density. P_f=∫I[g(x)≤0]f(x)dx. The formula depends on a defensible joint density f and limit state g. A practical next step is to define units, dependence and event.
Start with the mechanism: Importance sampling draws from q rather than f to visit the failure region more often. Consider this example: Shift a load distribution toward larger loads while retaining support over all relevant failures. The limit is equally important: If q is zero where failure under f is possible, correction is impossible. Students can test the idea when they check support before sampling.
A scenario twice as likely under q as f has weight 0.5. The mathematics behind it is precise: Each sampled scenario receives weight f(x)/q(x) to correct the changed sampling law. Yet the result has a boundary. Density ratios can overflow or become extremely variable. For a reliable investigation, compute log weights and inspect tails.
The ordinary importance-sampling estimate averages I_F(x_i)f(x_i)/q(x_i) for draws from q. In an illustrative case, Four failure draws with corrected contributions 0.002, 0.004, 0 and 0.006 average to 0.003. This should not be overread because unbiasedness does not guarantee low variance in a finite run. A careful team will report standard error and diagnostics. Primary research is available from https://doi.org/10.1016/j.strusafe.2016.06.002.
A zero-variance proposal would sample exactly from the failure-conditioned target, which depends on the unknown failure probability. The ideal distribution concentrates perfectly on relevant failures. It cannot be used directly because it requires the answer. A practical next step is to approximate it adaptively.
Start with the mechanism: SIS introduces intermediate distributions that gradually transform the original input density toward a failure-focused target. Consider this example: A temperature or threshold schedule moves particles through increasingly severe regions. The limit is equally important: Abrupt steps cause weight collapse; tiny steps waste computation. Students can test the idea when they adapt steps using weight diagnostics.
A particle compatible with the new severe distribution gains relative weight. The mathematics behind it is precise: At each bridge, particles receive a ratio between successive unnormalised targets. Yet the result has a boundary. Multiplying many ratios can underflow. For a reliable investigation, normalise in log space.
Divide weights by their sum to create shares that add to one. In an illustrative case, Weights 2, 3 and 5 become 0.2, 0.3 and 0.5. This should not be overread because normalisation hides whether all raw weights were extremely small. A careful team will preserve log normalising constants.
ESS≈1/sum of squared normalised weights measures weight concentration. Weights 0.5,0.3,0.2 give ESS≈2.63 rather than 3. ESS is a diagnostic, not independent observations literally. A practical next step is to set and justify a resampling threshold.
Start with the mechanism: Resampling duplicates high-weight particles and removes low-weight ones to restore equal weights. Consider this example: Systematic resampling creates offspring counts from cumulative weights. The limit is equally important: Repeated resampling reduces diversity and adds randomness. Students can test the idea when they track ancestors and unique particles.
A Gaussian-mixture independence proposal or conditional Metropolis step explores nearby states. The mathematics behind it is precise: After resampling, MCMC kernels move duplicates while preserving the current intermediate distribution. Yet the result has a boundary. Poor acceptance leaves clones; excessive moves waste model calls. For a reliable investigation, measure mixing and acceptance.
Reliability SIS can use smooth approximations to the failure indicator or tempering of a likelihood-like function. In an illustrative case, A sigmoid of the safety margin sharpens toward I[g≤0]. This should not be overread because the path changes efficiency and can introduce tuning sensitivity. A careful team will publish the full sequence. A second primary or official research source is https://doi.org/10.1016/0167-4730(89)90003-9.
Mixtures can represent several regions or failure modes better than one Gaussian. Separate components follow two disconnected failure domains. Fitting mixtures to too few particles can collapse a component. A practical next step is to use regularisation and mode checks.
Start with the mechanism: Conditional MCMC updates can remain practical when independent proposals struggle in many dimensions. Consider this example: Update coordinates of a large uncertain load vector while respecting the bridge density. The limit is equally important: Strong dependence can slow coordinate-wise mixing. Students can test the idea when they compare block and conditional kernels.
Multiply stage estimates or accumulate their log values. The mathematics behind it is precise: Sequential weights estimate ratios of normalising constants, which combine to the rare-event probability under the chosen construction. Yet the result has a boundary. A coding error in one stage contaminates the final product. For a reliable investigation, recompute from saved particles and weights.
Five sampled failure indicators are 1,1,0,1,0 with weights 0.01,0.03,0.02,0.04,0.05 before averaging. In an illustrative case, Corrected contribution sum is 0.08; under a simple five-draw average the estimate is 0.016. This should not be overread because self-normalised and ordinary estimators have different formulas and properties. A careful team will name the estimator explicitly.
Large likelihood-ratio variation makes the estimator unstable even when many failures are observed under q. One scenario supplies 90% of the total estimate. Failure count alone is not an efficiency diagnostic. A practical next step is to report ess, maximum share and repeated-run variation.
Start with the mechanism: A proposal centred on one design point can miss another important failure region. Consider this example: Two structural mechanisms fail under different load combinations. The limit is equally important: A low-variance estimate within one mode can be globally biased. Students can test the idea when they use mixtures, broad pilots and mode-specific checks.
Intermediate distributions approach an approximately optimal importance density. The mathematics behind it is precise: Primary research develops SIS for low- and high-dimensional component and system reliability using different MCMC kernels. Yet the result has a boundary. Published benchmark performance is conditional on its models. For a reliable investigation, reproduce on a transparent case first.
Estimate a rare dice event by oversampling high faces and correcting with known probability ratios. In an illustrative case, Compare the corrected estimate with an uncorrected biased count. This should not be overread because a dice proposal is far simpler than fitted high-dimensional densities. A careful team will keep every draw and weight.
Importance sampling connects fractions, probability, weighted averages and honest correction for deliberate bias. Ask why a scenario was oversampled and how its weight repairs that choice. More observed failures in the proposal sample do not mean real failure is common. A practical next step is to begin with a finite table of probabilities.
Start with the mechanism: Importance sampling does not change the real distribution, and ESS does not guarantee mode coverage. Consider this example: A proposal can look efficient while assigning no samples to a second failure mode. The limit is equally important: Self-normalisation can introduce bias. Students can test the idea when they audit support and estimator formula.
A cheap simulator with probability 0.05 may need no elaborate proposal. The mathematics behind it is precise: If failure is not rare or a good bridge and MCMC kernel cannot be built, direct or subset methods may be more transparent. Yet the result has a boundary. Sequential tuning creates implementation risk. For a reliable investigation, compare cost and reproducibility.
Primary structural-safety research develops SIS for rare reliability events, while foundational work explains importance sampling in structural systems. In an illustrative case, Connect these sources to subset simulation and reliability bounds. This should not be overread because no proposal can repair a wrong physical model. A careful team will use the hub to review weighted means. Continue with the Mathematics Learning Hub and the previous reliability article.
FAQs should cover support, ratios, ESS, resampling, MCMC moves, modes and uncertainty. Every answer should distinguish target, bridge and proposal. Do not report only the number of proposal failures. A practical next step is to publish weights and ancestry.
Extended practice laboratory
This laboratory turns sequential importance sampling for reliability into a sequence of checkable decisions. Students should keep a ledger of definitions, calculations, assumptions and later evidence. A parent or educator can ask what the model knew at the time and what would make the conclusion change. The goal is not to imitate advanced software; it is to understand the mathematical mechanism well enough to detect a wrong denominator, a leaked timestamp, an unsupported extrapolation or an overclaim.
A strong project on sequential importance sampling for reliability includes a simple baseline, one worked example, a sensitivity check and an untouched evaluation. It reports limits with the same care as results. This is how school algebra, probability, graphs and functions become transferable reasoning tools.
For a capstone, build a small reproducible investigation of sequential importance sampling for reliability. Freeze the question and data contract first. Preserve every intermediate value so another student can recompute the result. Compare at least two defensible methods, evaluate on later or held-out evidence, and include a failure case. End with a one-page decision record: what is supported, what remains uncertain, what action is allowed, and what new evidence would trigger review. The capstone succeeds when its reasoning survives a changed example, not when it produces the most impressive chart. A sequential-importance-sampling capstone should define the original density, bridge sequence, proposal or Markov kernels and exact estimator before simulation. Preserve log incremental weights, normalising constants, resampling events, ancestors and model outputs. Compare direct Monte Carlo, one-shot importance sampling and SIS under equal model-call budgets. Include two separated failure modes and a deliberately poor proposal so support failure and weight collapse are visible. Track effective sample size, maximum normalised weight, unique ancestors, acceptance rates and repeated-run variance. Recalculate with self-normalised and ordinary estimators only when each is mathematically appropriate, and explain their different bias properties. The final method card should state where proposal support was verified, how bridge steps were chosen, when resampling occurred and what tail or physical assumptions remain outside the sampling diagnostics.
Failure estimate and normalisers deeper audit
Use failure estimate and normalisers to practise sensitivity analysis. Sequential weights estimate ratios of normalising constants, which combine to the rare-event probability under the chosen construction. Start from this example: Multiply stage estimates or accumulate their log values. Change one assumption while keeping the others fixed and predict the direction before calculating. A coding error in one stage contaminates the final product. Recompute from saved particles and weights. Report whether the decision is stable or depends heavily on the changed assumption. Record the result in plain language for a student and technical language for a reviewer. The two versions should agree on the quantity and limitation even though their vocabulary differs.
A simple weighted example deeper audit
Build a small counterexample for a simple weighted example. Five sampled failure indicators are 1,1,0,1,0 with weights 0.01,0.03,0.02,0.04,0.05 before averaging. Corrected contribution sum is 0.08; under a simple five-draw average the estimate is 0.016. Then create a nearby case where the shortcut fails. This matters because self-normalised and ordinary estimators have different formulas and properties. Name the estimator explicitly. Explain the failure in plain language and connect it to the exact mathematical step that changed. Record the result in plain language for a student and technical language for a reviewer. The two versions should agree on the quantity and limitation even though their vocabulary differs.
Weight variance deeper audit
Treat weight variance as a model-checking problem. Large likelihood-ratio variation makes the estimator unstable even when many failures are observed under q. One scenario supplies 90% of the total estimate. Preserve inputs, intermediate values and output. Failure count alone is not an efficiency diagnostic. Report ESS, maximum share and repeated-run variation. Evaluate the same rule on later or held-out evidence and separate numerical correctness from real-world validity. Record the result in plain language for a student and technical language for a reviewer. The two versions should agree on the quantity and limitation even though their vocabulary differs.
Multiple failure modes deeper audit
Connect the mathematics of multiple failure modes to a decision. A proposal centred on one design point can miss another important failure region. Two structural mechanisms fail under different load combinations. Calculate the quantity, then translate it into an allowed action and an action that the evidence does not support. A low-variance estimate within one mode can be globally biased. Use mixtures, broad pilots and mode-specific checks. This distinction prevents an accurate calculation from becoming an overclaim. Record the result in plain language for a student and technical language for a reviewer. The two versions should agree on the quantity and limitation even though their vocabulary differs.
Sequential structural reliability deeper audit
Turn sequential structural reliability into a transparent worksheet. Primary research develops SIS for low- and high-dimensional component and system reliability using different MCMC kernels. Intermediate distributions approach an approximately optimal importance density. List every quantity, unit and timestamp, then recompute the example after changing one input. Published benchmark performance is conditional on its models. Reproduce on a transparent case first. Finish by naming the output as description, prediction, diagnosis or causal evidence and justify that label. Record the result in plain language for a student and technical language for a reviewer. The two versions should agree on the quantity and limitation even though their vocabulary differs.
Student activity deeper audit
Begin with the operational question behind student activity. Estimate a rare dice event by oversampling high faces and correcting with known probability ratios. Now examine the illustrative case: Compare the corrected estimate with an uncorrected biased count. Draw the relevant timeline or graph before calculating. The important boundary is that a dice proposal is far simpler than fitted high-dimensional densities. Keep every draw and weight. Compare a simple baseline and record what evidence would reverse the conclusion. Record the result in plain language for a student and technical language for a reviewer. The two versions should agree on the quantity and limitation even though their vocabulary differs.
Parent and educator guidance deeper audit
For this case, audit the mechanism rather than memorising vocabulary. Importance sampling connects fractions, probability, weighted averages and honest correction for deliberate bias. Ask why a scenario was oversampled and how its weight repairs that choice. Ask which denominator, risk set, neighbourhood or candidate pool makes the result meaningful. More observed failures in the proposal sample do not mean real failure is common. Begin with a finite table of probabilities. A second student should be able to reproduce the answer from the saved ledger alone. Record the result in plain language for a student and technical language for a reviewer. The two versions should agree on the quantity and limitation even though their vocabulary differs.
Practice case 1: Target integral
Setup: Use a small example for target integral.
Task: Define units, dependence and event.
Checking target: The formula depends on a defensible joint density f and limit state g.
Turn target integral into a transparent worksheet. Reliability failure probability is the expectation of a failure indicator under the true input density. P_f=∫I[g(x)≤0]f(x)dx. List every quantity, unit and timestamp, then recompute the example after changing one input. The formula depends on a defensible joint density f and limit state g. Define units, dependence and event. Finish by naming the output as description, prediction, diagnosis or causal evidence and justify that label.
Practice case 2: Proposal distribution
Setup: Use a small example for proposal distribution.
Task: Check support before sampling.
Checking target: If q is zero where failure under f is possible, correction is impossible.
Begin with the operational question behind proposal distribution. Importance sampling draws from q rather than f to visit the failure region more often. Now examine the illustrative case: Shift a load distribution toward larger loads while retaining support over all relevant failures. Draw the relevant timeline or graph before calculating. The important boundary is that if q is zero where failure under f is possible, correction is impossible. Check support before sampling. Compare a simple baseline and record what evidence would reverse the conclusion.
Practice case 3: Likelihood ratio
Setup: Use a small example for likelihood ratio.
Task: Compute log weights and inspect tails.
Checking target: Density ratios can overflow or become extremely variable.
For this case, audit the mechanism rather than memorising vocabulary. Each sampled scenario receives weight f(x)/q(x) to correct the changed sampling law. A scenario twice as likely under q as f has weight 0.5. Ask which denominator, risk set, neighbourhood or candidate pool makes the result meaningful. Density ratios can overflow or become extremely variable. Compute log weights and inspect tails. A second student should be able to reproduce the answer from the saved ledger alone.
Practice case 4: Unbiased estimator
Setup: Use a small example for unbiased estimator.
Task: Report standard error and diagnostics.
Checking target: Unbiasedness does not guarantee low variance in a finite run.
Use unbiased estimator to practise sensitivity analysis. The ordinary importance-sampling estimate averages I_F(x_i)f(x_i)/q(x_i) for draws from q. Start from this example: Four failure draws with corrected contributions 0.002, 0.004, 0 and 0.006 average to 0.003. Change one assumption while keeping the others fixed and predict the direction before calculating. Unbiasedness does not guarantee low variance in a finite run. Report standard error and diagnostics. Report whether the decision is stable or depends heavily on the changed assumption.
Practice case 5: Optimal but unavailable proposal
Setup: Use a small example for optimal but unavailable proposal.
Task: Approximate it adaptively.
Checking target: It cannot be used directly because it requires the answer.
Build a small counterexample for optimal but unavailable proposal. A zero-variance proposal would sample exactly from the failure-conditioned target, which depends on the unknown failure probability. The ideal distribution concentrates perfectly on relevant failures. Then create a nearby case where the shortcut fails. This matters because it cannot be used directly because it requires the answer. Approximate it adaptively. Explain the failure in plain language and connect it to the exact mathematical step that changed.
Practice case 6: Sequential bridge
Setup: Use a small example for sequential bridge.
Task: Adapt steps using weight diagnostics.
Checking target: Abrupt steps cause weight collapse; tiny steps waste computation.
Treat sequential bridge as a model-checking problem. SIS introduces intermediate distributions that gradually transform the original input density toward a failure-focused target. A temperature or threshold schedule moves particles through increasingly severe regions. Preserve inputs, intermediate values and output. Abrupt steps cause weight collapse; tiny steps waste computation. Adapt steps using weight diagnostics. Evaluate the same rule on later or held-out evidence and separate numerical correctness from real-world validity.
Practice case 7: Incremental weights
Setup: Use a small example for incremental weights.
Task: Normalise in log space.
Checking target: Multiplying many ratios can underflow.
Connect the mathematics of incremental weights to a decision. At each bridge, particles receive a ratio between successive unnormalised targets. A particle compatible with the new severe distribution gains relative weight. Calculate the quantity, then translate it into an allowed action and an action that the evidence does not support. Multiplying many ratios can underflow. Normalise in log space. This distinction prevents an accurate calculation from becoming an overclaim.
Practice case 8: Normalised weights
Setup: Use a small example for normalised weights.
Task: Preserve log normalising constants.
Checking target: Normalisation hides whether all raw weights were extremely small.
Turn normalised weights into a transparent worksheet. Divide weights by their sum to create shares that add to one. Weights 2, 3 and 5 become 0.2, 0.3 and 0.5. List every quantity, unit and timestamp, then recompute the example after changing one input. Normalisation hides whether all raw weights were extremely small. Preserve log normalising constants. Finish by naming the output as description, prediction, diagnosis or causal evidence and justify that label.
Practice case 9: Effective sample size
Setup: Use a small example for effective sample size.
Task: Set and justify a resampling threshold.
Checking target: ESS is a diagnostic, not independent observations literally.
Begin with the operational question behind effective sample size. ESS≈1/sum of squared normalised weights measures weight concentration. Now examine the illustrative case: Weights 0.5,0.3,0.2 give ESS≈2.63 rather than 3. Draw the relevant timeline or graph before calculating. The important boundary is that ess is a diagnostic, not independent observations literally. Set and justify a resampling threshold. Compare a simple baseline and record what evidence would reverse the conclusion.
Practice case 10: Resampling
Setup: Use a small example for resampling.
Task: Track ancestors and unique particles.
Checking target: Repeated resampling reduces diversity and adds randomness.
For this case, audit the mechanism rather than memorising vocabulary. Resampling duplicates high-weight particles and removes low-weight ones to restore equal weights. Systematic resampling creates offspring counts from cumulative weights. Ask which denominator, risk set, neighbourhood or candidate pool makes the result meaningful. Repeated resampling reduces diversity and adds randomness. Track ancestors and unique particles. A second student should be able to reproduce the answer from the saved ledger alone.
Practice case 11: Markov moves
Setup: Use a small example for markov moves.
Task: Measure mixing and acceptance.
Checking target: Poor acceptance leaves clones; excessive moves waste model calls.
Use markov moves to practise sensitivity analysis. After resampling, MCMC kernels move duplicates while preserving the current intermediate distribution. Start from this example: A Gaussian-mixture independence proposal or conditional Metropolis step explores nearby states. Change one assumption while keeping the others fixed and predict the direction before calculating. Poor acceptance leaves clones; excessive moves waste model calls. Measure mixing and acceptance. Report whether the decision is stable or depends heavily on the changed assumption.
Practice case 12: Intermediate-distribution design
Setup: Use a small example for intermediate-distribution design.
Task: Publish the full sequence.
Checking target: The path changes efficiency and can introduce tuning sensitivity.
Build a small counterexample for intermediate-distribution design. Reliability SIS can use smooth approximations to the failure indicator or tempering of a likelihood-like function. A sigmoid of the safety margin sharpens toward I[g≤0]. Then create a nearby case where the shortcut fails. This matters because the path changes efficiency and can introduce tuning sensitivity. Publish the full sequence. Explain the failure in plain language and connect it to the exact mathematical step that changed.
Practice case 13: Gaussian-mixture proposals
Setup: Use a small example for gaussian-mixture proposals.
Task: Use regularisation and mode checks.
Checking target: Fitting mixtures to too few particles can collapse a component.
Treat gaussian-mixture proposals as a model-checking problem. Mixtures can represent several regions or failure modes better than one Gaussian. Separate components follow two disconnected failure domains. Preserve inputs, intermediate values and output. Fitting mixtures to too few particles can collapse a component. Use regularisation and mode checks. Evaluate the same rule on later or held-out evidence and separate numerical correctness from real-world validity.
Practice case 14: High-dimensional reliability
Setup: Use a small example for high-dimensional reliability.
Task: Compare block and conditional kernels.
Checking target: Strong dependence can slow coordinate-wise mixing.
Connect the mathematics of high-dimensional reliability to a decision. Conditional MCMC updates can remain practical when independent proposals struggle in many dimensions. Update coordinates of a large uncertain load vector while respecting the bridge density. Calculate the quantity, then translate it into an allowed action and an action that the evidence does not support. Strong dependence can slow coordinate-wise mixing. Compare block and conditional kernels. This distinction prevents an accurate calculation from becoming an overclaim.