VIEW THIS AS

Auto mode follows the Route Engine until you choose a viewpoint.

YOU ARE HERE

ROUTE CHECK

CONNECTED TO

WHAT NEXT

Use the canonical route for this room, or HELP if you are unsure.

Why Mathematics? | Dual-Averaging Step-Size Adaptation for HMC Reliability Models: Acceptance Targets, Warmup Windows, Stability and Speed

The importance of mathematics becomes visible when Hamiltonian Monte Carlo must choose a numerical integration step size. Large steps move quickly but can accumulate energy error and cause rejection or divergence. Very small steps follow the trajectory accurately but require more gradient evaluations. Dual averaging adapts the log step size during warmup toward a target acceptance statistic.

In a teaching comparison, one transition travels integration time 1.6. With step size 0.20 it needs 1.6/0.20=8 leapfrog steps; with step size 0.05 it needs 32. The smaller step can improve numerical accuracy but costs four times as many steps for the same integration time. Real adaptation balances stability and efficiency while the metric and posterior geometry are also changing.

Full chapter index

Chapters 1–5 · Understand the question
  1. What step size controls
  2. Large-step risk
  3. Small-step cost
  4. Integration time
  5. Worked eight steps
Chapters 6–10 · Reproduce the example
  1. Worked 32 steps
  2. Acceptance statistic
  3. Target acceptance
  4. Dual averaging
  5. Why log step size
Chapters 11–15 · Check assumptions
  1. Regularisation toward a reference
  2. Warmup only
  3. Fast and slow parameters
  4. Metric interaction
  5. Target acceptance and divergences
Chapters 16–20 · Explore applications
  1. Target too high
  2. Target too low
  3. Step-size jitter
  4. Censored reliability geometry
  5. Hierarchical funnels
Chapters 21–25 · Study and verify
  1. Diagnostics after adaptation
  2. Comparing configurations
  3. Reproducibility
  4. Parent and student learning route
  5. Common misconceptions and next reading

CHAPTER 1 OF 25 · Understand the question

1. What step size controls

Back to contents

Begin with leapfrog discretisation. Epsilon sets the discretisation interval of the leapfrog integrator. Each update advances Hamiltonian time by epsilon. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. It is not the size of a direct parameter jump. Interpret it algorithmically.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Save target acceptance, final step size, warmup schedule, metric and software version. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use what step size controls as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Did You Know? Dual averaging adapts log step size, which keeps epsilon positive and turns additive optimisation updates into multiplicative changes. That is why a careful report includes the construction, not only the final scalar.

Contents · Next chapter

CHAPTER 2 OF 25 · Understand the question

2. Large-step risk

Back to contents

Now balance energy error against gradient cost. Large epsilon creates larger integration error and lower acceptance. A curved reliability funnel produces divergences. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Occasional acceptance does not make the path accurate. Inspect energy error.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Compare numerical stability and ESS per second across configurations. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use large-step risk as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 3 OF 25 · Understand the question

3. Small-step cost

Back to contents

Warmup optimisation stabilises a positive step size. Small epsilon improves local accuracy but increases leapfrog work. Many gradients are evaluated per transition. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Tiny steps can make a run impractically slow. Measure cost per effective draw.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Keep adaptation during warmup separate from fixed-kernel sampling. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use small-step cost as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 4 OF 25 · Understand the question

4. Integration time

Back to contents

Begin with leapfrog discretisation. For fixed HMC, integration time is approximately step count times epsilon. The teaching trajectory uses time 1.6. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. NUTS chooses path length dynamically. Keep concepts distinct.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Treat higher target acceptance as a tradeoff rather than a quality score. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use integration time as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Primary research: https://www.jmlr.org/papers/v15/hoffman14a.html. Read the authors’ definition and conditions before carrying the method into a censored reliability model.

Contents · Previous chapter · Next chapter

CHAPTER 5 OF 25 · Understand the question

5. Worked eight steps

Back to contents

Now balance energy error against gradient cost. Divide 1.6 by 0.20. The larger-step setup needs eight steps. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Rounding is needed when the ratio is not integral. State the convention.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Recheck divergences, tree depth, R-hat, ESS and MCSE after adaptation. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use worked eight steps as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

QuantityValue
Integration time1.6
Step size A0.20
Leapfrog steps A8
Step size B0.05
Leapfrog steps B32
Relative step count4 times
Step-size and integration-time example.

Contents · Previous chapter · Next chapter

CHAPTER 6 OF 25 · Reproduce the example

6. Worked 32 steps

Back to contents

Warmup optimisation stabilises a positive step size. Divide 1.6 by 0.05. The smaller-step setup needs 32 steps. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Four times the work does not guarantee four times the information. Compare ESS per second.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Save target acceptance, final step size, warmup schedule, metric and software version. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use worked 32 steps as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Did You Know? Dual averaging adapts log step size, which keeps epsilon positive and turns additive optimisation updates into multiplicative changes. That is why a careful report includes the construction, not only the final scalar.

Contents · Previous chapter · Next chapter

CHAPTER 7 OF 25 · Reproduce the example

7. Acceptance statistic

Back to contents

Begin with leapfrog discretisation. Adaptation uses an acceptance-related statistic that reflects integration error. A low statistic pushes toward smaller steps. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. A high acceptance rate alone can accompany inefficient tiny steps. Balance accuracy and speed.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Compare numerical stability and ESS per second across configurations. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use acceptance statistic as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 8 OF 25 · Reproduce the example

8. Target acceptance

Back to contents

Now balance energy error against gradient cost. The user or software specifies a desired average acceptance level. A target such as 0.8 guides adaptation. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. The target is not an achieved scientific confidence level. Use precise language.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Keep adaptation during warmup separate from fixed-kernel sampling. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use target acceptance as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 9 OF 25 · Reproduce the example

9. Dual averaging

Back to contents

Warmup optimisation stabilises a positive step size. A stochastic optimisation scheme updates log epsilon using cumulative discrepancy from the target. Warmup corrections become more stable over time. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. It is not ordinary averaging of step sizes. Follow the algorithm.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Treat higher target acceptance as a tradeoff rather than a quality score. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use dual averaging as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 10 OF 25 · Reproduce the example

10. Why log step size

Back to contents

Begin with leapfrog discretisation. Optimising on a logarithmic scale keeps epsilon positive and handles multiplicative changes. A log update halves or doubles epsilon naturally. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Forgetting the transform can produce invalid steps. Store both scales.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Recheck divergences, tree depth, R-hat, ESS and MCSE after adaptation. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use why log step size as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 11 OF 25 · Check assumptions

11. Regularisation toward a reference

Back to contents

Now balance energy error against gradient cost. Dual averaging shrinks early updates toward a reference log step size. Noisy first iterations do not dominate permanently. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Poor initialisation can still disrupt warmup. Use valid initials.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Save target acceptance, final step size, warmup schedule, metric and software version. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use regularisation toward a reference as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Did You Know? Dual averaging adapts log step size, which keeps epsilon positive and turns additive optimisation updates into multiplicative changes. That is why a careful report includes the construction, not only the final scalar.

Contents · Previous chapter · Next chapter

CHAPTER 12 OF 25 · Check assumptions

12. Warmup only

Back to contents

Warmup optimisation stabilises a positive step size. Adaptation normally ends before retained posterior sampling. Post-warmup uses a fixed kernel. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Continuing uncontrolled adaptation can invalidate stationary sampling. Separate phases.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Compare numerical stability and ESS per second across configurations. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use warmup only as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Second authoritative source: https://mc-stan.org/docs/reference-manual/mcmc.html. Compare its reference distribution, diagnostics and finite-simulation cautions with this worked example.

Contents · Previous chapter · Next chapter

CHAPTER 13 OF 25 · Check assumptions

13. Fast and slow parameters

Back to contents

Begin with leapfrog discretisation. Step size learns from local information; the metric needs broader covariance information. Warmup windows alternate adaptation roles. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Changing the metric alters the appropriate step size. Allow final retuning.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Keep adaptation during warmup separate from fixed-kernel sampling. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use fast and slow parameters as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 14 OF 25 · Check assumptions

14. Metric interaction

Back to contents

Now balance energy error against gradient cost. A well-scaled mass matrix permits larger stable steps across directions. Standardised shape and scale move comparably. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. A bad metric forces epsilon to satisfy the narrowest direction. Inspect conditioning.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Treat higher target acceptance as a tradeoff rather than a quality score. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use metric interaction as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 15 OF 25 · Check assumptions

15. Target acceptance and divergences

Back to contents

Warmup optimisation stabilises a positive step size. Raising the target often yields smaller steps and may reduce mild divergences. Move from 0.8 toward 0.95 after diagnosis. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. It can make runs expensive and cannot fix severe geometry. Reparameterise when needed.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Recheck divergences, tree depth, R-hat, ESS and MCSE after adaptation. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use target acceptance and divergences as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 16 OF 25 · Explore applications

16. Target too high

Back to contents

Begin with leapfrog discretisation. Near-one targets can produce tiny steps and very deep trajectories. Acceptance looks excellent while ESS per second falls. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. A high number is not automatically safer. Check computation cost.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Save target acceptance, final step size, warmup schedule, metric and software version. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use target too high as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Did You Know? Dual averaging adapts log step size, which keeps epsilon positive and turns additive optimisation updates into multiplicative changes. That is why a careful report includes the construction, not only the final scalar.

Contents · Previous chapter · Next chapter

CHAPTER 17 OF 25 · Explore applications

17. Target too low

Back to contents

Now balance energy error against gradient cost. A low target tolerates larger energy errors and more rejection. Trajectories fail in a narrow neck. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Fast iterations can yield biased exploration. Inspect divergences and R-hat.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Compare numerical stability and ESS per second across configurations. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use target too low as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 18 OF 25 · Explore applications

18. Step-size jitter

Back to contents

Warmup optimisation stabilises a positive step size. Random variation in epsilon can avoid resonant interactions in some fixed-HMC settings. Each transition uses a nearby step size. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Jitter can also increase cost or error and is often unnecessary. Use only with a reason.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Keep adaptation during warmup separate from fixed-kernel sampling. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use step-size jitter as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 19 OF 25 · Explore applications

19. Censored reliability geometry

Back to contents

Begin with leapfrog discretisation. Weakly identified shape and scale produce anisotropic curvature. Heavy censoring creates a long ridge. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Step-size tuning cannot create missing data information. Strengthen design or priors.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Treat higher target acceptance as a tradeoff rather than a quality score. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use censored reliability geometry as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 20 OF 25 · Explore applications

20. Hierarchical funnels

Back to contents

Now balance energy error against gradient cost. A single global epsilon struggles across wide and narrow regions. Batch-scale funnels cause local divergence. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Smaller epsilon may be only a partial workaround. Use centring analysis.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Recheck divergences, tree depth, R-hat, ESS and MCSE after adaptation. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use hierarchical funnels as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 21 OF 25 · Study and verify

21. Diagnostics after adaptation

Back to contents

Warmup optimisation stabilises a positive step size. Check divergences, tree depth, energy, R-hat, ESS and MCSE in retained draws. Warmup reports no error but post-warmup depth saturates. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. A final epsilon is not a quality certificate. Audit the full run.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Save target acceptance, final step size, warmup schedule, metric and software version. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use diagnostics after adaptation as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Did You Know? Dual averaging adapts log step size, which keeps epsilon positive and turns additive optimisation updates into multiplicative changes. That is why a careful report includes the construction, not only the final scalar.

Contents · Previous chapter · Next chapter

CHAPTER 22 OF 25 · Study and verify

22. Comparing configurations

Back to contents

Begin with leapfrog discretisation. Use repeated seeds and the same posterior target to compare ESS per second and warnings. Two acceptance targets are benchmarked. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Choosing after a favourable estimate invites selection. Predefine metrics.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Compare numerical stability and ESS per second across configurations. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use comparing configurations as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 23 OF 25 · Study and verify

23. Reproducibility

Back to contents

Now balance energy error against gradient cost. Save target, final epsilon, warmup schedule, metric and software version. A reviewer can rerun adaptation. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Defaults change across versions. Record explicit settings.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Keep adaptation during warmup separate from fixed-kernel sampling. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use reproducibility as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Contents · Previous chapter · Next chapter

CHAPTER 24 OF 25 · Study and verify

24. Parent and student learning route

Back to contents

Warmup optimisation stabilises a positive step size. Step-size adaptation connects division, logarithms, optimisation and error control. Verify eight versus 32 steps. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. The lesson is balancing accuracy with computational cost. Graph cost against epsilon.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Treat higher target acceptance as a tradeoff rather than a quality score. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use parent and student learning route as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Useful next reading: eduKate Mathematics Learning System, approach to learning mathematics, Bukit Timah Mathematics curriculum overview, cross-subject learning approach, previous Why Mathematics article, Mathematics Learning Hub. Choose the destination that matches the next question rather than treating links as a checklist.

Contents · Previous chapter · Next chapter

CHAPTER 25 OF 25 · Study and verify

25. Common misconceptions and next reading

Back to contents

Begin with leapfrog discretisation. Target acceptance is not posterior probability, model fit or a universal best setting. Dual averaging tunes one algorithmic quantity during warmup. This step matters because numerical accuracy, warmup optimisation and efficient Hamiltonian computation require the mathematical object, prediction target and reliability question to stay aligned. Good adaptation cannot validate the model. Read Hoffman and Gelman and current Stan guidance.

A reproducible audit should record variables, units, observation boundaries, censoring codes, model version and software settings. Recheck divergences, tree depth, R-hat, ESS and MCSE after adaptation. Rebuild the displayed number from saved inputs before interpreting it. A result that cannot be reproduced is not ready to guide reliability work.

Try a small learning experiment: divide integration time 1.6 by step sizes 0.20 and 0.05, then compare eight with 32 steps and discuss cost. Use common misconceptions and next reading as the focus. Predict the direction of change before using software, alter one defensible input, and finish with two sentences: what the calculation supports, and what remains uncertain. This habit turns mathematics education into transferable problem-solving rather than formula recall.

Quick answers

What must be fixed before checking a model or algorithm? Define the dataset unit, censoring mechanism, time scale, parameterisation, simulation design and diagnostic rule.

Does a satisfactory diagnostic prove the model is true? No. It supports a particular check under stated assumptions and simulation conditions.

Which mathematics matters? Probability defines the reference distribution, simulation makes it observable, proportions summarise tail events, variance describes uncertainty and graphs expose structured mismatch.

How should a student practise? Preserve intermediate quantities, predict directions before calculating, change one input at a time and explain the result in ordinary language.

What follows an internal check? Sensitivity analysis, independent computation, held-out or later-data prediction and a separate decision analysis when cost or safety matters.

Contents · Previous chapter · Mathematics Learning Hub

Extended practice laboratory

Practice case 1: What step size controls

Setup: Use this chapter’s illustrative case: Each update advances Hamiltonian time by epsilon.

Task: Interpret it algorithmically.

Checking target: It is not the size of a direct parameter jump.

Practice case 2: Large-step risk

Setup: Use this chapter’s illustrative case: A curved reliability funnel produces divergences.

Task: Inspect energy error.

Checking target: Occasional acceptance does not make the path accurate.

Practice case 3: Small-step cost

Setup: Use this chapter’s illustrative case: Many gradients are evaluated per transition.

Task: Measure cost per effective draw.

Checking target: Tiny steps can make a run impractically slow.

Practice case 4: Integration time

Setup: Use this chapter’s illustrative case: The teaching trajectory uses time 1.6.

Task: Keep concepts distinct.

Checking target: NUTS chooses path length dynamically.

Practice case 5: Worked eight steps

Setup: Use this chapter’s illustrative case: The larger-step setup needs eight steps.

Task: State the convention.

Checking target: Rounding is needed when the ratio is not integral.

Practice case 6: Worked 32 steps

Setup: Use this chapter’s illustrative case: The smaller-step setup needs 32 steps.

Task: Compare ESS per second.

Checking target: Four times the work does not guarantee four times the information.

Practice case 7: Acceptance statistic

Setup: Use this chapter’s illustrative case: A low statistic pushes toward smaller steps.

Task: Balance accuracy and speed.

Checking target: A high acceptance rate alone can accompany inefficient tiny steps.

Practice case 8: Target acceptance

Setup: Use this chapter’s illustrative case: A target such as 0.8 guides adaptation.

Task: Use precise language.

Checking target: The target is not an achieved scientific confidence level.

Practice case 9: Dual averaging

Setup: Use this chapter’s illustrative case: Warmup corrections become more stable over time.

Task: Follow the algorithm.

Checking target: It is not ordinary averaging of step sizes.

Practice case 10: Why log step size

Setup: Use this chapter’s illustrative case: A log update halves or doubles epsilon naturally.

Task: Store both scales.

Checking target: Forgetting the transform can produce invalid steps.