Choose your reading route
eduKateSG · Why Mathematics? · Bayesian Information Criterion: Sample Size and Parsimony
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 5–9
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 10–15
Check assumptions
See where design and calibration matter.Route 4 · Chapters 16–20
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 21–26
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why is mathematics important when a model-selection rule should become more cautious about extra parameters as the sample grows? The Bayesian Information Criterion, or BIC, combines maximised likelihood with a penalty k ln(n). The natural logarithm of sample size makes the complexity charge increase with n.
BIC is a relative ranking tool for comparable candidate models. It does not make a model Bayesian merely because of its name, and its absolute value is not a percentage or quality grade.
Did You Know? At n=100, each extra parameter costs about 4.605 BIC units before any gain in fit is counted.
Full chapter index
Chapters 1–4 · Understand the question
Chapters 5–9 · Reproduce the example
Chapters 10–15 · Check assumptions
Chapters 16–20 · Explore applications
Chapters 21–26 · Study and verify
A larger sample can detect small deviations from a simpler model. BIC counters this by increasing the parameter penalty with ln(n), favouring parsimony more strongly as n grows.
NIST gives BIC=-2 ln(L)+k ln(n), where L is the maximised likelihood, k is the number of estimated parameters and n is sample size. Read the NIST information-criterion reference.
Let n=100, k=4 and log-likelihood=-120. Then BIC=240+4 ln(100). Since ln(100)=4.60517, BIC=240+18.42068=258.421.
| n | k | Log-likelihood |
|---|---|---|
| 100 | 4 | -120 |
Better likelihood reduces -2 ln(L); extra parameters increase k ln(n). The preferred candidate has the smaller total within the declared model set.
| Fit term | Penalty | BIC |
|---|---|---|
| 240 | 18.421 | 258.421 |
When n=20, ln(n) is about 2.996. When n=1000, it is about 6.908. The same additional parameter therefore faces a larger penalty in the bigger sample.
Count the freely estimated parameters used in the likelihood, including dispersion or variance terms where applicable. Confirm software conventions before comparing reports from different packages.
BIC=258.4 has no standalone interpretation. Calculate differences from the minimum among comparable models and inspect near-ties rather than declaring a universal threshold.
Candidates should use the same response data and likelihood scale. Missing-value changes, response transformations or different weights can break a direct comparison.
AIC uses 2k; BIC uses k ln(n). BIC generally penalises extra parameters more once n exceeds about 7.4. AIC is often associated with predictive efficiency, while BIC has a different large-sample model-identification motivation.
Under regular conditions and particular prior assumptions, BIC differences approximate log evidence comparisons. This approximation is not a substitute for a fully specified Bayesian analysis.
Repeated or clustered rows may not provide n independent pieces of information. Treating every correlated row as independent can overstate effective sample size and distort the penalty and likelihood.
BIC can only rank the models supplied. A simple but wrong candidate set can yield a confident-looking winner that fails residual or predictive checks.
Small BIC differences often indicate that several models deserve attention. Report the alternatives, coefficient stability and the practical consequences of choosing each one.
Preserve design requirements, main effects supporting interactions and scientifically necessary controls. Do not let a numerical search violate the meaning of the model.
Inspect residuals, influence, collinearity and prediction. A low BIC does not repair nonlinearity, measurement error, dependence or data leakage.
Record n, ln(n), k, log-likelihood, fit term, penalty and BIC for every candidate. Use one common data-status column to confirm identical rows.
Fit candidates with the same likelihood and preprocessing, verify convergence and parameter counts, calculate Delta BIC and preserve model formulas. Validate the shortlist separately.
BIC appears in signal processing, economics, genetics, time series, environmental models and machine learning. It teaches how evidence, complexity and sample size interact.
Hold log-likelihood improvement constant and calculate the BIC effect of one extra parameter at n=20, 100 and 1000. Explain why the preferred model can change.
BIC is not a posterior probability, causal test or prediction score. Its name does not mean a prior was explicitly fitted. The smallest value does not excuse an implausible model.
At n=100, one additional parameter costs ln(100)=4.605 BIC units. Because a one-unit increase in log-likelihood reduces the fit term by 2, the richer model needs a log-likelihood gain greater than about 2.303 merely to overcome that penalty. A smaller gain can improve raw fit while worsening BIC. This calculation makes the trade-off visible and helps explain why BIC can select a simpler model than AIC from the same candidate set. The final decision still needs diagnostics and prediction evidence. Preserve the exact likelihood and parameter-count conventions.
The Mallows Cp guide shows another complexity trade-off. The condition indices guide checks whether selected coefficients are stable.
Students should calculate ln(n) explicitly and declare the parameter count. Parents can ask whether rows are independent, candidates are comparable and predictive evidence supports the selected model.
- Recalculate the worked example at n=200.
- Compare the penalty per parameter at n=20 and n=1000.
- Explain why clustered rows complicate n.
### Is smaller BIC better? Within a comparable candidate set, yes.
### Is BIC always more conservative than AIC? For ordinary sample sizes above about 7.4, its per-parameter penalty is larger.
### Is it a full Bayesian analysis? No.
Use the Mathematics Learning Hub to connect logarithms, likelihood and parsimony. Compare AIC, BIC and validation error for the same candidates. Record which criterion answers each question.
Extended practice laboratory
Practice case 1: Record n
Setup: Start from the article’s BIC worked example for practice case 1. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 2: Calculate natural log n
Setup: Start from the article’s BIC worked example for practice case 2. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 3: Count parameters
Setup: Start from the article’s BIC worked example for practice case 3. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 4: Calculate penalty
Setup: Start from the article’s BIC worked example for practice case 4. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 5: Record log likelihood
Setup: Start from the article’s BIC worked example for practice case 5. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 6: Calculate fit term
Setup: Start from the article’s BIC worked example for practice case 6. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 7: Calculate BIC
Setup: Start from the article’s BIC worked example for practice case 7. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 8: Compare second model
Setup: Start from the article’s BIC worked example for practice case 8. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 9: Calculate Delta BIC
Setup: Start from the article’s BIC worked example for practice case 9. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 10: Increase sample size
Setup: Start from the article’s BIC worked example for practice case 10. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 11: Add one parameter
Setup: Start from the article’s BIC worked example for practice case 11. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 12: Find required fit gain
Setup: Start from the article’s BIC worked example for practice case 12. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 13: Compare n 20
Setup: Start from the article’s BIC worked example for practice case 13. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 14: Compare n 100
Setup: Start from the article’s BIC worked example for practice case 14. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 15: Compare n 1000
Setup: Start from the article’s BIC worked example for practice case 15. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 16: Check clustered rows
Setup: Start from the article’s BIC worked example for practice case 16. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 17: Check identical observations
Setup: Start from the article’s BIC worked example for practice case 17. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 18: Check likelihood scale
Setup: Start from the article’s BIC worked example for practice case 18. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 19: Compare AIC
Setup: Start from the article’s BIC worked example for practice case 19. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 20: Compare validation error
Setup: Start from the article’s BIC worked example for practice case 20. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 21: Report near tie
Setup: Start from the article’s BIC worked example for practice case 21. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 22: Inspect residuals
Setup: Start from the article’s BIC worked example for practice case 22. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 23: Preserve hierarchy
Setup: Start from the article’s BIC worked example for practice case 23. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 24: Reject posterior-probability claim
Setup: Start from the article’s BIC worked example for practice case 24. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.
Practice case 25: Write a BIC conclusion
Setup: Start from the article’s BIC worked example for practice case 25. Task: Change the named input, recompute all dependent quantities and compare the result with the original.
Check: Check the defining identity, state the data and model conditions, and name one diagnostic or validation step needed before a decision. Show the arithmetic, name the response and predictor units, and separate the numerical result from the claim it supports. Then vary one defensible assumption, calculate again and explain whether the conclusion changed because of the data, the fitted model or the validation purpose. Finish with one compact table or graph that could reveal a failure hidden by the summary.