Find your next learning step
Double-Poisson regression keeps a Poisson-like mean structure while introducing a separate precision parameter. Its convenient mean and variance are approximate, so good analysis checks the exact normalisation and fitted moments instead of treating the approximation as an identity.
Open the chapter contents · Probability root · Gold Standard of Education · How X Works
Chapter contents
Choose the route closest to the question, or read from Chapter 1 for the complete argument. The teaching chapters remain open; these grouped routes form the index.
Route 1 · Build the double family
Route 2 · Work the dispersion
Route 3 · Reparameterise and diagnose
Route 4 · Check limits and evidence
Route 5 · Ask and continue
CHAPTER 1 OF 12 · BUILD THE DOUBLE FAMILY
1. Why double-Poisson regression exists
Ordinary Poisson regression fixes conditional variance equal to conditional mean. Efron’s double-exponential construction adds a precision parameter while keeping the familiar count support and a mean-like location parameter. In the double-Poisson case, precision above one usually compresses the count distribution; precision below one spreads it.
The attraction is practical. A log-linear predictor can describe the location while another predictor describes dispersion. The caution is equally practical: the usual statements E(Y) ≈ μ and Var(Y) ≈ μ/θ are approximations, not defining equalities.
This article follows the Probability root and complements COM-Poisson normalising-constant checks and generalized-Poisson diagnostics.
CHAPTER 2 OF 12 · BUILD THE DOUBLE FAMILY
2. Location, precision and the approximate moments
Write μ > 0 for the location and θ > 0 for precision. The double-Poisson probability is obtained by applying the double-exponential-family construction to a Poisson density and then normalising over y = 0,1,2,….
Near ordinary operating ranges, E(Y) is close to μ and Var(Y) is close to μ/θ. Thus θ = 1 recovers the Poisson direction, θ > 1 suggests underdispersion, and θ < 1 suggests overdispersion. Some software instead reports dispersion φ = 1/θ, giving Var(Y) ≈ φμ.
Those opposite conventions create an easy reporting error. A value 0.6 can mean overdispersion when it is θ and underdispersion when it is φ. Always name the parameter, equation and link rather than writing “dispersion = 0.6” alone.
CHAPTER 3 OF 12 · WORK THE DISPERSION
3. Worked count ledger: mean six, variance four
Take eight invented counts: 3, 8, 6, 5, 7, 4, 9 and 6. Their sample mean is 6 and sample variance is 4, giving a variance-to-mean ratio of 4/6 = 0.667.
| Quantity | Calculation | Result |
|---|---|---|
| Number of observations | n | 8 |
| Sample mean | Σy/n | 6.000 |
| Sample variance | Σ(y−ȳ)²/(n−1) | 4.000 |
| Variance/mean | 4/6 | 0.667 |
| Moment precision estimate | θ ≈ μ/Var | 1.500 |
The ledger points toward underdispersion. It does not establish that the double-Poisson likelihood is correct, because time dependence, bounded counts, omitted covariates or data filtering can also reduce residual spread.
CHAPTER 4 OF 12 · WORK THE DISPERSION
4. Regression separates the two linear predictors
A common structure uses log μᵢ = xᵢᵀβ for the count level and log θᵢ = zᵢᵀγ for precision. A constant θ gives one dispersion level. A dispersion regression lets spread change with batch, device, season or another measured condition.
An exposure offset belongs in the mean-like predictor when the scientific target is a rate: log μᵢ = xᵢᵀβ + log eᵢ. Because μ is only approximately the exact mean, verify fitted expectations numerically before interpreting exp(β) as an exact mean ratio.
Keep the precision predictor parsimonious. Mean and dispersion effects can imitate one another in small samples, especially when covariates overlap or counts occupy a narrow range.
CHAPTER 5 OF 12 · WORK THE DISPERSION
5. Reparameterisation must preserve the same model
Suppose software A reports θ = 1.5. Software B reports φ = 1/θ = 0.667. They describe the same approximate variance μ/θ = φμ = 4 when μ = 6.
| Reported parameter | Underdispersion direction | Approximate variance at μ = 6 |
|---|---|---|
| Precision θ = 1.5 | θ > 1 | 6/1.5 = 4 |
| Dispersion φ = 0.667 | φ < 1 | 0.667×6 ≈ 4 |
| Log precision log θ | positive | exp(log θ) used in denominator |
| Log dispersion log φ | negative | exp(log φ) used as multiplier |
When comparing packages, transform both estimates and covariance matrices. Negating a log-dispersion coefficient changes its sign; standard errors stay numerically equal only for that simple one-to-one log transformation.
CHAPTER 6 OF 12 · REPARAMETERISE AND DIAGNOSE
6. Approximate likelihood needs an exact-sum audit
The double-Poisson construction includes a normalising factor. Efron supplied a close approximation for broad use, but modern computation can sum unnormalised probabilities over a sufficiently wide count range and obtain an exact numerical normaliser for each fitted row.
Fit the approximate likelihood, then choose representative rows: the smallest and largest μ, the most extreme θ, and a typical row. Recompute exact probabilities until the omitted tail is negligible. Compare log likelihood, fitted mean, variance and decision-relevant tail probabilities.
If the approximate and exact fits lead to different rankings or intervals, report the exact calculation or tighten the operating region. Optimiser convergence cannot validate an inaccurate normalisation.
CHAPTER 7 OF 12 · REPARAMETERISE AND DIAGNOSE
7. Changed context: the mean moves into the tail
Suppose the fitted model is comfortable around μ = 6 and θ = 1.5, then a covariate change predicts μ = 60 at the same precision. A fixed truncation at y = 30 that worked for the first setting now discards central probability mass.
The transfer test is to generate the count grid around the new location, not reuse the earlier upper bound. Sum probabilities on a stable log scale, expand both tails until the remainder is controlled, and recover the numerical mean and variance.
This is why normalisation checks should follow fitted parameter ranges rather than a single demonstration point. Extrapolation changes the computational problem as well as the scientific one.
CHAPTER 8 OF 12 · REPARAMETERISE AND DIAGNOSE
8. Diagnose the mechanism, not only the ratio
Plot residual spread against fitted μ and exposure. Simulate replicated counts from the fitted model and compare zeros, central concentration and upper tails. A single Pearson dispersion statistic can average over opposite patterns.
Underdispersion may come from quotas, scheduling, negative dependence or bounded capacity. Overdispersion may come from latent heterogeneity or clustering. A distribution that matches variance without the right dependence can still produce misleading uncertainty.
Compare predictive performance with negative-binomial rate models when overdispersion dominates. For bidirectional flexibility, compare exact probabilities rather than family labels.
CHAPTER 9 OF 12 · CHECK LIMITS AND EVIDENCE
9. Limits and common misconceptions
μ is not necessarily the exact expectation, and θ is not a causal measure of process control. A fitted θ above one does not prove that a production system actively suppresses variation.
An approximate likelihood is not automatically poor. It becomes a problem when its error is material for the fitted range or decision. Conversely, an exact normaliser does not repair an incorrect mean structure, dependence or exposure.
Sparse data may not identify a dispersion regression. Boundary estimates, unstable standard errors or large changes under starting values signal that a simpler model or stronger evidence is needed.
CHAPTER 10 OF 12 · CHECK LIMITS AND EVIDENCE
10. Evidence and reproducibility route
Efron introduced double exponential families and their use in generalised linear regression in Double Exponential Families and Their Use in Generalized Linear Regression. The primary paper explains the second parameter, regression construction and approximate calculations.
Archive the exact density convention, whether θ or φ is reported, mean and dispersion links, exposure treatment, normalising algorithm, truncation tolerance, optimiser, starting values and simulation diagnostics. Preserve a small table comparing approximate and exact log probabilities at fitted extremes.
Reproduce the original count ledger independently. A numerical result becomes trustworthy when another reader can rebuild both the descriptive ratio and the likelihood calculation.
CHAPTER 11 OF 12 · ASK AND CONTINUE
11. Frequently asked questions
Is double-Poisson the same as generalized Poisson?
No. Both can model non-Poisson dispersion, but their probability laws, parameters and constraints differ.
Does θ = 1 recover Poisson?
Yes in the double-Poisson construction, subject to the same mean-model and independence assumptions.
Can I read exp(β) as an exact mean ratio?
Treat it as the ratio of μ parameters unless the implementation verifies that μ is the exact mean.
Why compute exact probabilities after fitting an approximation?
It checks whether approximation error matters for coefficients, tails and decisions in the observed parameter range.
CHAPTER 12 OF 12 · ASK AND CONTINUE
12. Final transfer check and connected reading
Recalculate the sample mean, variance and θ ≈ 1.5. Convert to φ = 0.667 and confirm that both parameterisations give approximate variance four. Then move μ from 6 to 60 and explain why the probability grid and normalisation check must change.
Continue through the Probability root, COM-Poisson regression, generalized-Poisson diagnostics and negative-binomial exposure offsets. The comparison separates a useful variance pattern from a correctly interpreted probability model.
Top · Contents · Previous · How X Works