Find your next learning step
The Conway–Maxwell–Poisson model can represent counts that vary less or more than a Poisson model expects. Its flexibility comes with a price: the rate-like parameter is not generally the mean, and every probability depends on a normalising constant that must be computed carefully.
Open the chapter contents · Probability root · Gold Standard of Education · How X Works
Chapter contents
Choose the route closest to the question, or read from Chapter 1 for the complete argument. The teaching chapters remain open; these grouped routes form the index.
Route 1 · Build the distribution
Route 2 · Work the probabilities
Route 3 · Diagnose and transfer
Route 4 · Check limits and evidence
Route 5 · Ask and continue
CHAPTER 1 OF 12 · BUILD THE DISTRIBUTION
1. Why one dispersion direction is not always enough
Poisson regression ties the conditional variance to the conditional mean. That equality is a model property, not a rule for every count process. Queues with capacity limits, tightly scheduled manufacturing defects and bounded attendance counts can be underdispersed; heterogeneous risk can create overdispersion.
The Conway–Maxwell–Poisson, or COM-Poisson, family adds a dispersion parameter while keeping a count support. It complements the generalized-Poisson guide and the negative-binomial guide. The important difference is not simply that all three are “flexible”. Their parameters, supports, likelihoods and diagnostic failure modes differ.
The practical question is therefore twofold: does a flexible count distribution improve the residual structure, and has its likelihood been evaluated accurately enough for that comparison to mean anything?
CHAPTER 2 OF 12 · BUILD THE DISTRIBUTION
2. The distribution and its normalising constant
For y = 0, 1, 2, …, one common COM-Poisson form is p(Y=y) = λʸ/(y!)ᵛ ÷ Z(λ,ν), with λ > 0 and ν > 0. The normalising constant is Z(λ,ν) = Σⱼ₌₀∞ λʲ/(j!)ᵛ. When ν = 1, Z = exp(λ) and the distribution is Poisson. Values ν > 1 usually produce underdispersion; values below 1 allow overdispersion.
The symbol λ is rate-like but is not generally E(Y). That distinction matters in regression: a one-unit covariate change in log λ is not automatically the same as a log mean ratio. Mean-parameterised COM-Poisson models were developed partly to make the mean relationship more direct.
The infinite sum is not decorative. Every log-likelihood contribution contains log Z. A computation that stops the series too early changes fitted probabilities, gradients and standard errors together.
CHAPTER 3 OF 12 · WORK THE PROBABILITIES
3. Worked probability ledger: λ = 4 and ν = 2
Compute unnormalised weights recursively: w₀ = 1 and wᵧ = wᵧ₋₁ × λ/yᵛ. Recursion avoids repeatedly evaluating factorials. With λ = 4 and ν = 2, the weights fall quickly after y = 4.
| y | Recursive weight wᵧ | Approximate probability wᵧ/Z |
|---|---|---|
| 0 | 1.0000 | 0.0885 |
| 1 | 4.0000 | 0.3539 |
| 2 | 4.0000 | 0.3539 |
| 3 | 1.7778 | 0.1573 |
| 4 | 0.4444 | 0.0393 |
| 5 | 0.0711 | 0.0063 |
| 6 | 0.0079 | 0.0007 |
The resulting mean is about 1.727 and the variance about 1.017, so variance/mean ≈ 0.589. This is an underdispersed example. Notice that λ = 4 is not the mean. Treating it as a mean would misread both the fitted level and any regression coefficient attached to it.
CHAPTER 4 OF 12 · WORK THE PROBABILITIES
4. Regression changes the λ ledger row by row
In the original regression formulation, log λᵢ = xᵢᵀβ while ν may be constant or modelled separately. Each observation has its own normalising constant Z(λᵢ,ν). Maximum likelihood therefore couples the linear predictor to repeated series evaluation or a controlled approximation.
An exposure offset can still be used, but its interpretation follows the chosen parameterisation. If log λᵢ = xᵢᵀβ + log eᵢ, doubling exposure doubles λᵢ, not necessarily the mean exactly. In a mean-parameterised implementation, log μᵢ = xᵢᵀβ + log eᵢ gives the familiar rate interpretation more directly.
Report the software family and parameterisation. “COM-Poisson regression” alone does not tell a reader whether coefficients govern λ, an approximation to μ, or μ itself.
CHAPTER 5 OF 12 · WORK THE PROBABILITIES
5. Normalising-constant checks are part of the model
A safe series calculation uses a stopping rule based on the remaining tail, not a fixed number of terms copied across all λ and ν values. Track the largest term, sum on a stable scale when necessary, and tighten tolerance until coefficients and log likelihood no longer change materially.
For the worked ledger, stopping after y = 3 gives Z ≈ 10.7778 instead of 11.3019, a 4.6% shortfall. Stopping after y = 6 gives 11.3012, much closer. The same truncation index will not be adequate in a high-mean or strongly overdispersed case because the mass can extend much farther.
Analytic approximations can be valuable, but they need a stated validity region and a comparison against a more accurate calculation on representative fitted points. Numerical agreement is an evidence check, not an optional implementation detail.
CHAPTER 6 OF 12 · DIAGNOSE AND TRANSFER
6. Diagnose more than one variance number
Begin with conditional checks. Bin observations by fitted mean or meaningful exposure and compare observed and simulated count distributions. Inspect zero frequency, upper tails, residual spread, and leverage. A global Pearson dispersion statistic can hide a model that fits one mean range and fails another.
| Diagnostic pattern | Possible explanation | Useful next check |
|---|---|---|
| Residual spread below Poisson across mean bins | Genuine underdispersion | COM-Poisson simulation envelope |
| Extra zeros but ordinary positive-count spread | Structural zero process | Hurdle or zero-inflated comparison |
| Heavy upper tail and clusters | Unobserved heterogeneity or dependence | Random effects, time or spatial residuals |
| Dispersion changes with the mean | Constant ν too restrictive | Dispersion regression or stratified checks |
Compare predictive checks under the actual fitted likelihood. A lower information criterion is not enough if the normalising constant was approximated differently across models or if validation observations are dependent on training observations.
CHAPTER 7 OF 12 · DIAGNOSE AND TRANSFER
7. Changed context: the exposure doubles
Suppose two sites share covariates but the second is observed twice as long. Under a mean-parameterised model with a log exposure offset, its expected count doubles while ν controls conditional dispersion around that mean. That is a clean rate statement.
Under a λ-parameterised COM-Poisson model, adding log 2 doubles λ, but the resulting mean must be obtained from the distribution. It may be close to, but is not defined as, twice the original mean. The difference can matter when ν is far from 1.
The transfer check is simple: ask the software for fitted means before and after the exposure change, rather than exponentiating a coefficient and assuming the result. Then simulate counts at both exposures and confirm that the prediction interval changes in the intended direction.
CHAPTER 8 OF 12 · DIAGNOSE AND TRANSFER
8. Compare flexible families by mechanism, not label
Negative-binomial regression is primarily an overdispersion model often motivated by gamma mixing of Poisson rates. Generalized-Poisson regression can cover both dispersion directions but uses a different probability recursion and parameter constraints. COM-Poisson changes the factorial penalty through ν and can approach Bernoulli-like behaviour under strong underdispersion.
These families need not produce the same tail probabilities even when their fitted means and variances are close. If a decision depends on P(Y ≥ k), compare that probability directly. If zeros drive the decision, compare P(Y = 0). Match the diagnostic to the use.
A flexible likelihood cannot repair a missing time trend, a broken exposure measure or dependence between observations. Fit those structures explicitly before treating the residual variance pattern as a distribution choice.
CHAPTER 9 OF 12 · CHECK LIMITS AND EVIDENCE
9. Limits and common misconceptions
ν is not a universal “quality” score; it is conditional on the mean model, covariates and sampling structure. Underdispersion does not prove a controlled mechanism, and overdispersion does not prove latent heterogeneity. Both are patterns requiring explanation.
The orthodox λ regression does not give mean ratios by inspection. A finite truncation is not exact merely because software returned a result. A converged optimiser can converge to the wrong approximate likelihood. Standard errors also depend on the accuracy of derivatives and the adequacy of the independent-observation assumption.
Very sparse data may not identify mean and dispersion effects separately. Boundary-like fits, large standard errors or strong dependence on starting values are reasons to simplify and disclose uncertainty, not to force a two-parameter story.
CHAPTER 10 OF 12 · CHECK LIMITS AND EVIDENCE
10. Evidence and reproducibility route
Sellers and Shmueli’s primary regression paper develops the COM-Poisson GLM, inference and diagnostics: A flexible regression model for count data. Huang develops a mean-parameterised formulation: Mean-parametrized Conway–Maxwell–Poisson regression models for dispersed counts.
A reproducible analysis records the exact probability form, regression links, exposure treatment, normalising-constant algorithm and tolerance, optimiser, starting values, fitted coefficient covariance, and the code used for simulation checks. Save a small grid comparing computed Z values against a tighter reference.
For each published probability, preserve enough digits to reproduce the decision but do not imply more measurement precision than the data support. Numerical accuracy and scientific uncertainty are different ledgers.
CHAPTER 11 OF 12 · ASK AND CONTINUE
11. Frequently asked questions
Is COM-Poisson only for underdispersion?
No. It can represent overdispersion, equidispersion and underdispersion, subject to the chosen parameterisation and valid parameter region.
Why not use quasi-Poisson?
Quasi-likelihood can adjust variance-based inference, but it does not supply a full count probability distribution for tail probabilities or likelihood comparison. The choice depends on the question.
Does ν = 1 prove Poisson regression is correct?
It recovers the Poisson distribution inside this family. The mean structure, independence, exposure and sampling assumptions still need checks.
Can I compare coefficients across COM-Poisson packages?
Only after confirming their parameterisations and link functions. The same label can hide different coefficient meanings.
CHAPTER 12 OF 12 · ASK AND CONTINUE
12. Final transfer check and connected reading
Given a fitted row, reconstruct several unnormalised weights, verify their sum under a tighter tolerance, recover the fitted mean numerically, and compare simulated variance with the claimed dispersion direction. Then change one exposure or covariate and repeat. If the coefficient story and the probability ledger disagree, the interpretation needs repair.
Continue through the Probability root, then compare generalized-Poisson mean–variance diagnostics and negative-binomial exposure offsets. The wider Gold Standard of Education connects the same habit—make the hidden assumption visible—to learning beyond statistics.
Top · Contents · Previous · How X Works