Find your next learning step
Start with the cluster mechanism, then test whether the fitted mean, variance and zero probability tell the same story.
Open the chapter contents · Probability root · Gold Standard of Education · How X Works
Chapter contents
Choose the route closest to the question, or read from Chapter 1 for the complete argument. The teaching chapters remain open; these grouped routes form the index.
Route 1 · Build the mechanism
Route 2 · Work the numbers
Route 3 · Fit and transfer
Route 4 · Check the limits
Route 5 · Verify and continue
CHAPTER 1 OF 12 · BUILD THE MECHANISM
1. Why a Poisson-cluster model is different
An ordinary Poisson model treats recorded events as if they arrive independently and one at a time. A Neyman Type A model begins one level earlier. Unobserved parent events arrive, and each parent produces a random cluster of recorded children. The final count is the total number of children, not the number of parents.
This distinction matters when one cause can create several observations: a contamination source releases several particles, one infection event generates a local cluster, or one production disturbance creates several defects. The extra variation is then attached to a mechanism rather than added as a free dispersion label.
Use the model beside the Probability root, Hermite dual-Poisson model and negative-binomial exposure model. It is a good candidate when cluster production is plausible; it is not a default answer to every large variance.
CHAPTER 2 OF 12 · BUILD THE MECHANISM
2. The two-stage construction determines the probability law
Let N follow a Poisson distribution with mean λ. Conditional on N, let the cluster sizes X₁,…,Xₙ be independent Poisson variables with mean μ. The observed count is Y = X₁ + ··· + Xₙ, with Y = 0 when N = 0.
The probability-generating function is G(s)=exp{λ[exp(μ(s−1))−1]}. Differentiating it gives E(Y)=λμ and Var(Y)=λμ(1+μ). The variance-to-mean ratio is therefore 1+μ. Unlike the ordinary two-component Hermite model, the Neyman Type A distribution does not stop at a ratio of two.
The parameters have separate jobs. λ controls how often parent clusters arise. μ controls the average number of recorded children per parent and therefore the amount of clustering. A fitted mean alone identifies only the product λμ; the rest of the count distribution is needed to distinguish the two.
CHAPTER 3 OF 12 · WORK THE NUMBERS
3. Worked ledger: three parents with two children on average
Take λ = 3 and μ = 2. The expected count is 3×2 = 6. The variance is 3×2×3 = 18, and the variance-to-mean ratio is 3. The probability of zero is exp{3(exp(−2)−1)} = 0.07472.
| Quantity | Calculation | Result |
|---|---|---|
| Parent-cluster rate | λ | 3.000 |
| Mean cluster size | μ | 2.000 |
| Expected observed count | λμ | 6.000 |
| Variance | λμ(1+μ) | 18.000 |
| Variance-to-mean ratio | 1+μ | 3.000 |
| Probability of zero | exp{λ(exp(−μ)−1)} | 0.07472 |
| Probability of one | P(0)λμexp(−μ) | 0.06067 |
| Probability of two | compound-Poisson coefficient | 0.08531 |
The zero probability is much larger than a Poisson count with mean six, whose zero probability is exp(−6)=0.00248. That difference is not a separate zero-inflation parameter. It arises because a Poisson parent can produce no children and because no parent may arrive at all.
CHAPTER 4 OF 12 · WORK THE NUMBERS
4. Recovering λ and μ from a mean and variance
The moment equations can be inverted when the variance exceeds the mean. If m is the mean and v the variance, then μ = v/m−1 and λ = m/μ. For m = 6 and v = 18, μ = 2 and λ = 3.
| Mean m | Variance v | Implied μ | Implied λ | Interpretation |
|---|---|---|---|---|
| 6 | 6 | 0 | undefined limit | Poisson boundary, no identifiable cluster size |
| 6 | 9 | 0.5 | 12 | Many small parent clusters |
| 6 | 18 | 2 | 3 | Worked interior case |
| 6 | 30 | 4 | 1.5 | Fewer, larger clusters |
| 6 | 5 | −0.167 | invalid | Underdispersion contradicts the model |
These are diagnostic values, not automatic maximum-likelihood estimates. Near μ = 0, λ must become large to hold the mean fixed, so the separate parameters become weakly identified even though the fitted count law approaches Poisson.
CHAPTER 5 OF 12 · WORK THE NUMBERS
5. Variance functions expose what the regression assumes
Because Var(Y)=E(Y)(1+μ), the variance function depends on how μ changes with covariates. If μ is constant, variance grows linearly with the mean. If both λ and μ vary, the mean–variance relationship can change across observations.
A common construction uses log λᵢ = xᵢᵀβ and log μᵢ = zᵢᵀγ. Log links keep both parameters positive. Covariates may describe parent opportunities, cluster propagation or both. When exposure changes the number of parent opportunities, log exposure belongs in the λ predictor. It should not automatically be copied into μ.
Plot fitted means against fitted variances and fitted μ values. A model that matches the total variance by forcing implausibly large clusters is not scientifically satisfactory. The predicted cluster mechanism should remain credible across the covariate range.
CHAPTER 6 OF 12 · FIT AND TRANSFER
6. Regression coefficients need mechanism-aware interpretation
A coefficient in the λ equation changes the expected number of parent clusters, holding μ fixed. A coefficient in the μ equation changes both the mean count and the dispersion ratio. The same numerical coefficient therefore has different consequences depending on its equation.
Suppose a binary condition doubles λ but leaves μ = 2 unchanged. Mean and variance both double from 6 and 18 to 12 and 36, while their ratio remains three. If the condition doubles μ instead, the mean becomes 12 but the variance becomes 60 because 12×(1+4)=60.
Report predicted λ, μ, mean, variance and a few event probabilities at meaningful settings. A single exponentiated coefficient is not a complete description of the change in the observed count distribution.
CHAPTER 7 OF 12 · FIT AND TRANSFER
7. Changed context: larger clusters, not more parents
Start from λ = 3 and μ = 2. Now suppose an intervention leaves the parent rate at three but reduces the mean cluster size to one. The expected count falls from six to three, the variance falls from eighteen to six, and the dispersion ratio falls from three to two.
The probability of zero becomes exp{3(exp(−1)−1)} = 0.150. Fewer children per parent can increase the chance of observing no children even though the parent rate is unchanged. This is a useful transfer lesson: lower mean and weaker clustering do not move every probability in the same intuitive direction.
Repeat the comparison through simulation or exact recursion. Check zeros, small counts and the upper tail, not just mean and variance. A model transfers when the whole predictive distribution remains plausible under the changed mechanism.
CHAPTER 8 OF 12 · FIT AND TRANSFER
8. Diagnostics should separate cluster size from tail repair
Compare observed and replicated counts within fitted-mean bands. Inspect zeros, ones, moderate counts and tail exceedances. Examine whether residual variance follows the fitted μ pattern and whether a few extreme observations dominate the cluster-size equation.
Use probability-integral-transform or randomized-quantile diagnostics suitable for discrete data. Check held-out log scores and calibration, not only in-sample deviance. Compare with generalized-Poisson regression, COM-Poisson regression and the negative binomial.
If another family predicts better, ask why. A negative binomial may describe persistent rate heterogeneity; Neyman Type A describes Poisson parent clusters with Poisson child counts. Similar mean–variance curves do not make those mechanisms interchangeable.
CHAPTER 9 OF 12 · CHECK THE LIMITS
9. Limits and common misconceptions
The model does not reveal individual clusters from aggregate counts. λ and μ describe a latent construction, and several mechanisms can produce similar marginal distributions. Causal language requires external design or process knowledge.
Neyman Type A cannot represent underdispersion. It also does not automatically handle serial dependence, spatial correlation, truncation, censoring, reporting heaping or a changing observation window. These features need explicit treatment.
An excess of zeros is not by itself proof of zero inflation. The compound-Poisson mechanism naturally creates zeros. Conversely, adding the model merely to absorb zeros can distort tail predictions if no cluster process is plausible.
CHAPTER 10 OF 12 · CHECK THE LIMITS
10. Evidence, computation and reproducibility
Neyman’s original biological-motivation paper introduced Type A, B and C compound distributions in On a new class of contagious distributions. Shenton examined estimation efficiency in On the efficiency of the method of moments and Neyman’s Type A distribution. Skellam later discussed derivation and applicability in On the derivation and applicability of Neyman’s Type A distribution.
Archive the exact parameterisation, offset placement, likelihood evaluation, recursion or series truncation, starting values, convergence diagnostics and uncertainty method. Record fitted ranges for λ and μ. Reproduce the λ = 3, μ = 2 probabilities independently by summing over possible parent counts and by using the generating-function coefficients.
Agreement between two implementations is especially valuable because a compound distribution can look numerically stable while a truncated sum has silently omitted material tail probability.
CHAPTER 11 OF 12 · VERIFY AND CONTINUE
11. Frequently asked questions
Is Neyman Type A the same as the negative binomial?
No. Both can be overdispersed, but Neyman Type A is a Poisson sum of Poisson cluster sizes. The negative binomial is commonly derived from Poisson rates mixed by a gamma distribution.
Is it the same as Hermite regression?
No. Hermite uses singleton and doublet Poisson components, so its ordinary form has a variance-to-mean ratio between one and two. Neyman Type A permits Poisson cluster sizes and can exceed two.
Can λ and μ be recovered from the mean alone?
No. The mean identifies only λμ. Variance and distributional shape are needed to separate the parent rate from cluster size.
What happens near μ = 0?
The distribution approaches a Poisson limit while λ becomes large for a fixed mean. Separate parameter estimates can then be unstable.
CHAPTER 12 OF 12 · VERIFY AND CONTINUE
12. Final transfer check and connected reading
Recalculate the worked ledger for λ = 3 and μ = 2. Then keep the mean fixed at six while changing μ to 0.5 and four. Recover the corresponding λ values, calculate the variances, and explain why the same mean can hide very different cluster structures.
Continue through the Probability root, Hermite count regression, double-Poisson approximation checks, generalized-Poisson diagnostics and negative-binomial exposure models. Together they turn overdispersion from a label into a testable choice among mechanisms.
Top · Contents · Previous · How X Works