How much sample size does a test spend on local power?
Pitman asymptotic relative efficiency turns two local power calculations into a careful sample-size comparison. This guide derives the ratio, works through the classical normal-location example and shows when the ranking can change.
This guide belongs to The Gold Standard of Probability, within The Gold Standard of Education and the verified How X Works hub.
Choose a reading route
Chapter contents
Open all twelve chapters
- 1. What asymptotic relative efficiency is comparing
- 2. Pitman alternatives move at the square-root rate
- 3. Read the local power curve before taking a ratio
- 4. Derive the sample-size ratio from local slopes
- 5. Worked example: sign, Wilcoxon and mean tests near normality
- 6. Turn efficiency into a planning ledger
- 7. Changed-context transfer: heavier tails can reverse the ranking
- 8. Nuisance parameters and estimated scale change the experiment
- 9. Le Cam’s third lemma supplies the shifted limit
- 10. Limits and common misconceptions
- 11. A practical evidence check for an efficiency claim
- 12. Use ARE as a map, not a verdict
1. What asymptotic relative efficiency is comparing
CHAPTER 1 OF 12 · Orient
Two tests can have the same nominal significance level and still need different sample sizes to detect the same small departure from a null model. Pitman asymptotic relative efficiency, or ARE, compares those local sample-size demands. It is a statement about a sequence of increasingly subtle alternatives, not a universal ranking for every finite data set.
The comparison needs a common target: the same null, the same direction of departure, the same limiting size and the same limiting power. If one test is one-sided and the other two-sided, or if nuisance parameters are handled differently, a numerical ARE can look precise while comparing different jobs.
2. Pitman alternatives move at the square-root rate
CHAPTER 2 OF 12 · Build
A standard local path writes the parameter as θₙ = θ₀ + h/√n. The fixed number h controls direction and local distance, while 1/√n keeps the signal on the same scale as ordinary sampling noise. Under regularity conditions this produces non-degenerate limiting power between the size and one.
If the alternative is fixed away from θ₀, consistent tests all approach power one and their local differences disappear. If it moves much faster than 1/√n, the experiment may become too easy; if it moves much slower, the limit may be indistinguishable from the null. The square-root path is therefore a comparison microscope, not a law for every problem.
3. Read the local power curve before taking a ratio
CHAPTER 3 OF 12 · Build
Suppose a standardised statistic tends under θₙ to a normal limit with mean hκ and variance one. A level-α upper-tail test then has limiting power 1 − Φ(z₁₋α − hκ). The coefficient κ is the local slope in the Gaussian limit: larger κ means the statistic separates nearby alternatives more quickly.
The local power function also exposes sign mistakes. Reversing the score or the alternative changes hκ. A reported efficiency ratio should never be accepted until the two underlying local power curves move in the intended direction.
4. Derive the sample-size ratio from local slopes
CHAPTER 4 OF 12 · Compare
Let tests A and B have local coefficients κ_A and κ_B under matched conventions. To produce the same limiting power, h_A κ_A = h_B κ_B. Because a fixed physical departure δ appears as h = √n δ, equal power requires √n_A κ_A = √n_B κ_B.
Therefore the Pitman efficiency of A relative to B is e(A,B) = (κ_A/κ_B)². If e(A,B)=0.90, test A needs about 1/0.90 times as many observations as B in this local model. The orientation matters: reversing A and B reciprocates the number.
5. Worked example: sign, Wilcoxon and mean tests near normality
CHAPTER 5 OF 12 · Work
For a symmetric normal location model with known scale, the sample-mean test is the local benchmark. Classical Pitman calculations give efficiency 2/π ≈ 0.637 for the sign test relative to the mean test, and 3/π ≈ 0.955 for the Wilcoxon signed-rank test relative to the mean test.
Interpretation: near the normal model and for very small location shifts, the sign test needs about π/2 ≈ 1.571 times the sample size of the mean test for equal limiting power. Wilcoxon needs about π/3 ≈ 1.047 times the sample size. These are asymptotic local conversions, not promises about a particular sample of size twenty.
6. Turn efficiency into a planning ledger
CHAPTER 6 OF 12 · Work
A planning ledger keeps the direction explicit. Start with the benchmark sample size, divide by the efficiency of the candidate relative to the benchmark, and then round up. The result is only a first-order local comparison; real planning still needs the anticipated effect, variance, missingness and design.
The table uses a benchmark requirement of 120 observations only to illustrate the arithmetic. It does not claim that 120 is suitable for a real study.
| Procedure | Efficiency relative to mean | Approximate observations for equal local power |
|---|---|---|
| Mean test | 1.000 | 120 |
| Wilcoxon signed-rank | 3/π ≈ 0.955 | ⌈120/0.955⌉ = 126 |
| Sign test | 2/π ≈ 0.637 | ⌈120/0.637⌉ = 189 |
7. Changed-context transfer: heavier tails can reverse the ranking
CHAPTER 7 OF 12 · Transfer
The normal example does not make the mean test universally best. Rank procedures can have larger local slope under heavy-tailed or logistic-like distributions because they spend less information on extreme magnitudes. A test that loses five per cent near normality may gain under another density.
Transfer check: before carrying an ARE into a new setting, identify the local path and the distribution that generated the slope. If either changes, recompute. The phrase distribution-free may describe null calibration while the efficiency calculation still depends on the alternative density.
8. Nuisance parameters and estimated scale change the experiment
CHAPTER 8 OF 12 · Audit
Unknown scale, regression adjustment, censoring, clustering and semiparametric nuisance functions alter the tangent space and the efficient score. A test can be optimal in a simple parametric model yet lose efficiency when it must estimate nuisance structure.
The right question is not simply which statistic has the largest textbook ARE. Ask whether both procedures use the same observations, the same nuisance information and the same admissible class of alternatives. Local asymptotic normality helps make those assumptions explicit.
9. Le Cam’s third lemma supplies the shifted limit
CHAPTER 9 OF 12 · Connect
The local coefficient κ often comes from a joint null limit of the statistic and the log-likelihood ratio. Le Cam’s third lemma then transports the statistic into the local alternative, usually producing a mean shift equal to a covariance with the central sequence.
This connects efficiency directly to the preceding GS1 guides on local asymptotic normality and the third lemma. ARE is not an isolated rule: it is a sample-size interpretation of how competing statistics respond inside the same limiting experiment.
10. Limits and common misconceptions
CHAPTER 10 OF 12 · Audit
ARE does not compare type-I error after optional stopping, repair model misspecification or guarantee better power at every effect size. It also does not say that a procedure with efficiency above one dominates for every distribution. The result is local, asymptotic and tied to a specified path.
Another common mistake is treating 0.637 as sixty-three-point-seven per cent power. It is a ratio of asymptotic sample-size requirements, derived from squared local slopes. Power itself remains a function of α, h and the limiting distribution.
11. A practical evidence check for an efficiency claim
CHAPTER 11 OF 12 · Audit
A defensible claim should identify the null model, the alternative sequence, the two statistics, the direction of the ratio, the limiting size and the regularity conditions. It should also show either the two local slopes or the resulting local power functions.
For a finite study, add simulation over realistic sample sizes and distributions. Report the Monte Carlo uncertainty and preserve the nominal size before comparing power. A local theorem and a finite-sample simulation answer different questions; they strengthen rather than replace each other.
| Field | Question to answer | Failure mode |
|---|---|---|
| Local path | What is θₙ and which h is fixed? | Comparing different alternatives |
| Calibration | Do both tests have the same limiting size? | Power gain caused by size inflation |
| Orientation | Is e(A,B) or e(B,A) reported? | Reciprocal sample-size error |
| Finite check | Was the theorem tested at realistic n? | Overconfident planning |
12. Use ARE as a map, not a verdict
CHAPTER 12 OF 12 · Use
ARE is most useful when it turns a vague efficiency claim into a traceable chain: choose a local alternative, derive the shifted limit, match power, convert the squared slope ratio into sample size, and then test the approximation in the intended design.
The Gold Standard is to keep both the insight and its boundary. A compact number can guide method choice, but only after the experiment, comparison direction and changed-context checks remain visible.
A changed-context transfer check
A team quotes the normal-model Wilcoxon efficiency 3/π while analysing a strongly skewed response with informative censoring. The correct move is not to multiply the sample size by π/3. Rebuild the local experiment with the censoring and nuisance estimation included, or present the normal number only as a clearly labelled benchmark.
If the recalculated local slopes are κ_A=0.72 and κ_B=0.60, then e(A,B)=(0.72/0.60)²=1.44. A now needs about 1/1.44 of B’s sample size for matched local power. This changed-context result does not contradict the normal example; it answers a different local question.
Frequently asked questions
Is an efficiency of 0.90 the same as 90% power?
No. It is a local asymptotic sample-size ratio. Power depends on the test size, the local parameter and the limiting distribution.
Can ARE be greater than one?
Yes. It means the numerator procedure needs fewer observations than the denominator procedure for matched local limiting power under the specified model and path.
Does distribution-free calibration make ARE distribution-free?
Usually not. Null calibration and local power are separate. The alternative density often enters the slope and therefore the efficiency.
Should I plan a real study from ARE alone?
Use it as a first-order comparison, then verify size and power with a design-specific calculation or simulation at realistic sample sizes.
Authoritative sources and evidence route
- Choi, Hall and Schick: asymptotically uniformly most powerful tests under local alternatives
- University of California, Berkeley: rank-test power and Pitman-efficiency notes
- Local asymptotic normality, central sequences and Gaussian shift limits
- Le Cam’s third lemma, contiguity and local power
- Local asymptotic minimax theorem, bowl-shaped losses and uniform risk