The Gold Standard of Experimental Design | Regression Discontinuity, Bandwidth Choice and Manipulation Checks

1. Map the assignment rule before fitting a model

Back to contents

Identify the running variable, cutoff, unit of assignment, treatment rule, compliance pattern and timing. In a sharp design, treatment status changes deterministically at the cutoff. In a fuzzy design, the probability of treatment jumps but some units do not follow the rule. Eligibility is not always receipt, and administrative exceptions can turn an apparently sharp rule into a fuzzy or invalid design. Preserve the original scale and coding direction, verify the rule against records and plot treatment probability near the threshold. Causal analysis starts with how decisions were actually made, not how policy documents summarise them.

2. State the effect as local

Back to contents

The identifying comparison concerns units near the cutoff. A scholarship effect estimated around a score of 70 may not describe students scoring 40 or 95, different years or another institution. In a fuzzy design, the estimand is local to compliers whose treatment responds to the threshold, under additional instrumental-variable assumptions. Report the population, window and treatment contrast rather than calling the result “the effect” without qualification. Local credibility is an advantage, not a defect, but it should not be silently converted into broad external validity.

3. Make the continuity assumption concrete

Back to contents

The central idea is that, without treatment, expected outcomes and relevant potential-outcome determinants would change smoothly through the cutoff. Units just below and above should face the same local environment except for the treatment jump. Continuity cannot be tested directly for unobserved potential outcomes. It becomes more plausible when the rule was not precisely manipulable, no other programme changes at the same value and baseline covariates show no systematic breaks. Institutional knowledge matters as much as a smooth fitted curve.

4. Choose bandwidth for the local trade-off

Back to contents

A bandwidth sets how far from the cutoff observations contribute to the estimate. A wider window uses more data and can improve precision, but it relies on the regression function over a larger range and increases bias. A narrower window is more local but may be noisy and sensitive to individual observations. Use a principled, prespecified or data-driven bandwidth rule suited to the estimand, then report sensitivity across credible alternatives. Do not search bandwidths until significance appears. The bandwidth is part of the design claim, not a cosmetic plotting choice.

5. Prefer transparent local fits

Back to contents

Local linear regression on each side of the cutoff is often a useful default because it focuses estimation near the boundary. Kernel weights can give closer observations more influence. High-order global polynomials may oscillate, assign leverage far from the cutoff and create artificial jumps; a visually flexible curve is not automatically a credible one. Report polynomial order, kernel, bandwidth and whether slopes differ across sides. Robust bias-corrected intervals can address leading smoothing bias, but they do not repair invalid assignment or hidden co-interventions.

6. Worked example: a scholarship threshold at 70

Back to contents

A scholarship is offered when an audited score reaches 70. Students scoring 68–69 and 70–71 appear similar before the award, and acceptance probability jumps from 8% to 82% at the cutoff. A wide bandwidth of 20 points suggests a large graduation effect, but the score–outcome relationship curves strongly below 60. A local linear estimate within an optimally chosen narrower window is smaller and less precise; robust bias correction widens the interval. The team reports both results, treatment uptake and the local interpretation. It does not prefer the wide estimate merely because it is significant or describe eligibility as universal receipt.

7. Investigate sorting and manipulation

Back to contents

If participants, administrators or measurement systems can precisely move scores across the threshold, near-cutoff groups may differ for reasons beyond treatment. Examine institutional procedures, score heaping, appeals, rounding and opportunities for discretion. A density test can flag an unusual change in the distribution of the running variable near the cutoff, but failure to reject is not proof of no manipulation and a density jump may have an innocent administrative explanation. Combine formal tests with histograms at sensible bins, record audits and analyses around heaped or contested values.

8. Use covariates and placebos as diagnostics

Back to contents

Predetermined covariates should generally evolve smoothly through a valid cutoff. Plot and estimate discontinuities for age, prior outcomes and other baseline measures without turning every noisy difference into a verdict. Test placebo cutoffs where treatment does not change and outcomes that treatment could not yet affect. A donut analysis that removes observations immediately around the threshold can probe suspected manipulation, but it changes the target and may discard the design’s most informative units. Diagnostics build a pattern of evidence; none alone certifies validity.

9. Changed-context transfer check: age-based eligibility

Back to contents

Move from an exam score to a policy that starts at age 65. Date of birth is difficult to manipulate, which may strengthen assignment credibility, but other programmes may begin at the same age and behaviour may change in anticipation. The running variable can also be recorded discretely in months or years, creating mass points. The transferable workflow is to map the exact rule, identify co-interventions, choose a local window and inspect continuity. The specific threats change even though the cutoff design looks formally similar.

10. Handle discrete scores and mass points honestly

Back to contents

Running variables often take only integer values, and many units may share a score. Treating thousands of observations as if they provide thousands of distinct local support points can understate uncertainty. Display the distribution, report how many unique values lie in each bandwidth and use inference appropriate to discrete support or clustering when needed. Rounding can blur which side a unit truly belongs to. If there are too few support points near the cutoff, sophisticated software cannot manufacture the local variation the design requires.

11. Audit every analyst choice

Back to contents

Archive the assignment rule, exceptions, raw running variable, treatment receipt, outcome timing, exclusions, manipulation evidence, covariates, code and analysis plan. Present the treatment-probability jump, outcome scatter or binned means, local fits and observation counts by side. Report bandwidth selection, kernel, polynomial order, conventional and robust bias-corrected intervals, and sensitivity to nearby bandwidths, donut exclusions and plausible clustering. Readers should be able to tell which findings were planned, which were diagnostic and whether the conclusion rests on one convenient specification.

12. The gold standard earns a local causal claim

Back to contents

Regression discontinuity is compelling when a real assignment threshold creates units that are comparable in a narrow neighbourhood. Credibility comes from the rule and institutional process, not from a vertical line added to a graph. The gold standard verifies treatment uptake, limits the estimand, balances bias and precision transparently, examines manipulation and co-interventions, and shows robustness without specification fishing. When those links hold, a local jump can support causation. When they do not, a polished curve is only a discontinuous description.