The Gold Standard of Experimental Design | Negative Controls, Falsification Tests and Residual Bias

1. A negative control targets a bias pathway

Back to contents

Suppose an observational study links a medication with fewer injuries. Health-seeking behaviour could affect both medication use and injury reporting. A negative control outcome that the medication cannot plausibly change, but that shares the reporting pathway, can reveal that bias.

The control is not “negative” because its estimated association happens to be small. It is selected before analysis because causal knowledge rules out the target effect while preserving exposure to the suspected bias.

2. Distinguish negative control outcomes and exposures

Back to contents

A negative control outcome should not be caused by the exposure of interest but should share relevant confounding or measurement processes. A negative control exposure should not cause the target outcome but should share the bias structure of the main exposure.

Draw a causal diagram and state the exclusion assumptions. If the control can affect the target through another pathway, an association may be real rather than a falsification.

3. Prespecify the threat and expected pattern

Back to contents

Name the threat first: unmeasured health status, site-level practice, calendar trend, recall bias or laboratory batch. Then choose a control sensitive to that threat and define the expected direction, timing and effect scale.

A long list of candidate controls searched after the primary result invites selective reporting. Register the variables, models, multiplicity handling and decision rule before inspecting associations.

4. A failed falsification test is a diagnosis

Back to contents

If the exposure is associated with a valid negative control outcome, at least one design or modelling assumption is doubtful. Investigate selection, measurement, functional form, interference and confounding.

The test does not identify which pathway failed, and deleting the control does not solve the problem. Revise the design, estimate sensitivity, collect missing measurements or narrow the causal claim.

5. A passed test does not prove no bias

Back to contents

The control may be weakly related to the hidden confounder, measured imprecisely or too rare for useful power. A null estimate can therefore coexist with important residual bias in the target association.

Report confidence intervals and the smallest bias signal the design could detect. Avoid language such as “the negative control confirmed validity.” It found no detectable trace through one chosen channel.

6. Worked example: one control exposes a calendar trend

Back to contents

A programme begins in July. The primary outcome improves by 6 points after participation. Researchers choose a pre-programme outcome measured in May as a negative control outcome because July participation cannot cause an earlier score.

The same model estimates a 3-point association with the May score, suggesting participants were already on a different trajectory or selected differently. Subtracting 3 from 6 is not automatically valid. The control diagnoses residual selection; quantitative correction requires additional structural assumptions.

7. Separate detection from bias correction

Back to contents

Simple falsification uses controls to detect a violation. Calibration and double-negative-control methods can estimate or correct bias only when stronger assumptions connect the control association to the target bias.

State relevance conditions, exclusion restrictions, bridge or comparability assumptions and model form. A numerically corrected estimate is not automatically safer than a transparent sensitivity range.

8. Changed-context transfer check

Back to contents

Move the example to a randomised school trial in which pupils share materials across classes. Baseline confounding is reduced by randomisation, but spillover can contaminate the contrast. A negative control outcome measured before assignment cannot detect later spillover.

A control sensitive to cross-class exposure or a network measure may be more relevant. Transfer the threat-to-control logic; do not reuse a control selected for a different bias mechanism.

9. Limits and common misconceptions

Back to contents

Placebo treatment groups, laboratory blanks, manipulation checks and negative control variables have related names but different jobs. A pre-treatment outcome may be caused by earlier versions of an exposure and fail the exclusion condition. Multiple controls create multiplicity and interpretation problems.

Do not choose a control because it yields p greater than 0.05. Do not treat balance on one control as balance on all unmeasured causes. Do not correct bias with simple subtraction unless the scale and assumptions justify it.

10. Audit the control as carefully as the outcome

Back to contents

Check timing, causal exclusion, shared bias pathway, measurement quality, missingness, sample overlap, model specification and power. Compare the planned control with every reported one.

Use simulations or sensitivity analysis to ask whether plausible confounding would move both associations as expected. Report controls that fail, controls that pass and controls abandoned with reasons.

11. Frequently asked questions

Back to contents

Is a placebo test the same as a negative control?

It can serve a similar falsification role, but the exact variable and causal assumptions must be stated rather than relying on the label.

Should a negative control association be exactly zero?

Under the intended exclusion it should have no target causal effect, but sampling variation, measurement and other pathways affect the observed estimate.

Can negative controls repair an observational study?

They can diagnose bias and, under stronger assumptions, support calibration or correction. They do not automatically create randomisation.

12. The gold standard tries to falsify the design honestly

Back to contents

A useful negative control is a deliberate opportunity for the study to fail. It is tied to a named threat, measured well and interpreted with the same care as the primary outcome.

Before calling the result reassuring, ask which hidden biases the control could reveal, which it could miss and how large a signal the study was capable of seeing.