The Gold Standard of Experimental Design | Missing Outcomes, Intercurrent Events and Estimand Alignment

1. Start with the effect, not the missing-data method

Back to contents

An estimand describes the treatment effect to be estimated: treatment conditions, population, outcome variable, handling of intercurrent events and population-level summary. It turns a broad intention such as “Does the programme work?” into a question that data and analysis can address.

Choosing last observation carried forward, multiple imputation or a mixed model before defining the estimand reverses the logic. The same method can answer different questions under different assumptions, and different methods can target the same question.

2. Missing outcomes and intercurrent events are different

Back to contents

A missing outcome is a value that would be meaningful for the estimand but was not observed. An intercurrent event occurs after treatment starts and affects the interpretation or existence of measurements: rescue medication, discontinuation, switching or a terminal event are common examples.

An outcome after rescue may be fully observed yet answer a different question from the no-rescue outcome. Death may make a later functional score nonexistent, not merely unrecorded. Classify the event before deciding that the spreadsheet has a blank to fill.

3. Name the strategy for each intercurrent event

Back to contents

A treatment-policy strategy uses outcomes regardless of the event. A hypothetical strategy targets a stated scenario in which the event would not occur. A composite strategy incorporates the event into the outcome. A while-on-treatment strategy restricts the relevant observation period, and a principal-stratum strategy targets people defined by potential event status.

Different events in one trial can use different strategies. The choice follows the clinical, educational or policy question. It should not be selected because one approach produces a more favourable contrast.

4. Design follow-up around the estimand

Back to contents

If the treatment-policy question needs outcomes after discontinuation or rescue, continue follow-up even when assigned treatment stops. Separate consent for study measurement from adherence to treatment. Record event timing, reason, additional interventions and outcome windows.

Reduce preventable missingness with accessible visits, flexible modes and participant-centred retention. Do not make post-event observation contingent on a site’s belief that the participant is “off study.” The protocol should say which data remain valuable after each event.

5. Align estimator and assumptions with the question

Back to contents

The estimator is the analysis procedure used to estimate the estimand. Its assumptions include outcome model form, missingness mechanism, treatment adherence information and how post-event outcomes are used or predicted.

An intention-to-treat label does not specify an intercurrent-event strategy. A mixed model does not automatically produce a treatment-policy estimate if post-discontinuation outcomes are absent. State why the available data and model identify the target effect.

6. Worked example: rescue is not automatically missing

Back to contents

Two hundred participants are randomised equally to a new therapy or control. By week 12, 30 in the new-therapy arm and 10 in control have used rescue medication. Week-12 outcomes are collected for 92 and 94 participants respectively, including many after rescue.

A treatment-policy estimand can use those observed outcomes regardless of rescue and needs assumptions for the remaining missing values. A hypothetical no-rescue estimand must model outcomes under a clearly stated world without rescue. A composite estimand might define rescue as failure if that is clinically meaningful. Deleting all post-rescue values would create avoidable missingness and would not, by itself, estimate any of these effects.

7. Sensitivity analysis tests the fragile assumptions

Back to contents

The primary analysis should be valid under its stated assumptions and aligned with the estimand. Sensitivity analyses vary the assumptions that cannot be verified from observed data, especially departures from missing-at-random models or assumptions about outcomes after intercurrent events.

Use clinically interpretable departures: delta adjustments, tipping points, reference-based assumptions or bounds where appropriate. Supplementary analyses can target different estimands, but they should not be mislabeled as sensitivity analyses for the original question.

8. Changed-context transfer check

Back to contents

Move the framework to an education trial comparing tutoring with usual support. A student may switch schools, receive private tuition, stop attending sessions or sit a different examination. These events affect meaning differently.

A policy question may include private tuition as part of real-world uptake; a hypothetical question may ask what would happen without it. Moving school may leave an outcome measurable through records, while taking a different exam may require a harmonised outcome definition. The strategies transfer; the substantive choices must be rebuilt.

9. Limits and common misconceptions

Back to contents

No statistical method recovers information without assumptions. Multiple imputation does not cure informative missingness merely by producing several complete datasets. Complete-case analysis can change the target population and lose randomisation-based comparability.

Do not code death as an ordinary missing score, treat treatment discontinuation as outcome withdrawal, or use a composite whose components have no coherent meaning together. Principal strata concern potential event status under treatments, not the observed subset who tolerated one assigned arm.

10. Audit question, data and analysis as one chain

Back to contents

Compare protocol, registry, analysis plan and report. For each intercurrent event, trace the prespecified strategy, data collected after the event, estimator and sensitivity assumptions. Reconcile every deviation and timing change.

Plot event incidence and missingness by arm, time and reason. Check whether follow-up differs after discontinuation. Reproduce the primary estimate and prespecified sensitivity range. A result is fragile when modest, credible departures reverse its interpretation.

11. Frequently asked questions

Back to contents

Is treatment policy the same as intention to treat?

It is one strategy that may reflect an intention-to-treat question for particular events, but a full estimand also specifies population, variable and summary.

Should all post-event outcomes be collected?

Collect outcomes needed for the estimand, subject to ethics and burden. Stopping treatment need not stop outcome follow-up.

Can a trial have several estimands?

Yes. A primary estimand and clearly labelled supplementary estimands can answer distinct decisions, provided their hierarchy and analyses are prespecified.

12. The gold standard keeps the question stable

Back to contents

Post-randomisation events do not merely complicate a clean experiment; they help define the treatment effect people actually need to understand. The estimand framework makes those choices visible before analysis.

Before trusting a result, ask which event changed interpretation, which outcome remained meaningful, what data were still collected, and whether the estimator and sensitivity analyses target the same question. Alignment is what turns a table of incomplete outcomes into a defensible effect.