The Gold Standard of Experimental Design | Encouragement Designs, Treatment Uptake and Instrument Strength

1. Randomise the encouragement, not the treatment

Back to contents

Define the eligible population, encouragement, treatment uptake, outcome and follow-up window before randomisation. The primary randomised contrast is the intention-to-treat effect of being offered encouragement. Treatment recipients in encouraged and unencouraged groups are not exchangeable because uptake is chosen. A simple comparison by treatment received loses the protection of randomisation.

2. Map assignment, uptake and outcome

Back to contents

Use three separate variables: Z for randomised encouragement, D for treatment received and Y for outcome. Estimate the effect of Z on D—the first stage—and the effect of Z on Y. Under suitable assumptions, their ratio can estimate a complier average causal effect. The labels matter because an encouragement effect may be important in its own right even when a treatment-effect interpretation is unavailable.

3. State the instrumental-variable assumptions plainly

Back to contents

Randomisation supports independence of encouragement from baseline potential outcomes. Relevance requires encouragement to change uptake. Exclusion requires encouragement to affect the outcome only through treatment, not through reassurance, information or another pathway. Monotonicity rules out people who would take treatment only when not encouraged. Interference must also be considered when one person’s encouragement changes another person’s uptake or outcome.

4. Design an encouragement with a credible exclusion story

Back to contents

A reminder that merely reduces friction may be easier to defend than an educational message that changes health behaviour independently of treatment. Financial incentives can alter stress or spending; personalised coaching can deliver part of the intervention. Pilot uptake and document every component. The strongest encouragement is not necessarily the best instrument if its direct effects make exclusion implausible.

5. Measure instrument strength before promising precision

Back to contents

The first-stage difference in uptake determines how much randomised variation is available for treatment-effect estimation. Weak encouragement yields unstable ratios and wide intervals. Power calculations should model uptake in both arms, clustering and outcome variability, not only the sample size. Report the first stage with uncertainty and avoid treating a conventional test threshold as proof that weakness is solved.

6. Worked example: an invitation to preventive screening

Back to contents

In an invented trial, 1,200 eligible adults are randomised equally to a priority booking invitation or usual information. Screening uptake is 42 percent under encouragement and 22 percent without it, a first stage of 0.20. The mean six-month risk score is 6.8 versus 7.4, so the encouragement intention-to-treat effect is -0.6. The Wald ratio is -0.6 divided by 0.20, or -3.0 points among compliers, if exclusion, monotonicity and other assumptions hold. That is not the effect for everyone screened. If the invitation itself supplied personalised risk advice, exclusion would be doubtful and the ratio would lose this clean interpretation.

7. Keep the estimand local and policy-relevant

Back to contents

Compliers are people whose treatment uptake changes because of this encouragement in this setting. Always-takers and never-takers contribute to the randomised contrasts but not directly to the complier treatment effect. Different encouragements can create different complier groups. Report the encouragement effect, uptake effect and conditional complier effect separately so readers can decide which quantity informs the policy question.

8. Changed-context transfer check: a tutoring voucher

Back to contents

Move from screening to a school voucher that encourages optional tutoring. Random assignment still protects the offer effect, and voucher redemption is the first stage. The exclusion story is harder if the voucher signals that a student is struggling, changes parent attention or can be spent on materials. Peer spillovers and shared teachers create interference. The method transfers, but assumptions must be rebuilt around the educational mechanism.

9. Know the limits and common misconceptions

Back to contents

Random encouragement does not make treatment received random. A strong first stage cannot repair a direct pathway from encouragement to outcome. Monotonicity may fail when encouragement crowds out some existing users. Exclusion is not fully testable from trial data, and the complier effect need not generalise to all eligible people. Two-stage least squares can also mislead with nonlinear outcomes or mishandled standard errors.

10. Audit assignment, uptake and assumption stress tests

Back to contents

Publish randomisation code, allocation concealment, encouragement delivery, contamination, uptake definition, timing, missingness and analysis plan. Plot uptake by arm and prespecified subgroup, retain the encouragement intention-to-treat estimate and report weak-instrument diagnostics suited to the analysis. Test plausible direct effects and interference scenarios. Explain who the compliers could be using observed baseline patterns without claiming to identify each individual.

11. Frequently asked questions

Back to contents

Is encouragement the same as treatment? No. Can I compare users with non-users? Not causally without stronger adjustment. What does the ratio estimate? Under assumptions, an average treatment effect for compliers. Does a larger incentive always help? It can strengthen uptake while weakening exclusion or changing the complier group. Can exclusion be proven? No; it must be defended from design and subject knowledge. Should the offer effect still be reported? Yes.

12. The gold standard keeps the instrument tied to a mechanism

Back to contents

A credible encouragement design begins with a useful randomised offer and a clearly measured uptake pathway. It shows the first stage, states exclusion and monotonicity in ordinary language, keeps the complier estimand local and tests where direct effects or spillovers could enter. Randomisation is the start of the evidence chain. The treatment-effect claim is earned only when the encouragement behaves like the instrument the analysis assumes.