Adaptive randomisation changes future assignment probabilities using accumulating outcomes. That can be useful, but time becomes part of the design.
This guide explains the mechanism, works a numerical example and shows how pre-specification, simulation, contemporaneous controls and time-aware analysis protect inference.
Choose a reading route
Understand adaptation and bias
Design, simulate and calculate
Transfer and define limits
Audit and source the trial
Chapter contents
- 1. What adaptive randomisation changes
- 2. Why time trends matter
- 3. Routes to allocation bias
- 4. Pre-specify the adaptive rule
- 5. Simulate plausible worlds
- 6. Worked example: improvement over time
- 7. Match analysis to randomisation
- 8. Changed-context transfer check
- 9. Limits and misconceptions
- 10. Frequently asked questions
- 11. Practical design audit
- 12. Sources and reporting template
1. What adaptive randomisation changes
Fixed randomisation usually keeps assignment probabilities stable. Response-adaptive randomisation uses outcomes observed so far to change probabilities for later participants. A treatment that appears better may receive more future assignments.
The rule is part of the experiment. It must specify when probabilities update, what outcomes count and how extreme an imbalance may become.
| Feature | Fixed allocation | Response-adaptive allocation |
|---|---|---|
| Probability | Usually stable | Updates from accrued outcomes |
| Information | Baseline design inputs | Previous unblinded outcomes |
| Relation to time | Often balanced on average | Can become linked to treatment |
| Analysis | Standard comparison | Must respect adaptation |
2. Why time trends matter
Suppose outcomes improve over calendar time because sites learn, eligibility changes or supportive care improves. If the rule sends more later participants to the arm favoured early, that arm receives more of the high-response period. Treatment and time become associated.
Randomisation has not vanished, but a naive crude estimator can mix treatment effect with background trend. Risk depends on the rule, outcome delay, trend shape and analysis.
3. Routes to allocation bias
Background risk may change gradually or abruptly. Recruitment may shift between sites, outcome assessment may speed up, and visible unequal probabilities can influence investigators. Delayed outcomes can make the rule adapt to an unrepresentative subset.
| Route | Diagnostic record | Response |
|---|---|---|
| Secular trend | Calendar block and site | Model or stratify time |
| Outcome delay | Assignment-to-outcome lag | Simulate realistic delays |
| Extreme imbalance | Probability history | Use floors, ceilings or burn-in |
| Operational bias | Recruitment and assessment changes | Blinding and independent monitoring |
4. Pre-specify the adaptive rule
Define the burn-in, update schedule, outcome, probability formula, minimum and maximum allocation, stopping rules and primary analysis before outcomes are seen. State who can view interim results and how software and random seeds are controlled.
- Fix the estimand and primary endpoint.
- Log the probability used for every assignment.
- Describe delayed and missing outcomes.
- Protect a contemporaneous control where needed.
- Separate simulation development from conduct.
5. Simulate plausible worlds
Evaluate no effect, realistic effects, delayed responses, missingness and several time trends. Record type I error, power, estimation bias, confidence-interval coverage, expected allocation and the chance of extreme imbalance.
Simulation is how a team learns whether a rule that feels fair inside the trial produces reliable inference under real conditions.
6. Worked example: improvement over time
Two hundred participants enter in four equal calendar blocks. Background response rises from 40% to 60% as supportive practice improves. Early random variation favours B, so the rule shifts B’s allocation from 30% to 70%.
| Block | Background response | Share assigned B | Expected B count |
|---|---|---|---|
| 1 | 40% | 30% | 15 |
| 2 | 47% | 45% | 23 |
| 3 | 53% | 60% | 30 |
| 4 | 60% | 70% | 35 |
B receives far more participants during the later, higher-response period. Even with identical causal effects, B can show a better crude outcome. A time-aware analysis or preserved contemporaneous controls may reduce distortion, but validity must be demonstrated for the selected rule.
7. Match analysis to randomisation
Calendar-block adjustment can help, but adding time to a model is not a universal cure. Pre-specify and simulate the estimator. Depending on the design, valid approaches may use randomisation-based inference, model adjustment, allocation-probability weighting or specialised estimators.
Report allocation and outcome histories together. Otherwise readers cannot see whether the apparent effect tracks changing probabilities.
8. Changed-context transfer check
Move to an online education experiment where tutoring prompts are allocated adaptively. Student success rises during term as learners master the platform. A prompt receiving more late-term assignments may look better without causing improvement. Use time blocks, probability logs, delay simulations and a stable contemporaneous comparison.
The domain changed, but the causal problem remains: allocation learns from a stream whose background also changes.
9. Limits and misconceptions
Adaptive randomisation does not automatically benefit participants, increase power or reduce sample size. Rules can react to noise, delayed outcomes or trends. Ethical and statistical claims require realistic evaluation.
| Misconception | Correction |
|---|---|
| More people on the apparent winner is always better | Early noise can mislead |
| Randomisation prevents every bias | Changing probabilities can interact with time |
| Post-hoc adjustment fixes the problem | Analysis should be planned and validated |
| Adaptive means improvised | Valid adaptation is prospective and rule-based |
10. Frequently asked questions
Is unequal randomisation adaptive?
Not necessarily. A fixed 2:1 ratio is unequal but not response-adaptive. Adaptive probabilities update from accumulating information.
Should investigators know the ratio?
Knowledge can alter recruitment or assessment. Access should follow the pre-specified operational plan.
Can calendar time be ignored?
Not safely when probabilities and background outcomes both change. Assess the design-specific risk.
11. Practical design audit
An independent reviewer should be able to replay the allocation sequence and recover the promised operating characteristics.
- Is every probability time-stamped?
- Were update rules and bounds fixed in advance?
- Do simulations include trends, delays and missingness?
- Are interim results protected from operational influence?
- Does the final analysis match the rule and estimand?
12. Sources and reporting template
Report: “Participants were assigned using [rule] after [burn-in], with probabilities bounded by [limits] and updated every [interval] from [outcome]. Simulations covered [trend and delay scenarios]. The primary estimator [method] accounted for [feature]. Allocation and response histories appear by calendar block.”
Continue through the Gold Standard ecosystem
- Experimental Design root guide.
- The Gold Standard of Education.
- The Gold Standard collection.
- How X Works hub.
- Factorial Designs, Interaction Estimands and Marginal Effects.
- Treatment-Effect Heterogeneity, Subgroup Multiplicity and Interaction Tests.
- Attrition, Missing Outcomes and Post-Randomisation Selection.
- Noncompliance, Instrumental Variables and Complier Effects.