The Gold Standard of Experimental Design | Multi-Arm Trials, Shared Controls and Multiplicity

1. Begin with the comparison map

Back to contents

Three active arms and one control do not create one question. They may create three treatment-versus-control contrasts, dose–response questions, active–active comparisons or a gatekeeping sequence. Write every primary contrast before choosing sample size or adjustment.

Mark which claims can succeed independently and which belong to one joint decision. Multiplicity is not determined by the number of columns in a results table; it is determined by the family of opportunities to make a claim.

2. Shared control creates efficiency and correlation

Back to contents

Separate two-arm trials would each recruit their own control group. A coordinated multi-arm design can reuse one concurrent control, reducing total control recruitment and applying common eligibility, outcomes and follow-up.

Because each experimental contrast contains the same control observations, the estimates and test statistics are correlated. That correlation matters for simultaneous confidence intervals and Dunnett-type procedures; Bonferroni remains valid but may ignore available structure.

3. Allocation should support the planned contrasts

Back to contents

Equal allocation across all arms is simple, but it is not always most efficient when several arms share one control. For k experimental arms with equal variances and equal experimental-group sizes, a common approximation assigns about √k times as many participants to control as to each experimental arm.

For three experimental arms, √3 is about 1.73. A practical ratio might be 1.7:1:1:1, subject to outcome type, costs, expected dropout and power calculations. Allocation must be prespecified and concealed.

4. Define the multiplicity family from the decision

Back to contents

If any one of three doses can support the same product claim, the family naturally includes all three dose-versus-control tests. If three unrelated interventions are scientifically separate and would have been run as separate trials, some frameworks treat the comparisons separately, but the choice must be justified.

Regulatory, clinical and scientific contexts can demand different control of family-wise error. Avoid universal rules such as “multi-arm always means adjust” or “separate hypotheses never need adjustment.” State the claim family and consequence of a false positive.

5. Choose an error-control method that matches the family

Back to contents

Bonferroni allocates α/m across m tests and requires no correlation model. Holm tests ordered p values and is uniformly at least as powerful. Dunnett procedures use the correlation created by a shared control for treatment-versus-control comparisons. Closed testing or gatekeeping can reflect hierarchical claims.

The method should be in the protocol and analysis plan. Report adjusted p values or simultaneous confidence intervals together with unadjusted effect estimates. Adjustment changes the evidence threshold, not the estimated treatment effect.

6. Worked example: three arms share one control

Back to contents

A trial compares interventions A, B and C with control. The unadjusted two-sided p values are 0.012, 0.028 and 0.40. With Bonferroni family-wise α = 0.05, each comparison uses 0.0167: A passes, B and C do not.

Holm orders the values. A is compared with 0.05/3 = 0.0167 and passes; B is then compared with 0.05/2 = 0.025 and does not, so later hypotheses are not rejected. The conclusion is not that B has no effect. It is that this family and procedure do not support its confirmatory claim at the planned error rate.

7. Power every primary comparison and joint decision

Back to contents

A sample-size statement should identify the effect, variance or event rate, allocation, dropout, comparison-specific power and multiplicity procedure. Power for each contrast is not the same as the probability that all effective arms succeed.

Simulate when endpoints are correlated, allocation is unequal, arms may stop early or a joint success rule is used. Record both marginal power and disjunctive or conjunctive power where those quantities drive the decision.

8. Changed-context transfer check

Back to contents

Move the example from three doses of one medicine to three unrelated classroom supports sharing usual teaching. Shared-control efficiency and correlated contrasts transfer. The claim family may change because each programme can be adopted independently.

Cluster randomisation may now occur at class or school level. Power, standard errors and multiplicity must respect intraclass correlation and the unit of assignment. The clinical allocation ratio cannot be copied without redesign.

9. Limits and common misconceptions

Back to contents

A multi-arm trial is not automatically factorial: factorial arms represent combinations and interactions. A platform trial that adds arms over time raises concurrent-control and time-trend issues beyond a fixed multi-arm design. Dropping an arm can affect operating characteristics if not prespecified.

Do not adjust only the p values while leaving confidence intervals ordinary and then call both confirmatory. Do not select the multiplicity family after seeing which arms look promising. Do not interpret non-rejection as equivalence.

10. Audit the protocol against the final comparison set

Back to contents

Compare registry, protocol, statistical analysis plan and report. Reconcile every added, dropped, renamed or secondary comparison. Check recruitment periods, allocation changes, control concurrence, missing outcomes and arm-specific losses.

Recalculate adjusted decisions from the archived rule and full comparison set. The CONSORT multi-arm extension asks for all hypotheses, planned and unplanned comparisons, group flow and the rationale and method for multiplicity adjustment—or an explicit statement that none was applied.

11. Frequently asked questions

Back to contents

Does a shared control count twice?

No. The same control observations enter several contrasts, creating correlation. They should not be duplicated as if they were independent participants.

Is Bonferroni always too conservative?

It can be conservative because it ignores correlation, but it is transparent and valid. A prespecified Dunnett procedure may gain power for shared-control comparisons.

Can one successful arm make the whole trial positive?

Only if the protocol defines that joint decision and its error control. Otherwise report results contrast by contrast.

12. The gold standard makes the claim family visible

Back to contents

A multi-arm design earns its efficiency through coordination, not through relaxed standards. The shared control, allocation and error procedure should all point to the same prespecified comparison map.

Before accepting a headline, ask which contrasts were primary, which claims formed one family, how the shared control induced correlation and whether the reported intervals match the decision rule. If those pieces align, the result is both efficient and interpretable.