The Gold Standard of Experimental Design | Non-Inferiority Margins, Assay Sensitivity and Constancy

1. State why non-inferiority is the right question

Back to contents

Use a non-inferiority design when losing a limited amount of efficacy could be acceptable in exchange for a real advantage such as fewer harms, easier delivery or lower burden, and when placebo or no-treatment comparison is unsuitable. The advantage should be named before the trial.

If the new treatment offers no compensating value, demonstrating that it is slightly worse is not a useful success. The design begins with the decision, not the statistical test.

2. Write the estimand before the margin

Back to contents

Define treatment, population, outcome, time point, effect measure and handling of intercurrent events. A risk difference, risk ratio and hazard ratio place margins on different scales. The historical active-control effect must correspond closely to the current estimand.

Do not borrow a margin from another trial simply because the disease name matches. A changed endpoint, follow-up window or rescue-treatment strategy can change the effect being preserved.

3. Separate M1 from the clinically acceptable loss

Back to contents

M1 represents the effect of the active control over placebo supported by reliable historical evidence. The final non-inferiority margin is usually smaller, preserving a clinically important fraction of that effect. Both statistical uncertainty and clinical judgement enter the choice.

The margin must be fixed before current outcomes are inspected. A wide post hoc margin can turn almost any disappointing result into success.

4. Understand assay sensitivity

Back to contents

Assay sensitivity is the trial’s ability to distinguish an effective treatment from a less effective or ineffective one. In a placebo-controlled superiority trial, a positive result can demonstrate this directly. In an active-control non-inferiority trial, both treatments can look similar even when neither was delivered or measured effectively.

Design features that preserved the control effect historically—eligibility, adherence, endpoint ascertainment, dose and follow-up—must be credible in the current trial.

5. Defend the constancy assumption

Back to contents

Constancy assumes that the active control would have a similar effect relative to placebo now as in the historical studies used to derive M1. Changes in supportive care, pathogen resistance, diagnostic criteria, outcome risk or concomitant treatment can weaken that assumption.

Compare historical and current trials systematically. Where important differences remain, reduce the assumed control effect, choose another design or acknowledge that non-inferiority may be uninterpretable.

6. Worked example: the convenient antibiotic

Back to contents

A new oral antibiotic requires one daily dose instead of three. Historical placebo-controlled trials suggest the active control improved cure by 15 percentage points. After accounting for uncertainty, the team sets M1 at 12 points and decides that preserving at least two-thirds of that effect allows a 4-point loss.

In the current trial, cure is 88% with the new treatment and 90% with control. The estimated difference is −2 points, with a 95% confidence interval from −5 to +1. Because the lower bound crosses the −4-point margin, non-inferiority is not established even though the ordinary superiority test is non-significant.

The team also finds lower adherence than historical control trials. That weakens assay sensitivity and makes similarity easier to obtain. The correct result is inconclusive, not “equivalent.”

7. Interpret the confidence interval against the margin

Back to contents

For a scale where negative values favour the control, non-inferiority is established only when the confidence interval excludes effects worse than the negative margin. The point estimate alone is insufficient. Direction and scale must be stated clearly.

A result can be non-inferior and statistically superior, non-inferior without superiority, inconclusive or inferior. Prespecify the testing sequence and control multiplicity when making several claims.

8. Changed-context transfer check: a shorter teaching programme

Back to contents

Move the logic to a six-week learning programme compared with a twelve-week programme. The shorter course saves time, but the acceptable loss in mastery must be justified from meaningful educational consequences, not chosen from the observed score difference.

Historical evidence that the twelve-week programme beats ordinary instruction may not transfer if the assessment, student population or teaching quality changes. The concepts of margin, assay sensitivity and constancy still apply outside medicine.

9. Know why analysis choices pull in different directions

Back to contents

Non-adherence and crossovers can make treatments look similar, creating a false non-inferiority conclusion in an intention-to-treat analysis. Per-protocol analyses can reduce this dilution but lose randomisation protection through selection. Regulators and reporting guidance commonly expect both, with consistent conclusions.

Do not choose the analysis set that passes. Prespecify estimands, handle missing outcomes carefully and explain discrepancies between analyses.

10. Audit whether the trial could have failed honestly

Back to contents

Trace the rationale, historical evidence, M1 derivation, clinical preservation fraction, final margin, current control performance, adherence, protocol deviations, missingness and both primary analysis sets. Compare event rates and treatment delivery with the studies supporting constancy.

Run sensitivity analyses for plausible missing outcomes and historical-effect uncertainty. If modest changes reverse the conclusion, say so.

11. Frequently asked questions

Back to contents

Is non-significance in a superiority test enough?

No. Absence of detected superiority can reflect low power or poor conduct and does not establish non-inferiority.

Does non-inferior mean equivalent?

No. It means the data exclude losses larger than a prespecified margin under stated assumptions.

Can the margin be changed after results appear?

Not without undermining the confirmatory interpretation. The margin and rationale must be prespecified.

12. The gold standard protects a meaningful amount of effect

Back to contents

A strong non-inferiority trial makes the permitted loss clinically defensible, anchors it in conservative historical evidence and shows that the current trial could detect a failure. Similar-looking arms are not enough.

Before accepting the claim, ask what effect was preserved, why the active control should still work and whether adherence and measurement maintained assay sensitivity. If those questions are weak, the conclusion is weak too.