Why Science? | FDA Multiple Endpoints, Multiplicity and False-Positive Control

Testing multiple endpoints, doses, time points or populations can increase the probability of at least one erroneous efficacy conclusion. FDA’s guidance focuses on multiple endpoints within one trial and explains why unadjusted selection among results can create false or misleading claims.

For multiplicity appears when several chances to claim success coexist, each formal question spends part of the study’s error budget. Testing multiple endpoints, doses, time points or populations can increase the probability of at least one erroneous efficacy conclusion. FDA’s guidance focuses on multiple endpoints within one trial and explains why unadjusted selection among results can create false or misleading claims. Keep this boundary visible: The problem is the decision process across tests, not the mere presence of many measured variables.

A strong science note separates observation from interpretation. List every analysis that could support a formal success claim and group them into one or more testing families. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Next chapter

FDA identifies the guidance as final, dated October 2022, under docket FDA-2016-D-4460, issued by CBER and CDER. It describes grouping, ordering and statistical methods for managing multiple endpoints in human drug and biologic trials.

A multiplicity strategy makes the october 2022 guidance is final a family decision rather than a hunt for the smallest p-value. FDA identifies the guidance as final, dated October 2022, under docket FDA-2016-D-4460, issued by CBER and CDER. It describes grouping, ordering and statistical methods for managing multiple endpoints in human drug and biologic trials. Preserve this guardrail: The document does not cover every source of multiplicity or every specialised platform design.

A strong science note separates observation from interpretation. Create a source receipt with title, final status, date, docket, issuing centres and defined scope. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

A primary endpoint is selected to provide the most clinically relevant and convincing evidence for the principal objective. More than one primary endpoint may be appropriate when one outcome cannot capture the treatment effect, but the success rule must be explicit.

To test primary endpoints carry the central effectiveness claim, follow alpha and clinical meaning together. A primary endpoint is selected to provide the most clinically relevant and convincing evidence for the principal objective. More than one primary endpoint may be appropriate when one outcome cannot capture the treatment effect, but the success rule must be explicit. The conclusion stops here: Calling several endpoints “primary” without a decision rule leaves room to choose success after results are seen.

A strong science note separates observation from interpretation. State the primary family and exact rule that determines whether the trial meets its objective. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

Secondary endpoints may measure additional treatment effects, explain the primary result or support further claims. When formal conclusions are intended, they may need inclusion in a multiplicity strategy. Exploratory endpoints generate hypotheses and should be labelled accordingly.

For secondary endpoints have planned supportive roles, each formal question spends part of the study’s error budget. Secondary endpoints may measure additional treatment effects, explain the primary result or support further claims. When formal conclusions are intended, they may need inclusion in a multiplicity strategy. Exploratory endpoints generate hypotheses and should be labelled accordingly. Keep this boundary visible: A nominally significant secondary result is not automatically confirmatory when its test was not protected.

A strong science note separates observation from interpretation. Classify endpoints as primary, multiplicity-protected secondary, supportive or exploratory and justify each role. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

The family-wise error rate is the probability of making one or more false-positive conclusions within a defined family when relevant null hypotheses are true. Procedures allocate or sequence the available alpha so that the overall risk remains controlled.

A multiplicity strategy makes family-wise type i error protects the set of claims a family decision rather than a hunt for the smallest p-value. The family-wise error rate is the probability of making one or more false-positive conclusions within a defined family when relevant null hypotheses are true. Procedures allocate or sequence the available alpha so that the overall risk remains controlled. Preserve this guardrail: Error control depends on the prespecified family and procedure, not a post hoc list of favourable tests.

A strong science note separates observation from interpretation. Draw the endpoint family and label how the total alpha can reach each hypothesis. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

Co-primary endpoints may be used when treatment benefit must be demonstrated on each of several distinct outcomes. Because all must succeed, the rule can make false-positive risk smaller but can reduce power and require a larger sample.

To test co-primary endpoints require success on all components, follow alpha and clinical meaning together. Co-primary endpoints may be used when treatment benefit must be demonstrated on each of several distinct outcomes. Because all must succeed, the rule can make false-positive risk smaller but can reduce power and require a larger sample. The conclusion stops here: Failure of one required co-primary endpoint means the prespecified joint success criterion is not met.

A strong science note separates observation from interpretation. Write the clinical reason each component is necessary and simulate the decision when only one succeeds. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

A design may allow effectiveness to be established through any one of several primary endpoints when each independently represents meaningful benefit. This creates multiplicity and generally requires adjustment so the chance of a false success across alternatives is controlled.

For alternative primary endpoints allow success on at least one, each formal question spends part of the study’s error budget. A design may allow effectiveness to be established through any one of several primary endpoints when each independently represents meaningful benefit. This creates multiplicity and generally requires adjustment so the chance of a false success across alternatives is controlled. Keep this boundary visible: At least-one success should be clinically coherent, not a lottery among weakly related outcomes.

A strong science note separates observation from interpretation. Define why each alternative would independently justify the claim and choose an adjustment rule. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

A composite endpoint treats the first occurrence or presence of any specified component as the outcome. It can increase event counts and capture several clinically important consequences, but components may differ in frequency, severity and treatment effect.

A multiplicity strategy makes composite endpoints combine events into one variable a family decision rather than a hunt for the smallest p-value. A composite endpoint treats the first occurrence or presence of any specified component as the outcome. It can increase event counts and capture several clinically important consequences, but components may differ in frequency, severity and treatment effect. Preserve this guardrail: A positive composite can be driven by a frequent, less important component while severe components show little benefit.

A strong science note separates observation from interpretation. Report the composite result beside component counts, effects, timing and clinical importance. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

Some endpoints combine component measurements using scores, rankings, responder rules or other algorithms. The construction should be clinically interpretable, validated where needed and prespecified, with attention to missing components and weighting.

To test multicomponent endpoints can summarise several dimensions, follow alpha and clinical meaning together. Some endpoints combine component measurements using scores, rankings, responder rules or other algorithms. The construction should be clinically interpretable, validated where needed and prespecified, with attention to missing components and weighting. The conclusion stops here: A single combined score can conceal opposing treatment effects among components.

A strong science note separates observation from interpretation. Change one component weight and test whether the conclusion or interpretation changes. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

In a fixed-sequence procedure, hypotheses are tested in a prespecified order, and testing continues only while earlier tests meet their criteria. The method can preserve full alpha for each tested hypothesis but makes later claims depend on earlier success.

For fixed-sequence testing spends alpha in a planned order, each formal question spends part of the study’s error budget. In a fixed-sequence procedure, hypotheses are tested in a prespecified order, and testing continues only while earlier tests meet their criteria. The method can preserve full alpha for each tested hypothesis but makes later claims depend on earlier success. Keep this boundary visible: A late endpoint cannot be promoted when an earlier gate fails unless another protected route was prespecified.

A strong science note separates observation from interpretation. Draw the sequence and evaluate three scenarios where success stops at different points. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

Gatekeeping procedures allow testing to move from a primary family to one or more secondary families under prespecified conditions. Parallel, serial or mixed structures can reflect clinical priorities while controlling overall error.

A multiplicity strategy makes gatekeeping protects families in stages a family decision rather than a hunt for the smallest p-value. Gatekeeping procedures allow testing to move from a primary family to one or more secondary families under prespecified conditions. Parallel, serial or mixed structures can reflect clinical priorities while controlling overall error. Preserve this guardrail: A complex gate should remain understandable enough for reviewers to reproduce the decision.

A strong science note separates observation from interpretation. Create a family diagram with gates, alpha transfers and stopping conditions. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

Bonferroni divides alpha among hypotheses and works without assumptions about dependence, often at a cost in power. Holm’s step-down method orders p-values and can be more powerful while controlling family-wise error under broad conditions.

To test bonferroni and holm use broad error bounds, follow alpha and clinical meaning together. Bonferroni divides alpha among hypotheses and works without assumptions about dependence, often at a cost in power. Holm’s step-down method orders p-values and can be more powerful while controlling family-wise error under broad conditions. The conclusion stops here: A procedure’s mathematical validity does not guarantee that the endpoint family is clinically sensible.

A strong science note separates observation from interpretation. Apply Bonferroni and Holm to the same example and compare which claims remain protected. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

Step-up or resampling-based procedures may gain power by using ordering or dependence information. Their assumptions, implementation and operating characteristics should be justified for the planned endpoints and sample size.

For hochberg and resampling methods need conditions, each formal question spends part of the study’s error budget. Step-up or resampling-based procedures may gain power by using ordering or dependence information. Their assumptions, implementation and operating characteristics should be justified for the planned endpoints and sample size. Keep this boundary visible: A more powerful method can be invalid when its dependence assumptions or implementation are wrong.

A strong science note separates observation from interpretation. Document the dependence assumption and run a simulation or sensitivity check for plausible correlations. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

Graphical multiple-testing procedures represent hypotheses as nodes and alpha-transfer rules as edges. When one hypothesis is rejected, its allocated alpha may move according to prespecified weights. The diagram can make complex sequential strategies easier to inspect.

A multiplicity strategy makes graphical procedures make alpha transfer visible a family decision rather than a hunt for the smallest p-value. Graphical multiple-testing procedures represent hypotheses as nodes and alpha-transfer rules as edges. When one hypothesis is rejected, its allocated alpha may move according to prespecified weights. The diagram can make complex sequential strategies easier to inspect. Preserve this guardrail: A visually attractive graph must still correspond exactly to the implemented calculations.

A strong science note separates observation from interpretation. Trace alpha through two success patterns and confirm the final claim set with independent code. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

Endpoint roles, families, hypotheses, testing order, alpha allocation and analysis methods should be specified before unblinding. Changes made for legitimate reasons should be documented with timing and effects on interpretation.

To test prespecification prevents result-driven testing, follow alpha and clinical meaning together. Endpoint roles, families, hypotheses, testing order, alpha allocation and analysis methods should be specified before unblinding. Changes made for legitimate reasons should be documented with timing and effects on interpretation. The conclusion stops here: A post hoc adjustment cannot fully restore confirmatory credibility after results guided the strategy.

A strong science note separates observation from interpretation. Compare protocol, statistical analysis plan and final report for changes to endpoint status or order. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

Multiplicity adjustment protects against certain false-positive conclusions but does not establish clinical importance, measurement validity or absence of bias. Results should include effect estimates, uncertainty, endpoint meaning and consistency across related evidence.

For interpretation needs effect size as well as error control, each formal question spends part of the study’s error budget. Multiplicity adjustment protects against certain false-positive conclusions but does not establish clinical importance, measurement validity or absence of bias. Results should include effect estimates, uncertainty, endpoint meaning and consistency across related evidence. Keep this boundary visible: A multiplicity-adjusted p-value can support a claim that is still clinically trivial or poorly measured.

A strong science note separates observation from interpretation. Write a result sentence containing effect size, interval, endpoint role, adjustment method and clinical interpretation. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter · Next chapter

The receipt records objectives, endpoint definitions, roles, families, hypotheses, success rules, alpha allocation, correlations, assumptions, software, simulations, deviations, adjusted results and limits. It lets a reviewer reproduce why each claim is confirmatory, supportive or exploratory.

A multiplicity strategy makes a complete multiplicity receipt protects the claim family a family decision rather than a hunt for the smallest p-value. The receipt records objectives, endpoint definitions, roles, families, hypotheses, success rules, alpha allocation, correlations, assumptions, software, simulations, deviations, adjusted results and limits. It lets a reviewer reproduce why each claim is confirmatory, supportive or exploratory. Preserve this guardrail: A flawless testing procedure cannot rescue an endpoint that is irrelevant or biased.

A strong science note separates observation from interpretation. Ask an independent analyst to reproduce the protected claim set from the receipt alone. Record the source, version or date, population or device context, method, result and one uncertainty. If the official record does not show a field, write “not shown” instead of filling the gap from memory.

Contents · Previous chapter