THE GOLD STANDARD OF EXPERIMENTAL DESIGN
Find your next learning step
A factorial design can study more than one factor in the same experiment, but efficiency does not remove the need to define the effect being estimated. When factors interact, an average marginal effect and a controlled effect at one level of the other factor answer different questions.
Start with the mechanism, follow the worked case, then change the context and see whether the explanation still holds.
Return to the Gold Standard of Experimental Design · Learning that transfers into life · The Gold Standard hub · How X Works
Chapter contents
Route A · Cells, effects and estimands
Route B · Worked factorial analysis
Route C · Transfer, limits and misconceptions
Route D · Audit, FAQs and sources
CHAPTER 1 OF 13 · UNDERSTAND
1. A factorial design crosses factors
In a 2 × 2 design, Factor A has two levels and Factor B has two levels, producing four cells: neither, A only, B only and both. With balanced independent randomisation, the design can estimate the factors efficiently and can examine whether their effects combine additively on the chosen scale.
The four cell means are the foundation. Labels such as ‘main effect’ are summaries of specified contrasts, not properties floating free of the table.
CHAPTER 2 OF 13 · UNDERSTAND
2. Define the estimand before the estimator
An estimand states the treatment conditions, population, outcome, population-level summary and handling of events that affect interpretation. In a factorial study, it must also clarify what happens to the other randomised factor in the comparison.
| Question | Possible estimand | Interpretation |
|---|---|---|
| Effect of A averaged over B | Marginal effect of A | Average across the randomised distribution of B |
| Effect of A when B is absent | Controlled effect at B=0 | Compare A versus no A among cells without B |
| Does A depend on B? | Interaction contrast | Difference between the two controlled effects of A |
Contents · Previous chapter · Back to contents · Next chapter
CHAPTER 3 OF 13 · UNDERSTAND
3. Interaction is scale-specific
No interaction on an additive outcome scale does not imply no interaction on a ratio or odds scale. State the scale before interpreting synergy or antagonism. A visual departure from parallel lines may help, but the estimand and uncertainty belong in the formal analysis.
Choose the scale that matches the scientific question and reporting context. Do not hunt across scales until one produces a dramatic label.
Contents · Previous chapter · Back to contents · Next chapter
CHAPTER 4 OF 13 · BUILD
4. Start with the four cell means
A fictional learning experiment
Suppose two independently randomised supports are tested: A is a retrieval reminder and B is a worked-example sheet. Mean assessment scores are 60 with neither, 68 with A only, 64 with B only and 80 with both.
| B absent | B present | |
|---|---|---|
| A absent | 60 | 64 |
| A present | 68 | 80 |
Every later effect in this example is a contrast of these four means. The table should remain visible even when a regression model is used.
Contents · Previous chapter · Back to contents · Next chapter
CHAPTER 5 OF 13 · BUILD
5. Calculate controlled and marginal effects
The controlled effect of A when B is absent is 68 − 60 = 8 points. When B is present, it is 80 − 64 = 16 points. The effect of A therefore depends on B in this fictional table.
The marginal mean with A is (68 + 80)/2 = 74. Without A it is (60 + 64)/2 = 62. The marginal effect of A is 12 points. That is the average across the 50:50 randomised distribution of B, not the effect in either controlled setting.
Contents · Previous chapter · Back to contents · Next chapter
CHAPTER 6 OF 13 · BUILD
6. Calculate and interpret the interaction
On the additive scale, the interaction contrast is (80 − 64) − (68 − 60) = 16 − 8 = 8 points. Equivalently, 80 − 68 − 64 + 60 = 8. A positive value means the combined cell exceeds what simple addition of the two separate effects would predict.
| Contrast | Calculation | Estimate |
|---|---|---|
| A effect when B absent | 68 − 60 | 8 |
| A effect when B present | 80 − 64 | 16 |
| Marginal A effect | 74 − 62 | 12 |
| A×B interaction | 16 − 8 | 8 |
Sampling uncertainty still matters. These means alone do not show whether an interaction estimate is precise enough for the intended decision.
Contents · Previous chapter · Back to contents · Next chapter
CHAPTER 7 OF 13 · BUILD
7. Match the primary analysis to the question
If the primary aim is the marginal effect of A under the trial’s randomisation of B, an at-the-margins contrast may be appropriate. If the aim is A in the absence of B, compare the relevant cells or fit a model that returns that controlled contrast. If interaction is the scientific target, power and reporting must reflect it.
Pre-specify the model, covariate adjustment, missing-data strategy and contrasts. Reporting only two ‘main effects’ can conceal a scientifically important interaction.
Contents · Previous chapter · Back to contents · Next chapter
CHAPTER 8 OF 13 · TRANSFER
8. Changed-context transfer: fertiliser and irrigation
Move the design to agronomy. Factor A is fertiliser and Factor B is irrigation; the outcome is crop yield. A marginal fertiliser effect averages over the experiment’s irrigation allocation. A controlled fertiliser effect at low irrigation answers a different farming question.
| Learning experiment | Agronomy experiment |
|---|---|
| Reminder A | Fertiliser A |
| Worked examples B | Irrigation B |
| Assessment score | Crop yield |
| A effect averaged over B | Fertiliser effect averaged over irrigation allocation |
| A×B interaction | Whether fertiliser response changes with irrigation |
If real farms use a different irrigation mix from the trial, the marginal effect may not transport unchanged. Controlled cell contrasts can help explain why.
Contents · Previous chapter · Back to contents · Next chapter
CHAPTER 9 OF 13 · TRANSFER
9. Use plots to reveal the contrast
Show cells and uncertainty
Plot the four cell estimates with confidence intervals, then connect levels of one factor within each level of the other. Parallel lines suggest little additive interaction; diverging or crossing lines suggest dependence. The plot is a diagnostic and communication tool, not a substitute for the pre-specified contrast.
Also report cell sample sizes and missing outcomes. An apparently balanced allocation can become imbalanced after attrition.
Contents · Previous chapter · Back to contents · Next chapter
CHAPTER 10 OF 13 · LIMITS
10. Misconceptions and evidence limits
- A factorial design is not automatically powered to detect interaction.
- A non-significant interaction is not proof of exact additivity.
- A marginal effect depends on how the other factor is distributed.
- ‘Main effect’ can be ambiguous when interaction is present unless the contrast is defined.
- Fractional factorial designs may alias effects, so not every interaction is separately estimable.
Efficiency comes from structure and assumptions. Make both visible.
Contents · Previous chapter · Back to contents · Next chapter
CHAPTER 11 OF 13 · VERIFY
11. Practical factorial audit
Before recruitment or data collection, list every cell, randomisation mechanism, primary estimand, outcome scale and interaction contrast. Confirm whether sample size addresses each primary question. After analysis, reconcile cell means with model-based estimates.
| Check | Pass condition | Warning |
|---|---|---|
| Cells | All combinations represented as planned | Structural empty cell |
| Estimand | Other factor handled explicitly | Undefined ‘main effect’ |
| Interaction | Scale and contrast pre-specified | Post hoc synergy claim |
| Reporting | Cell means, uncertainty and sample sizes shown | Only model coefficients reported |
Contents · Previous chapter · Back to contents · Next chapter
CHAPTER 12 OF 13 · VERIFY
12. Useful FAQs
Can I ignore interaction if it is not significant?
Not automatically. Consider precision, planned estimand and whether the analysis assumes no important interaction.
Is the marginal effect the average of two controlled effects?
In a balanced 2 × 2 additive contrast, yes, with equal weighting. Other allocations or estimands may use different weights.
Do I need equal cell sizes?
Not always, but balance usually improves clarity and precision. Analysis must reflect the actual design.
What changes in a fractional factorial design?
Some effects can be aliased. The estimable contrasts depend on the design resolution and assumptions.
- NIST two-way factorial experiment and interaction model
- Estimands for factorial trials
- CONSORT and SPIRIT guidance for factorial randomised trials
- EMA and ICH E9 estimand guidance
Contents · Previous chapter · Back to contents · Next chapter
CHAPTER 13 OF 13 · VERIFY
13. Sources and a reproducible reading route
These sources connect the classical factorial model to current estimand and reporting guidance. Use them to define the scientific question before choosing the convenient estimator.
Contents · Previous chapter · Back to contents · Return to the Experimental Design root
Connected reading across eduKate
Continue through the routes that materially extend this method: