For the How Science Works topic on why a funnel plot cannot diagnose publication bias on its own, read sections 35–39 below. They explain missing evidence, alternative causes of asymmetry, limits of asymmetry tests and sensitivity methods. Continue through the Science Mastery guide or return to the How Science Works reading map.
Meta-analysis works by translating results from sufficiently compatible studies onto a common effect scale, weighting those study estimates according to their information and the chosen statistical model, combining them into a summary while quantifying uncertainty and between-study variation, and then asking whether the pooled result remains credible after considering study quality, heterogeneity, missing evidence and reasonable alternative analytic decisions.
Meta-analysis is sometimes described as:
an average of studies.
That description is useful only if we immediately make it more difficult.
Which studies?
Found how?
Measuring what?
Compared with what?
Using which effect measure?
Weighted how?
Under what assumption about whether the underlying effects are the same or vary?
What if the studies disagree because the populations, interventions, methods or biases differ?
What if unfavourable studies were never published?
The governing question: do these studies represent sufficiently related versions of the same scientific question that combining their estimates creates more knowledge rather than a more precise blur?
Quick Read
QUESTION → PROTOCOL → SYSTEMATIC SEARCH → ELIGIBILITY → STUDY SELECTION → RISK OF BIAS → DATA EXTRACTION → COMMON EFFECT MEASURE → STUDY ESTIMATES → WEIGHTS → HETEROGENEITY → FIXED-EFFECT / RANDOM-EFFECTS MODEL → SUMMARY ESTIMATE → CONFIDENCE INTERVAL → PREDICTION INTERVAL → SENSITIVITY ANALYSIS → MISSING-EVIDENCE ASSESSMENT → INTERPRETATION
Cochrane’s current Handbook describes most meta-analysis methods as variations on a weighted average of effect estimates from different studies. It also stresses that heterogeneity must be considered and that random-effects models do not make heterogeneity disappear. Prediction intervals can help show the range of underlying effects expected across similar settings.
1. A Systematic Review Is Not the Same as a Meta-Analysis
A systematic review is a structured process for identifying, selecting, appraising and synthesising evidence relevant to a defined question.
A meta-analysis is a statistical method for quantitatively combining compatible study results.
A systematic review may contain no meta-analysis if the studies are too heterogeneous or the outcomes cannot be combined meaningfully.
A meta-analysis performed without a systematic search and transparent eligibility process can produce a precise answer from a biased selection of studies.
The statistical synthesis should sit inside an evidence-identification architecture, not replace it.
2. The Review Question Defines Which Studies Belong Together
Before combining anything, reviewers define the question.
In intervention reviews this is often framed through dimensions such as population, intervention, comparator and outcome.
A meta-analysis of “education studies” is meaningless.
A meta-analysis of a particular retrieval-practice intervention compared with restudy, measured on delayed retention in a defined learner population, is much more coherent.
Compatibility is established conceptually before it is established mathematically.
3. Eligibility Criteria Are the Sampling Frame for the Evidence Base
A meta-analysis samples studies.
Inclusion criteria determine which studies can enter.
Year limits, language restrictions, publication type, population, design, outcome and intervention definitions all shape the evidence sample.
This is why How Sampling Works applies at a second level: the units are now studies rather than people.
If eligibility is decided after seeing which studies produce favourable results, the synthesis is selected before the statistics begin.
4. A Protocol Preserves the Review Plan Before the Evidence Is Fully Seen
Systematic reviews benefit from prospective protocols specifying the question, eligibility criteria, outcomes, search strategy and analysis plans.
This is the synthesis-level analogue of preregistration.
Without a protocol, inclusion criteria and analytic decisions can shift as reviewers discover which choices produce the desired summary.
The review itself is research and needs decision provenance.
5. The Search Strategy Determines Which Evidence Becomes Visible
Searching one convenient database is rarely enough for a high-stakes systematic review.
Relevant studies can live in different bibliographic databases, trial registries, conference proceedings, dissertations, regulatory records and unpublished datasets.
If positive studies are easier to publish and index, a narrow search can preferentially retrieve favourable evidence.
Cochrane’s current guidance on missing evidence explicitly recommends broad attempts to identify study reports and results rather than assuming the published journal record is complete.
6. PRISMA Improves Reporting of the Review Route
PRISMA 2020 provides reporting guidance for systematic reviews, including how studies were identified, selected, appraised and synthesised.
The familiar flow diagram shows how many records were identified, screened, excluded and included.
PRISMA does not guarantee methodological quality.
It helps readers recover the evidence-selection route.
7. Study Selection Should Be Reproducible Enough to Audit
Titles and abstracts can be ambiguous.
Eligibility decisions involve judgement.
Reviews often use independent screening by more than one person or verification procedures to reduce arbitrary inclusion.
Exclusion reasons at full-text stage should be preserved because silently removing inconvenient studies can bias the synthesis.
8. Risk of Bias Must Be Assessed Before We Treat Every Study as Equally Trustworthy
A randomised trial with concealed allocation and blinded outcome assessment is not evidentially identical to a confounded observational comparison with severe attrition.
Meta-analysis weights studies statistically according to information.
That is not the same as weighting them according to credibility.
A very large biased study can receive enormous statistical weight.
Risk-of-bias assessment and sensitivity analysis therefore sit alongside numerical weighting.
9. Data Extraction Is a Measurement Process
Reviewers extract sample sizes, means, standard deviations, event counts, effect estimates, follow-up times and covariates.
Errors here propagate directly into the meta-analysis.
Cochrane notes that severe apparent heterogeneity can sometimes result from simple data-extraction errors, such as confusing standard errors with standard deviations.
The meta-analysis can be mathematically flawless and factually wrong if the extracted inputs are wrong.
10. Different Outcomes Need a Common Effect Scale Before They Can Be Combined
Studies may report outcomes as means, proportions, risks, odds, rates or time-to-event measures.
Meta-analysis requires compatible effect measures.
- Mean difference
- Standardised mean difference
- Risk ratio
- Odds ratio
- Risk difference
- Hazard ratio
The chosen measure changes interpretation.
An odds ratio is not a risk ratio. A standardised mean difference is not a raw score difference.
Effect-size conversion is a scientific translation, not clerical formatting.
11. Mean Difference Preserves the Original Scale
If all studies measure blood pressure in mmHg, the difference in mean blood pressure can be combined directly.
This preserves clinical interpretability.
Likewise, if all education studies use the same examination scale, a raw mean difference can be more meaningful than a standardised effect.
Common scale is a gift when the underlying outcome is genuinely comparable.
12. Standardised Mean Difference Trades Interpretability for Compatibility
Studies may measure depression with different validated scales or learning with different tests.
The standardised mean difference divides the mean difference by a standard deviation so effects can be placed on a dimensionless common scale.
This can enable synthesis.
But standard deviations can differ because populations differ, measurement reliability differs or scales behave differently.
Standardisation solves one comparability problem and can create interpretive distance from the real-world outcome.
13. Relative Effects and Absolute Effects Answer Different Questions
A treatment can reduce relative risk by 20%.
If baseline risk is 50%, the absolute reduction is large.
If baseline risk is 1%, the absolute reduction is small.
Meta-analyses often combine relative effects because they may be more stable across risk levels.
Decision-makers still need absolute consequences.
The receiver does not live inside a ratio.
14. Study Estimates Are Weighted, Not Counted
Imagine five tiny studies and one very large precise study.
A simple vote count—five studies positive, one negative—treats them as equally informative.
Meta-analysis ordinarily gives more weight to more precise effect estimates.
Inverse-variance weighting is common: the smaller the variance of the estimate, the greater the statistical weight.
This is why meta-analysis is not democratic voting among papers.
15. Precision Weight Is Not Credibility Weight
A huge observational study can estimate a confounded association with extraordinary precision.
A smaller randomised trial may be causally more credible but less precise.
The numerical weight in a meta-analysis cannot fully encode design validity.
Reviewers need stratification, sensitivity analyses, risk-of-bias judgements and substantive interpretation.
Statistics cannot infer methodological credibility from standard errors alone.
16. Fixed-Effect Meta-Analysis Assumes One Underlying Effect
Under a conventional fixed-effect model, the included studies are treated as estimating the same underlying intervention effect, with observed differences attributed to sampling variation.
The combined estimate is therefore a precision-weighted estimate of that common effect for the included evidence.
This can be appropriate when the common-effect assumption is scientifically defensible.
It can be misleading when populations, interventions or contexts genuinely produce different effects.
17. Random-Effects Meta-Analysis Assumes Effects Vary Across Studies
A random-effects model assumes the studies estimate different but related effects drawn from a distribution of underlying effects.
The summary is interpreted as an estimate of the average effect across that distribution under the model.
Between-study variance enters the weights.
This generally gives smaller studies relatively more weight than they receive under fixed-effect analysis when heterogeneity is present.
Cochrane explicitly warns that random effects do not solve heterogeneity. They model it.
18. The Choice Between Fixed and Random Effects Is Scientific, Not Merely Statistical
One should not choose random effects simply because a heterogeneity test is “significant” or fixed effect because it is not.
Cochrane’s current guidance states that the choice should not be made based only on a statistical test for heterogeneity.
The question is whether there is a scientific reason to expect one common effect or a distribution of related effects.
Model choice should reflect the world the studies came from.
19. Heterogeneity Is Information About Differences Among Studies
Studies can disagree because of chance.
They can also disagree because the effects genuinely differ.
Reasons include:
- different populations;
- different doses or treatment versions;
- different comparators;
- different outcome definitions;
- different follow-up periods;
- different implementation quality;
- different bias structures;
- and different statistical methods.
Heterogeneity is not automatically a nuisance to average away.
It can reveal the operating envelope of the phenomenon.
20. The Chi-Squared Test Asks Whether Variation Exceeds Chance Expectations
Meta-analysis software often reports a heterogeneity test based on a Q or chi-squared statistic.
A low p-value suggests the observed study differences are difficult to explain by within-study sampling variation alone.
But the test can have low power when few studies exist and excessive power when many studies exist.
Absence of statistical heterogeneity evidence is not proof of identical effects.
21. I² Describes the Proportion of Observed Variation Attributed to Heterogeneity
I² is widely reported as a percentage.
It summarises the proportion of observed variability in effect estimates that is associated with between-study heterogeneity rather than sampling error under the method.
But I² should not be read from a rigid universal table.
Cochrane notes that its importance depends on the magnitude and direction of effects and the strength of evidence for heterogeneity. Its uncertainty can be substantial when few studies are included.
An I² of 50% does not mean “half the studies are wrong”.
22. Tau-Squared Estimates the Variance of Underlying Effects
Random-effects models estimate a between-study variance commonly written as τ².
Unlike I², τ² is on the scale of the effect measure.
Its magnitude therefore depends on the chosen effect scale.
Several estimators exist, and estimates can be unstable when few studies are available.
The existence of a sophisticated estimator does not make sparse evidence rich.
23. Prediction Intervals Show Where a New Similar Study Might Land
A random-effects confidence interval describes uncertainty around the average effect.
It does not show the spread of underlying effects across settings.
A prediction interval incorporates estimated between-study heterogeneity and provides a range for the underlying effect in a new study similar to those synthesised, under the model assumptions.
This can change interpretation dramatically.
The average effect may be beneficial while the prediction interval includes no benefit or harm in some settings.
Cochrane encourages prediction intervals when the number of studies is sufficient and the assumptions are defensible.
24. Forest Plots Show the Evidence Before the Summary
A forest plot displays each study’s effect estimate and confidence interval, usually alongside its weight, then the pooled result.
The visual order matters.
Readers can see whether one huge study dominates, whether effects point in different directions, whether small studies are extreme and whether the summary hides important variation.
The diamond at the bottom should never replace the forest above it.
25. A Statistically Significant Pooled Effect Can Hide Important Heterogeneity
Suppose ten studies average to a clear benefit.
Five show large benefit.
Three show almost none.
Two show harm.
The average may be statistically precise.
The decision problem is now “Why do effects differ?” rather than merely “Is the average above zero?”
Average effect and effect variability are separate pieces of knowledge.
26. Subgroup Analysis Tries to Explain Heterogeneity
Reviewers may compare studies by age, dose, setting, intervention type, risk of bias or another characteristic.
This can reveal real effect modification.
It can also generate false stories when many subgroup splits are tried with few studies.
Subgroup hypotheses should preferably be pre-specified, limited, mechanistically plausible and tested through interaction rather than by comparing “significant here” with “not significant there”.
27. Meta-Regression Extends Subgroup Analysis to Study-Level Predictors
Meta-regression models study effect estimates as a function of study-level characteristics.
Does effect size change with dose?
Does it differ by average age?
Does study quality explain variation?
But study-level relationships can suffer ecological bias. A relationship across study averages need not be the same as the relationship among individuals within studies.
Meta-regression also becomes unstable when too few studies are available.
28. More Studies Do Not Automatically Mean More Independent Evidence
Multiple papers may report overlapping participants from the same trial.
One cohort may generate several follow-up publications.
Treating every paper as an independent study double-counts evidence.
Reviewers must identify the underlying study unit and link multiple reports correctly.
Publication count is not study count.
29. Multiple Outcomes From One Study Need Dependence-Aware Handling
A study may report several correlated outcomes or several time points.
Including all of them as if independent gives that study excessive weight.
Reviewers may select one pre-defined outcome, combine outcomes appropriately, use multivariate methods or robust variance approaches depending on the question.
Dependence does not disappear because the rows look separate in software.
30. Rare Events Need Special Methods
When outcomes are rare, some studies contain no events in one or both groups.
Standard log odds-ratio or risk-ratio methods can behave poorly or require continuity corrections.
Cochrane discusses specialised methods and warns that studies with no events can contribute differently depending on the effect measure.
Method choice should follow the data structure rather than default software settings.
31. Zero Events in Both Arms Do Not Mean “No Effect”
If no participant experiences an event in either arm, the study provides little or no information about a relative-risk effect for that outcome.
It may still inform absolute risk under some models.
The absence of events can reflect genuinely low risk or simply small sample size.
Zero is an observation, not infinite certainty.
32. Sensitivity Analysis Tests Whether the Summary Depends on Fragile Choices
What if high-risk-of-bias studies are excluded?
What if a different τ² estimator is used?
What if one influential study is removed?
What if fixed-effect and random-effects models disagree?
Cochrane emphasises sensitivity analysis because meta-analysis contains many judgement calls.
A robust conclusion should not depend entirely on one arbitrary analytic switch.
33. Leave-One-Out Analysis Can Reveal Dominant Studies
Removing each study in turn shows whether the pooled conclusion is driven by one influential result.
If excluding one study reverses the conclusion, the meta-analysis is structurally fragile.
That does not mean the influential study should be discarded.
It means the evidence landscape should be described honestly rather than compressed into a falsely stable summary.
34. Missing Evidence Can Bias the Entire Meta-Analysis
Meta-analysis combines what is available.
If availability depends on result direction, magnitude or p-value, the evidence sample can be systematically biased.
Cochrane’s current Chapter 13 treats this as risk of bias due to missing evidence, including missing entire studies or selectively missing results within known studies.
A beautifully weighted average of a selectively visible literature remains selectively visible.
35. Publication Bias Is Only One Form of Missing Evidence
Entire studies may remain unpublished.
Published papers may omit particular outcomes.
Results may appear only in registries.
Positive results may appear faster or in more visible journals.
Language and database coverage can also shape discovery.
The missing-evidence problem begins before the funnel plot.
36. Funnel Plots Display Small-Study Effects, Not “Publication Bias” Directly
A funnel plot graphs effect estimates against a measure of precision.
In an idealised setting without important bias or heterogeneity, small imprecise studies scatter more widely and large precise studies cluster nearer the true effect, creating a funnel shape.
Asymmetry can arise from selective publication.
It can also arise from real heterogeneity, methodological differences, chance or biases that disproportionately affect small studies.
Cochrane explicitly warns that funnel-plot asymmetry is not diagnostic of publication bias.
37. Funnel-Plot Tests Usually Need Enough Studies
Statistical tests for funnel-plot asymmetry have low power when few studies are included.
Cochrane uses a rule of thumb that such tests should generally be considered only when at least ten studies are available and other conditions are appropriate.
Running an Egger test on four studies does not turn sparse evidence into reliable bias detection.
38. Trim-and-Fill Is Not a Time Machine
Methods such as trim-and-fill attempt to adjust for funnel-plot asymmetry by imputing hypothetical missing studies.
They can be useful as sensitivity tools.
They cannot know which studies truly exist or why evidence is missing.
Modelled missingness should never be mistaken for recovered data.
39. Meta-Analysis Can Amplify Bias When Small Biased Studies Receive Weight
Random-effects models give relatively more weight to smaller studies than fixed-effect models when heterogeneity exists.
If smaller studies systematically report larger effects because of publication bias or weaker methods, the random-effects summary can be pulled toward that distortion.
Cochrane explicitly notes this risk.
Random effects is not automatically the “more conservative” model.
40. Network Meta-Analysis Extends Synthesis Across Multiple Treatments
Conventional pairwise meta-analysis compares A versus B.
Network meta-analysis combines direct and indirect evidence across a network such as A versus B, B versus C and A versus C.
This can estimate relative effects even for pairs not directly compared in every trial.
The additional power comes with additional assumptions, especially transitivity and consistency across the network.
More connections create more opportunities for mismatch.
41. Individual Participant Data Meta-Analysis Can Recover More Resolution
Most meta-analyses use summary statistics from publications.
Individual participant data meta-analysis obtains participant-level data from each study.
This can harmonise outcomes, apply consistent analysis, examine interactions and sometimes handle missing data more coherently.
It is resource intensive and may still suffer if some trial datasets are unavailable.
42. Prospective Meta-Analysis Plans Synthesis Before Individual Results Are Known
In a prospective meta-analysis, eligible studies are identified and synthesis methods planned before results of those studies are known.
This can reduce outcome-selection and study-selection bias.
Large collaborative prospective meta-analyses can coordinate core outcomes and analysis plans across trials.
This is preregistration lifted from the study level to the evidence-network level.
43. Bayesian Meta-Analysis Makes Prior and Hierarchical Assumptions Explicit
Bayesian meta-analysis can place prior distributions on average effects and heterogeneity, then update them using study data.
This can be especially useful when data are sparse and prior information is genuinely available.
The result depends on prior choices as well as likelihood.
Transparent priors and sensitivity analyses are therefore part of the evidence route.
44. Meta-Analysis of Observational Studies Needs Extra Causal Restraint
Combining ten confounded associations does not create a randomised effect.
Studies may adjust for different covariates, define exposures differently and carry different residual confounding.
A pooled observational estimate can be useful.
Its causal interpretation should not outrun the identification strategy of the underlying studies.
Bias is inherited before it is averaged.
45. Meta-Analysis of Education Research Has a Special Intervention-Identity Problem
Two studies may both call an intervention “retrieval practice”.
One uses low-stakes quizzes with immediate feedback twice a week.
Another uses flashcards daily without feedback.
The label is the same.
The treatment packages differ.
A good education meta-analysis needs enough intervention coding to understand what is actually being pooled.
46. Meta-Analysis Can Hide Implementation Quality
A programme can work when delivered by expert teams and fail when implemented poorly.
Pooling both may produce a modest average effect.
The actionable question may be whether implementation fidelity explains the difference.
Synthesis should not compress mechanisms out of existence.
47. Meta-Analysis Can Create False Authority Through Visual Precision
A forest plot with a narrow diamond looks authoritative.
But the narrow confidence interval may describe only uncertainty around the average under the model.
It does not eliminate heterogeneity, bias, missing studies or external-validity limits.
Visual precision is not epistemic completeness.
48. The Hostile Test: Pooling Apples, Oranges and Weather Stations
A review finds twelve “learning intervention” studies.
Some test primary students.
Some test university students.
Some measure immediate recall.
Some measure six-month transfer.
Some interventions last one day.
Some last one year.
Software can calculate a pooled standardised mean difference.
The existence of a common unit does not prove the studies answer a common question.
49. The Second Hostile Test: Meta-Analysis of Biased Studies
Twenty observational studies compare tutoring users with non-users.
Motivation and parental support are poorly measured in all twenty.
The pooled estimate is precise.
The shared confounding remains.
Repetition of the same bias can produce confidence without causal identification.
50. The Third Hostile Test: Random Effects Used to Avoid Thinking About Heterogeneity
Study effects vary dramatically.
The reviewer selects random effects and writes:
Heterogeneity was accounted for.
It was not explained.
The model widened uncertainty and estimated an average across heterogeneous effects.
The scientific question—why effects differ—remains alive.
51. The Fourth Hostile Test: Funnel Plot Treated as Proof of Publication Bias
Eight studies produce an asymmetric funnel plot.
The review declares publication bias proven.
The studies differ systematically in population and intervention intensity.
Small-study effects can arise for many reasons.
Funnel asymmetry is a clue, not a diagnosis.
52. The Fifth Hostile Test: The Average Effect Nobody Actually Receives
Effects are strongly positive in younger patients and near zero in older patients.
The random-effects average is moderately positive.
No actual patient is the average of those populations.
For decision-making, effect modification may matter more than the grand mean.
Meta-analysis should compress evidence without compressing away the receiver.
53. Primary School: Meta-Analysis Begins as “Do Not Judge From One Test Alone”
A child tests three paper aeroplane designs on one windy afternoon.
Another class repeats the experiment indoors.
A third class uses heavier paper.
Instead of picking the most exciting single result, students can compare all experiments and ask why they differ.
The early intuition is:
Many pieces of evidence can be stronger together, but only if I remember how each piece was made.
54. Secondary School: Separate “More Studies” From “Better Evidence”
Secondary students can learn that ten studies do not automatically beat one careful study.
They should ask:
- Were the studies asking the same question?
- Were their methods reliable?
- Did they measure the same outcome?
- Were some much larger?
- Did results disagree?
- Could negative studies be missing?
This is evidence synthesis before formal weighting.
55. JC and University: Meta-Analysis Becomes a Model of a Modelled Literature
At higher levels, learners should reconstruct:
- review question;
- protocol;
- search;
- eligibility;
- risk of bias;
- effect measure;
- unit of analysis;
- weighting;
- fixed versus random effects;
- heterogeneity;
- I² and τ²;
- prediction intervals;
- subgroups or meta-regression;
- missing evidence;
- and sensitivity analyses.
The summary estimate becomes the end of a long chain, not the beginning of interpretation.
56. Where Meta-Analysis Fits in the eduKateSG “How Works” Landscape
- How Scientific Research Works — individual studies entering cumulative knowledge.
- How Sampling Works — study selection as an evidence-sampling problem.
- How Research Bias Works — bias inherited from primary studies and missing evidence.
- How Research Validity Works — whether pooled interpretation is fit for purpose.
- How Statistical Power Works — precision and detectability within primary studies and asymmetry tests.
- How Preregistration Works — protocols and decision timing at study and review level.
- How Uncertainty Works — confidence and prediction intervals.
- How Comparison Works — the common frame needed before effect estimates can be combined.
Meta-analysis owns the statistical synthesis layer: when multiple studies are sufficiently comparable, how can their effect estimates be combined without erasing uncertainty, heterogeneity, bias or missing evidence?
57. What This Article Does Not Claim
- A meta-analysis is not the same as a systematic review.
- More studies do not automatically mean better evidence.
- Statistical weighting is not the same as credibility weighting.
- A random-effects model does not make heterogeneity disappear.
- I² is not a percentage of studies that are inconsistent or wrong.
- A narrow pooled confidence interval does not show that effects are similar across settings.
- Funnel-plot asymmetry does not prove publication bias.
- Meta-analysis cannot recover studies or outcomes that were never found or reported.
- Combining biased studies can produce a precise biased summary.
- Studies should not be pooled merely because software allows a common effect-size transformation.
58. A Compact Meta-Analysis Audit
- What exact review question is being answered?
- Was a protocol written prospectively?
- How comprehensive was the search?
- What eligibility criteria selected studies?
- Could selection depend on results?
- Are the populations compatible?
- Are interventions and comparators compatible?
- Are outcomes measuring the same construct?
- What risk of bias do the included studies carry?
- What effect measure was chosen?
- Were effect-size conversions justified?
- Are multiple reports from one study linked correctly?
- Are multiple outcomes from one study handled as dependent?
- How are studies weighted?
- Why was fixed-effect or random-effects modelling chosen?
- How much heterogeneity exists?
- What does I² mean in this context?
- What is τ²?
- Is a prediction interval reported?
- Do subgroup or meta-regression analyses have enough studies?
- How sensitive is the result to influential or high-risk studies?
- Could evidence be missing selectively?
- Is funnel-plot analysis appropriate for the number and structure of studies?
- Does the pooled average hide important effect modification?
- What decision can the summary actually support?
59. Frequently Asked Questions
What is meta-analysis?
Meta-analysis is the statistical synthesis of compatible effect estimates from multiple studies using explicit weighting and modelling assumptions to estimate a summary effect and its uncertainty.
What is the difference between a systematic review and a meta-analysis?
A systematic review is the broader structured process of finding, selecting, appraising and synthesising evidence. Meta-analysis is an optional quantitative synthesis method within a systematic review when studies are sufficiently compatible.
What is heterogeneity?
Heterogeneity is variation among study effects beyond ordinary within-study sampling variation, potentially arising from real differences in populations, interventions, outcomes, methods or bias.
What is the difference between fixed-effect and random-effects meta-analysis?
Fixed-effect analysis treats included studies as estimating one underlying effect, whereas random-effects analysis allows underlying effects to vary across studies and estimates an average across an assumed distribution of effects.
What does I² mean?
I² summarises the proportion of observed variability in effect estimates associated with between-study heterogeneity rather than sampling error under the model. Its interpretation depends on effect magnitude, direction, number of studies and uncertainty.
What is a prediction interval?
In a random-effects meta-analysis, a prediction interval estimates a range within which the underlying effect of a new sufficiently similar study might lie, incorporating estimated between-study heterogeneity and uncertainty.
60. Authoritative Research Corridor
- Cochrane Handbook Chapter 10 — Analysing Data and Undertaking Meta-Analyses, updated November 2024
- Cochrane Handbook Chapter 13 — Risk of Bias Due to Missing Evidence in a Meta-Analysis
- PRISMA 2020 Statement — reporting systematic reviews
- eduKateSG — How Research Bias Works
- eduKateSG — How Preregistration Works
Final Thought: The Average Is Not the Evidence
A meta-analysis ends with a number.
That is why it is so easy to misunderstand.
The number came from a literature.
The literature came from studies.
The studies came from samples.
The samples came through selection rules.
The outcomes came through measurement systems.
The papers came through publishing filters.
The effect sizes came through analytic choices.
The weights came through a statistical model.
The average sits on top of all of them.
A world-class synthesis therefore does not ask only:
What is the pooled effect?
It asks:
What kind of evidence had to survive which filters before this pooled effect could exist—and what differences did the average hide?