Three school Science projects test whether compost changes seedling growth. One reports a noticeable increase, another a tiny difference and the third shows slightly less growth in the compost group. A reader might choose the most attractive graph, declare that the majority wins, or average every number together. None of those reactions necessarily makes good scientific sense. The gold standard of school scientific multi-study evidence synthesis is to combine what different investigations genuinely establish while protecting the meaning, limitations and independence of each study.
Quick answer: Identify the shared research question, list each study’s actual population, treatment, comparison, outcome, time frame and method; verify whether measurements are sufficiently comparable; distinguish planned tests from exploratory observations; report the direction, size and uncertainty of findings; examine contradictory evidence rather than discard it; and write a bounded conclusion. Combining studies is not simply adding all their measurements to one spreadsheet.
A practical reader’s route
- Why one successful experiment is not the whole story
- Define a common question before combining data
- The study is the unit of comparison
- Worked example: three seedling experiments
- Why the same centimetre difference need not mean the same thing
- Arithmetic that clarifies instead of concealing
- Quality and bias across studies
- How to treat contradictory findings
- Protect against selecting only positive studies
- What independent replication adds
- Explain the balance of evidence clearly
- A classroom teaching sequence
- Three learner pathways
- Break-tests for synthesis mistakes
- Frequently asked questions
- Sources and connected reading
1. One experiment can answer one question well and still leave a wider question open
A school investigation takes place under particular conditions. It may use one plant variety, one classroom, a short observation period and a small set of pots. When well planned, it can provide useful evidence about what happened in those conditions. But a confident statement such as ‘compost always increases plant growth’ extends beyond that evidence. A second or third independently designed investigation can reveal whether the relationship appears again and whether it transfers to a different context.
Synthesis is a way of reasoning across the available studies. It asks whether the investigations measured comparable things, whether their evidence is credible and which conclusions survive the differences among their results. It is not a popularity contest between teams. One careful experiment can provide more relevant evidence than several poorly controlled demonstrations. Equally, a genuine disagreement between well-conducted studies may identify an important environmental factor rather than a mistake.
Pupils should learn to describe both common ground and differences. If all three studies show that seedlings grew, that observation does not establish whether the compost treatment changed growth relative to the comparison. The meaningful contrast is between the treatment and control conditions within each study. Synthesis should first preserve those contrasts and their original definitions.
This guide provides a school-level framework. It does not ask Primary pupils to conduct professional meta-analysis or claim that a simple pooled mean can establish causality. Its purpose is to build the habit of assessing evidence collectively without erasing the design decisions behind each result.
2. Start with a question precise enough for all studies to answer
Suppose three titles all contain the words ‘compost and plant growth’. One project measures stem height after two weeks, another measures leaf number after a month and another records germination success. They are not automatically three replications of the same outcome. They ask related but different questions. Before comparing results, specify the scientific quantity and conditions that would make a result relevant.
A useful common question might be: ‘Under comparable growing conditions, does the specified compost-medium treatment increase mean stem-height gain over fourteen days for a particular seedling variety?’ A study using a different species or observation period can still be informative, but its directness should be described. A study that measures only leaf colour cannot simply be inserted into the same height-gain calculation because it happens to show a favourable effect.
The experimental unit also matters. Six leaves from one treated plant are not the same as six independently prepared and treated pots. A study with one large tray per condition might produce many height measurements but offer limited independent treatment replication. The table of studies should record how treatments were assigned, not only how many rows appear in the spreadsheet.
A defined outcome also prevents retrospective switching. If an initial study selected leaf colour only after seeing a striking pattern, distinguish its exploratory role from a new independent test that planned a colour measure in advance. Both can contribute to understanding, but they do not carry identical evidentiary weight for a single confirmatory claim.
3. The first comparison is between studies, not between their individual rows
Each experiment has a package of assumptions: who or what was sampled, how the treatment was assigned, what other conditions were kept comparable, how the outcome was measured and when observations were recorded. Pooling raw rows before understanding those packages can make different studies appear as though they were one uniform large experiment. That hides changes in exposure, duration or measurement rules.
A study-level matrix is an accessible first tool. It names the research question, groups, sample size, measured outcome, time frame and significant limitations. The child can then compare study contrasts in a coherent way. The objective is not maximum detail; it is enough information to judge whether the same claim is being tested.
| Study feature | Why it matters | Synthesis question |
|---|---|---|
| Seedling variety and starting state | Biological response can vary | Are these comparable plants? |
| Independent units and assignment | Controls treatment allocation | Were treatments independently assigned? |
| Treatment and reference condition | Defines the actual contrast | Was the compost mixture identical or different? |
| Measured quantity and unit | Protects semantic comparability | Are both studies reporting stem-height gain? |
| Observation period | Changes opportunity for growth | Did measurements span the same duration? |
| Method and observer criteria | Influences reading consistency | Were heights measured from a common reference? |
| Missing data and survival | May alter the analysed sample | Which original units contributed to the result? |
A compact comparison table is often more useful than a single average calculated too early. If differences are important, the appropriate output may be a narrative synthesis of each study’s findings and limitations instead of one numerical pooled estimate.
4. Worked example: three fictional compost investigations
Consider three invented classroom investigations with independent pots assigned to compost and base-medium comparisons. All record stem-height gain in centimetres, but they differ in period and plant variety. The values are teaching examples, not real research findings, and do not establish how any particular commercial compost performs.
| Study | Plant and period | Independent pots per group | Control mean gain | Compost mean gain | Difference |
|---|---|---|---|---|---|
| A | Variety V, 14 days | 6 + 6 | 3.0 cm | 3.5 cm | +0.5 cm |
| B | Variety V, 21 days | 10 + 10 | 4.0 cm | 4.1 cm | +0.1 cm |
| C | Variety W, 14 days | 4 + 4 | 3.0 cm | 2.8 cm | −0.2 cm |
In Study A the compost group gained 0.5 cm more on average. In Study B it gained 0.1 cm more over a different duration. In Study C the compost group gained 0.2 cm less for a different seedling variety. A weak conclusion says, ‘Compost worked in two studies out of three, so the majority proves that it works.’ That conclusion treats all results as equal votes and ignores magnitude, measurement uncertainty, period and variety.
Another weak conclusion is ‘The three studies contradict one another; therefore Science is unreliable.’ The studies may have measured genuine differences under different conditions. The correct synthesis asks how much variation a model of compost response could reasonably predict, whether the experiments were sufficiently comparable and what evidence would distinguish chance variation from a material contextual effect.
The observed direction alone does not describe certainty. A small positive difference could be within routine trial variability; a negative difference may reflect a different treatment mixture or water exposure. Without the underlying variation, valid measurement resolution and design information, we should not invent a statistical significance result.
5. Do not pool different durations as though they were one identical measurement
Studies A and B report growth over fourteen and twenty-one days respectively. Their mean gains are both in centimetres, but the time frame is part of the outcome. A total gain over three weeks is not automatically comparable with a total gain over two weeks. One could calculate simple centimetres per day for illustration, but that assumes a rate model appropriate to the growth process. Seedlings may grow nonlinearly, and time-averaging cannot eliminate all environmental differences.
Study C uses another variety. Even with the same fourteen-day interval, its underlying biological response may differ. A synthesis can report that the result for Variety W was negative in this small study, but should not treat it as proof that Variety V observations were false. A broader hypothesis that compost acts identically on all varieties is different from a narrow hypothesis about Variety V under one classroom’s conditions.
The treatment itself should be examined. ‘Compost’ can refer to mixtures with different nutrients, moisture retention and structure. A base medium with different physical properties may also change plant response. If studies use materially different mixtures, a single label can conceal different interventions. A good evidence table records what was actually compared.
Measurements can differ even when units and periods match. Some teams may measure from soil level and others from the pot rim, or use different definitions of the tallest point. Observer agreement and appropriate apparatus affect whether a small study-level difference deserves weight. Comparable numbers require comparable meaning.
6. Arithmetic can illustrate the problem with naïve synthesis
A simple arithmetic mean of the three illustrative study differences is (+0.5 + 0.1 − 0.2) ÷ 3 = approximately +0.13 cm. The calculation is correct for those three numbers. However, that average ignores unequal group sizes, time frames, varieties and the precision of the studies. It is not a scientifically justified overall treatment-effect estimate.
If someone weights the contrasts by the number of pots per treatment arm, the calculation is (6 × 0.5 + 10 × 0.1 + 4 × −0.2) ÷ (6 + 10 + 4) = (3.0 + 1.0 − 0.8) ÷ 20 = +0.16 cm. This number is arithmetically valid for the chosen weighting scheme. It is not automatically a reliable pooled effect either. We have not established that sample-size weighting is an appropriate statistical model or that the study outcomes can be treated as equivalent.
Formal research synthesis may consider study variance, uncertainty, heterogeneity and the question of whether an effect should be expected to vary across settings. Those methods go beyond the needs of most Primary Science lessons. The child-friendly lesson is simpler: a convincing-looking calculation can conceal important differences between what the studies actually measured.
For this teaching example, a responsible synthesis might say that observed study-level differences ranged from −0.2 cm to +0.5 cm, with two positive and one negative result, but the studies used different periods and varieties. The evidence does not establish one universal compost effect from these three examples. That is a more meaningful statement than presenting +0.16 cm as though it were a verified prediction for every plant.
7. Study quality matters more than which result a reader likes
Evaluate whether each study had a meaningful control, appropriate assignment, independent treatment units, comparable relevant conditions and a consistent outcome definition. A study with a dramatic difference may have placed all treated plants in stronger light, while a smaller study may have carefully randomised and measured its units. The size of the observed effect is not a substitute for design quality.
Also check whether the outcome was selected in advance. An exploratory finding identified among numerous plant features is useful for generating a hypothesis, but should not be treated as identical to a controlled follow-up that pre-specified that exact outcome. The more appealing the finding, the more important it is to ask what else was measured and whether contrary results were also reported.
Missing outcome data deserve attention. Suppose the weakest compost seedlings die and are omitted from final-height measurements. A survivor-only average may overstate the treatment’s overall performance. If another study reports all assigned pots, comparing the two means without noting attrition can be misleading. Good synthesis makes achieved sample size and exclusion reasons visible.
Measurement fidelity is another factor. A thermometer study with inadequate resolution may fail to detect a real small difference; a height comparison with ambiguous endpoints may manufacture one. Quality assessment must identify the problem and its likely effect rather than simply label a whole study ‘good’ or ‘bad’.
8. Contradictory evidence is often the reason to investigate further
If results disagree, begin by comparing the scientific questions, not by searching for blame. Did the same factor change? Were the controls comparable? Were the measured quantities identical? Did the observations occur at the same time or development stage? A mismatch may explain why the studies produced different findings without implying dishonesty by either group.
A second possibility is ordinary sampling variation. Small groups of living organisms can differ even after a fair assignment. Repeating a study with genuinely new independent units may help determine whether a pattern is stable. However, a repeated result is not automatic proof; a shared systematic flaw can be reproduced faithfully.
A third possibility is genuine context dependence. Compost could perform differently in two growing media or under different moisture conditions. In that case, the scientific model should include those conditions rather than demand one direction everywhere. A useful follow-up might test a clearly specified interaction between growing medium and water availability, under teacher-approved safe methods.
Choose the next investigation based on the disagreement that needs resolving. If the main difference is observation period, establish a comparable time frame. If observer criteria differ, calibrate the measurement. If plant variety differs, explicitly ask whether the treatment effect varies with variety. Repeating the same vague experiment without addressing the suspected cause may produce another ambiguous graph.
9. Beware of the missing studies that never reached the poster
A class display often highlights the best results. Projects showing small or contrary effects may receive less attention. If a teacher synthesises only positive-looking graphs, the overall impression may be biased even when each displayed graph is accurate. A fair evidence review attempts to include all relevant studies according to a clear rule, not just those whose findings support the preferred story.
Students can learn a simple inclusion plan: identify which projects measured the same outcome over a relevant period and include their results regardless of direction. If one project measured only leaf colour, explain why it belongs in a separate analysis rather than exclude it silently as inconvenient. The reader should see the rule and how it was applied.
The absence of published or submitted negative results can also limit professional evidence synthesis. School-level readers need not conduct formal publication-bias modelling to understand the core risk. When only winners are displayed, the audience may overestimate how often an effect appears.
Honest synthesis can contain a mixed verdict. ‘Some studies found a higher mean gain, some did not, and differences in design prevent a firm universal conclusion’ is an appropriately useful summary when that is what the evidence shows.
10. Independent replication adds evidence only within its real scope
A second class measuring the same original plants is not necessarily an independent replication. It may improve observer agreement or identify a recording error, but it does not produce new treatment assignments. A fresh study that allocates new independent pots and follows a defined outcome can test whether the relationship reappears. The two activities contribute different kinds of evidence.
A replication that uses the same teaching materials and a common measurement protocol can reduce some methodological differences. However, other differences may remain: light, water, seedling variety, season or instructor. Document those rather than assume they are identical. If a replication is deliberately conducted under another condition, say that it tests transfer rather than exact repeatability.
When several studies agree, confidence may improve, but the shared research question and study quality still matter. Ten repeated trials using one flawed balance do not establish an accurate weight. Similarly, ten projects conducted with one mislabeled treatment cannot become reliable evidence merely by repetition.
The synthesis should record both successes and failed replications. A negative follow-up is a scientific result, not a failed public-relations exercise. It may narrow confidence in the original exploratory claim or reveal a condition that deserves new testing.
11. How to communicate a balance of evidence to readers
Use a short conclusion that answers the original question immediately, followed by a table of the studies and a paragraph identifying the main reason their results may differ. Readers should not have to scroll through methodological terminology to discover that the studies produced mixed findings. At the same time, a one-line verdict without units, periods or source descriptions is too thin.
A suitable summary for the fictional example is: ‘Across three small teaching investigations, the difference in mean stem-height gain between compost and base-medium groups ranged from 0.2 cm lower to 0.5 cm higher. The studies differed in observation duration and plant variety, so we cannot treat the differences as one directly comparable treatment-effect estimate. A new study using a common fourteen-day outcome and the same variety would help.’
Use accessible charts. Show distinct studies rather than blending them into an unexplained pooled bar. Label treatment, reference condition, time frame and units near the data. Colour should not be the only way to distinguish positive and negative contrasts. Students and parents should be able to interpret the result without assuming that the tallest bar automatically represents the best evidence.
This communication habit matters outside school. Readers encounter studies of health, climate and education that differ in context and quality. Asking what was measured, what was compared and how independent the evidence is helps prevent confident claims from outrunning their sources.
12. An illustrative 90-minute teaching sequence
| Time | Activity | What the learner demonstrates |
|---|---|---|
| 0–10 minutes | Read three contrasting project headlines. | Recognises that majority vote is not enough. |
| 10–25 minutes | Map study outcome, population and duration. | Identifies similarities and differences. |
| 25–45 minutes | Compute each study contrast and simple illustrative averages. | Performs arithmetic without declaring unjustified pooling. |
| 45–60 minutes | Identify bias, missingness and measurement issues. | Explains quality differences concretely. |
| 60–75 minutes | Debate explanations for conflicting findings. | Considers context and chance without blame. |
| 75–85 minutes | Write a balanced evidence summary. | States result and limitations clearly. |
| 85–90 minutes | Suggest an appropriate confirming follow-up. | Targets one meaningful uncertainty. |
The timing is a teaching suggestion, not an official MOE requirement. The teacher may use the invented data without conducting three live plant experiments. Real fieldwork should follow appropriate school safety and observation procedures. The learning outcome is an ability to compare evidence, not to produce a winning treatment.
13. Repair, stabilise and extend different learners
Repair: the learner thinks two out of three proves the majority
Show three simple charts with clear differences in sample size and observation time. Ask whether one study of a different variable should count as a vote about the original question. The child should first identify the shared measured quantity and the importance of conditions. Avoid adding weighted formulas until the logical difference between the studies is understood.
Use a contrasting example: two students measure the height of one plant from different reference points while another measures correctly. A majority of similar results does not make the method accurate. Synthesis requires examining how each result was produced.
Stabilise: the learner can compute contrasts but combines them carelessly
Give the seedling matrix and ask for +0.5, +0.1 and −0.2 cm contrasts. Then ask why the 21-day study differs from a 14-day study. Have the learner write a summary that reports all three without calling an unweighted mean a universal treatment effect. Correct the reasoning by identifying the exact assumption that pooling would require.
Introduce one relevant study-quality concern at a time, such as a different plant variety or survivor-only reporting. Pupils should explain how that factor could change interpretation instead of making vague declarations that every study has errors.
Extend: the learner can design a stronger synthesis plan
More advanced students can propose inclusion criteria, a consistent outcome definition and an independent quality checklist before looking at study results. They can distinguish fixed versus changing contexts, evaluate when rates may be comparable and explain why different weights need scientific justification. This is introductory evidence synthesis, not a requirement to perform formal meta-analysis.
Invite pupils to design a new controlled comparison that addresses one observed disagreement. Their question should be specific enough that a result in either direction will add useful evidence, not merely reproduce a preferred story.
14. Break-tests for a tempting but misleading pooled number
- Two positive results and one negative result appear. Does the majority automatically establish causation?
- One study measures height over fourteen days and another over twenty-one. Can their raw centimetre differences be pooled without a method?
- A study has 24 leaf readings from two shared trays. Is that stronger independent replication than twelve separate pots?
- One team reports only surviving seedlings. Can its mean be compared uncritically with a study including all original units?
- The same measurements are copied into two posters. Do those count as independent studies?
- A negative study is removed because it spoils the headline. What kind of selection bias results?
- Three averages produce a precise-looking overall decimal. Does correct arithmetic establish scientific comparability?
- The same effect appears across two plant varieties. Does that alone establish a result for every species?
Passing these checks means the learner understands the balance of evidence rather than treating a synthesis as a contest among bar charts. A convincing summary needs a common question, meaningful data and honest treatment of the studies that disagree.
15. Frequently asked questions
Is evidence synthesis the same as repeating an experiment?
No. Repeating or replicating collects or evaluates a new test. Synthesis compares and interprets findings across studies, considering their methods, quality and relevance.
Can I average the effects from different studies?
Arithmetic averages can be calculated, but a scientifically meaningful pooled estimate requires comparable outcomes and an appropriate justified method. Do not assume equal weighting or sample-size weighting is always correct.
Does a majority of studies prove a hypothesis?
No. Study quality, independence, outcome meaning, uncertainty and possible bias matter more than simply counting positive signs.
What if the studies use different plant varieties?
Their results can still inform a broader question, but the difference limits direct comparability. A follow-up could examine whether the treatment response changes across varieties.
Is a negative replication useless?
No. It can limit confidence in a broad claim, expose methodological differences or identify which conditions deserve further study.
Can several worksheets from the same original dataset count as separate studies?
Not as independent experimental evidence. They may be different analyses of the same underlying observations, which should be described honestly.
Do Primary pupils need formal meta-analysis statistics?
No. They can learn evidence synthesis by comparing the questions, measured outcomes and strengths of several investigations and writing a balanced conclusion.
What is the most important rule?
Do not make the combined conclusion stronger than the studies’ actual meaning and quality justify.
16. Sources and connected Gold Standard reading
The Singapore MOE Primary Science Teaching and Learning Syllabus develops observing, comparing, analysing, inferring and communicating evidence. The multi-study synthesis approach in this guide is an educational extension, not an official instruction that every Primary pupil must undertake professional statistical meta-analysis.
Read Replication Disagreement Interpretation for comparing conflicting experiments; Exploratory Findings Confirmation Planning for designing fresh evidence; Pre-Specified Outcome Definitions for outcome meaning; and Missing Outcome Data Analysis for attrition and incomplete results. This page has the separate task of combining the implications of more than one study without erasing their differences.
Final principle: A trustworthy scientific synthesis does not turn several experiments into one bigger-looking number before asking whether they answered the same question. It lets every study contribute what its evidence actually supports, including the findings that complicate the preferred story.