Peer review works by placing a scholarly manuscript in front of editors and independent people with relevant expertise, asking them to identify important weaknesses, uncertainties and strengths, giving authors an opportunity to respond and revise, and leaving the final publication decision with the editor rather than treating reviewer recommendations as a vote or a certificate of truth.
There is a small sentence that can cause a great deal of confusion:
This study was peer reviewed.
To a casual reader, that can sound like:
Experts checked it, so it is correct.
That is not what peer review means.
Peer review is much more interesting—and much more human—than a truth stamp. It is a structured attempt to make a manuscript encounter informed resistance before it enters a journal’s published record. Reviewers may find a missing control, an unjustified inference, a statistical weakness, a neglected literature, a confusing explanation, an ethical concern or a conclusion that travels farther than the evidence permits. Authors may repair the problem, defend their reasoning, perform additional analyses where justified, narrow the claim or discover that the paper cannot yet survive scrutiny.
Sometimes reviewers disagree strongly with one another. Sometimes an editor rejects a paper despite one favourable review. Sometimes an editor accepts after authors resolve serious concerns. Sometimes peer review misses an error that becomes obvious after publication.
That messiness is not an accidental detail.
It tells us what peer review actually is: a human checkpoint inside a larger self-correcting research system.
The governing question: before this claim becomes part of the durable scholarly record, have competent outsiders had a meaningful opportunity to challenge the route from question to evidence to conclusion?
Quick Read
MANUSCRIPT → JOURNAL FIT → EDITORIAL TRIAGE → REVIEWER SELECTION → CONFLICT CHECK → REVIEW SCOPE → INDEPENDENT CRITIQUE → REPORTS → EDITORIAL INTEGRATION → AUTHOR RESPONSE → REVISION → RE-REVIEW WHERE NEEDED → EDITORIAL DECISION → PUBLICATION → POST-PUBLICATION SCRUTINY → CORRECTION / REPLICATION / SYNTHESIS
The chain is not universal. Different journals use different models. Some review anonymously. Some publish reviewer reports. Some invite specialist statistical review. Some use registered reports that evaluate study plans before results exist. Some manuscripts are rejected editorially without external review. But the central structure remains recognisable: peer review creates a deliberate encounter between a manuscript and informed criticism before, during or around publication.
1. Peer Review Begins After Research but Before Publication—Usually
In the conventional journal model, researchers first conduct a study, analyse the evidence and write a manuscript. The manuscript is then submitted to a journal.
Peer review enters after submission and before the final publication decision.
But even this familiar sequence now has important exceptions. Registered Reports move part of peer review earlier, before data collection or before researchers inspect the relevant existing dataset. Preprints can make manuscripts publicly visible before formal journal peer review is complete. Post-publication commentary can continue scrutiny long after the journal issue appears.
So “peer review” is best understood as a family of structured scrutiny processes rather than one single historical ritual.
2. The Editor Is the First Gatekeeper
A manuscript does not automatically go from submission to external reviewers.
Editors typically perform an initial assessment. They may ask whether the paper fits the journal’s scope, reaches the expected level of importance or technical contribution, follows essential policies, contains obvious methodological problems, and is sufficiently complete to justify the time of external reviewers.
Nature’s current peer-review policy makes this explicit: all submitted manuscripts are read by editorial staff, and only those considered likely to meet the journal’s editorial criteria are sent for formal review.
This first gate is sometimes called editorial triage or desk review.
3. A Desk Rejection Is Not the Same as “Science Says the Paper Is Wrong”
A journal can decline a paper without claiming that its central result is false.
The work may be too specialised for that journal, insufficiently novel for its editorial mission, outside scope, too preliminary, poorly presented, technically weak, or simply better suited to another venue.
This matters when readers interpret prestige. Rejection from a selective journal is not a universal scientific verdict. Acceptance by one is not a universal proof of truth either.
Journal selection and scientific validity overlap, but they are not identical dimensions.
4. Reviewer Selection Is Part of the Scientific Architecture
If the editor sends a manuscript for external review, the next problem is deceptively hard:
Who is qualified to challenge this particular paper?
A manuscript may combine several kinds of expertise. A climate study may need domain science, statistical modelling and remote sensing. A clinical trial may need clinical knowledge, trial methodology and statistics. An AI paper may need model architecture, evaluation methodology, security or domain expertise depending on the claim.
No single reviewer necessarily sees the whole machine.
Editors therefore assemble a small temporary panel of perspectives, often two or three reviewers in many journal systems, and may add specialised review when a particular technique or statistical issue requires it.
5. “Peer” Does Not Mean “Person Who Agrees With the Authors”
The useful peer is someone sufficiently knowledgeable to understand the work and sufficiently independent to criticise it.
That independence is important.
A reviewer should not be chosen because they are likely to approve the paper. Nor should they be chosen because they are known to oppose the authors. The purpose is not to manufacture a verdict. It is to obtain informed scrutiny from people whose expertise is relevant to the manuscript’s claims and methods.
This is one reason reviewer selection remains an editorial judgement rather than a purely mechanical matching task.
6. Conflicts of Interest Can Corrupt the Review Before the First Comment Is Written
Reviewers can have financial, professional, intellectual or personal relationships that make impartial assessment difficult.
A close collaborator may be too invested in the authors’ programme. A direct competitor may have incentives to delay or obstruct. A reviewer may have a financial interest in a competing technology. A public dispute may make fair judgement difficult.
Good journal systems therefore ask reviewers to disclose conflicts and decline assignments they cannot evaluate impartially.
The principle is simple: expertise is necessary, but expertise without independence can create a distorted gate.
7. Confidentiality Protects Unpublished Work—Unless the Review Model Is Explicitly Open
Traditional peer review often treats the manuscript and review process as confidential. Reviewers should not use unpublished ideas, methods or results for their own advantage, and they should not casually share the manuscript with colleagues.
But peer-review systems vary. Some journals publish reviewer reports, author responses and decision letters. Some reveal reviewer names with permission. Others keep identities hidden.
The correct rule is therefore not “peer review is always secret”.
It is: follow the declared review model, preserve confidentiality where promised, and never treat privileged access to unpublished work as personal property.
8. The Reviewer Needs to Know the Scope of the Assignment
Reviewing a manuscript does not automatically mean verifying every possible part of the research.
Is the reviewer expected to inspect raw data? Re-run code? Judge image integrity? Audit every reference? Evaluate only statistical methods? Review the whole manuscript?
Different journals and article types ask for different levels of scrutiny.
A high-resolution review system makes the scope explicit because readers otherwise imagine a much stronger verification process than actually occurred.
9. Reviewers First Reconstruct the Paper’s Claim
Before judging a manuscript, a careful reviewer needs to understand what it is actually claiming.
- What question is being asked?
- What is the central contribution?
- What evidence supports it?
- What population, material, system or setting does the claim cover?
- Is the paper descriptive, causal, predictive, mechanistic, methodological or interpretive?
A surprising number of review disagreements begin because different people have reconstructed different claims from the same manuscript.
Clarity is therefore not cosmetic. A paper that cannot state its own claim clearly is difficult to evaluate fairly.
10. The Research Question Must Be Worth the Method Used to Answer It
A technically impressive analysis can still be attached to a trivial, poorly framed or already resolved question.
Reviewers often examine whether the research question matters, whether it is positioned accurately against prior literature, and whether the paper exaggerates novelty by ignoring earlier work.
This is where citation becomes part of peer review. References do not merely decorate an introduction; they show the lineage of the question and whether the claimed gap actually exists.
11. Methods Are the Recoverable Route Between Claim and Evidence
A reviewer cannot judge the result properly if the method is too vague to reconstruct.
Who was sampled? What was measured? What instrument was used? What exclusions occurred? What intervention was delivered? What controls existed? What preprocessing happened? Which outcomes were primary? How were missing data handled? Which model produced the estimate?
Reviewers look for mismatches between the scientific job and the chosen design.
A paper may ask a causal question using data that support only association. It may make population claims from a convenience sample. It may use a proxy without validating its relationship to the construct. It may treat repeated measurements from one unit as independent observations.
The methods section is where many such failures become visible.
12. Sampling Is a Peer-Review Problem, Not Just a Statistics Problem
A reviewer should ask whether the sample can support the population the authors discuss.
A very large sample can still exclude important groups. Attrition can turn the final analytic sample into something quite different from the recruited sample. Clustered samples may require special variance estimates. Rare subgroups may be too small for confident claims.
The relevant question is not only “Is n large enough?”
Who had a path into the evidence, and who did not?
See How Sampling Works.
13. Measurement Is Another Hidden Gate
A manuscript may discuss learning, stress, biodiversity, poverty, system reliability or disease severity as though these were directly visible.
They are usually represented through instruments, operational definitions, coding rules, sensors, tests or proxies.
A reviewer asks whether those measurements are reliable and valid enough for the conclusion being made.
Precise statistics cannot rescue an instrument that measures the wrong construct.
14. Statistical Review Asks Whether the Numbers Support the Inference
Not every domain expert is a specialist in every statistical technique.
Complex papers may therefore need a reviewer with specific methodological or statistical expertise.
The questions can include:
- Does the model match the data-generating structure?
- Are observations independent where independence is assumed?
- Were multiple comparisons handled appropriately?
- Are effect sizes and uncertainty reported?
- Were model assumptions checked?
- Were missing data treated defensibly?
- Do figures and tables match the analysis described?
- Is a p-value being interpreted as something it is not?
Statistical sophistication should increase scrutiny, not lower it.
15. Reviewers Should Look for the Difference Between Association and Causation
This is one of the most consequential language checks in research.
If two variables move together, the paper may legitimately describe an association. A causal claim requires a stronger route through design, counterfactual logic and alternative explanations.
Reviewers often perform an important service simply by forcing verbs to match the evidence:
associated with is not automatically caused by.
predicted is not automatically explained.
consistent with is not automatically proved.
16. Reviewers Test the Distance Between Results and Conclusions
The Results section may be technically correct while the Discussion outruns it.
A modest effect becomes “transformative”. A short-term outcome becomes “long-term improvement”. A laboratory model becomes a claim about society. A subgroup analysis becomes a universal recommendation.
One of peer review’s central jobs is claim calibration.
Does the conclusion stop where the evidence stops?
17. Reviewers Also Inspect What the Paper Does Not Show
A manuscript can be persuasive partly because missing information is invisible.
Were outcomes dropped? Were exclusions introduced after results were known? Were null findings omitted? Was a control condition inadequately described? Are raw counts missing behind percentages? Was a subgroup selected after the main analysis failed?
Reviewers cannot always reconstruct what they were never given, but reporting guidelines, preregistration, trial registration, protocols, data availability and code can create additional surfaces for comparison.
Visibility is a methodological asset.
18. Reporting Guidelines Help Reviewers Ask Whether Essential Information Is Missing
Different study designs have developed reporting frameworks such as CONSORT for randomised trials, STROBE for observational studies and PRISMA for systematic reviews.
These do not guarantee that a study is good.
They improve the chance that readers and reviewers can see important design and reporting details needed to evaluate it.
A beautifully written paper can still be impossible to audit if essential methodological information never enters the manuscript.
19. Ethics Review and Peer Review Are Different Gates
Research involving people, animals, sensitive records or other regulated domains may require formal ethics approval before or during the research process.
Journal peer reviewers may inspect whether appropriate ethical approvals, consent processes and disclosures are reported, but journal peer review is not a substitute for institutional ethics review.
Nor does prior ethics approval guarantee that every scientific claim is valid.
The gates protect different dimensions of the work.
20. A Reviewer Is Not Supposed to Redesign the Paper Into Their Own Study
Peer review can become destructive when a reviewer treats personal preference as universal methodology.
“I would have done this differently” is not enough.
The relevant question is whether the authors’ chosen method can answer the stated question under defensible assumptions.
A good review distinguishes:
- a fatal methodological flaw;
- a necessary clarification;
- a useful robustness check;
- a valuable future study;
- and the reviewer’s personal preference.
Without that distinction, review can become scope creep disguised as rigour.
21. “Reviewer 2” Is a Joke Because Review Is Also Social
Researchers joke about the impossibly demanding reviewer who requests three new experiments, a rewritten theory and perhaps a new universe.
The joke persists because peer review is not performed by abstract rational agents. It is performed by people.
Reviewers can be generous, impatient, insightful, territorial, brilliant, tired, biased or mistaken.
A strong editorial process does not pretend those human properties disappear. It uses multiple reviewers, conflict checks, editorial judgement, author rebuttal and sometimes additional expertise to reduce the chance that one reviewer’s personality becomes the entire decision system.
22. Reviewers Can Disagree Because Expertise Is Distributed
One reviewer may admire the method and question the interpretation. Another may accept the interpretation but notice a sampling problem. A third may identify a technical issue neither of the first two recognised.
Disagreement is therefore not automatically evidence that peer review has failed.
A 2026 analysis of thousands of editorial records found that reviewer discordance was common, especially in early rounds, reinforcing a crucial point: peer review operates as adjudication, not simple majority voting.
Different eyes resolve different parts of the paper.
23. The Editor Integrates Arguments Rather Than Counting Votes
Suppose three reviewers recommend:
- accept;
- major revision;
- reject.
There is no scientifically meaningful average of those labels.
The editor needs to examine the reasons.
Does the rejecting reviewer identify a fatal design flaw? Is the objection based on a misunderstanding the authors can clarify? Does the favourable reviewer possess the relevant specialist expertise? Are the requested experiments essential to the claim or merely interesting?
Nature explicitly states that editorial decisions are not made by counting reviewer votes and that the editor weighs the arguments and the journal’s responsibilities to readers and the scientific community.
24. The Reviewer Report Is Evidence for an Editorial Decision
A useful reviewer does not merely say “accept” or “reject”.
They explain why.
A strong report reconstructs the paper, identifies major issues, separates essential from optional changes, points to specific evidence and gives the editor enough information to understand the reasoning.
This is a small version of the scientific method itself:
Do not only give the conclusion. Preserve the route.
25. Major and Minor Comments Should Mean Different Things
A major concern threatens the central inference, validity, reproducibility, ethical acceptability or interpretability of the work.
A minor concern may involve clarity, missing context, local reporting details, figure labels or wording that can be repaired without changing the paper’s underlying scientific structure.
If every comment is presented at the same priority, authors and editors cannot see what actually determines the decision.
Good review therefore includes prioritisation.
26. Authors Are Allowed to Disagree With Reviewers
Revision does not mean obedience.
An author can explain why a requested change would be inappropriate, impossible, outside scope or based on a misunderstanding.
The burden is to respond with evidence and reasoning rather than irritation.
A good response letter shows each concern, the authors’ response, the exact change made—or a clear justification for not making it.
This turns disagreement into a traceable intellectual exchange.
27. Revision Is Where Peer Review Often Creates Its Greatest Value
The public usually sees only the published paper.
It may never see the earlier version with an unclear method, overextended claim, weak figure or missing robustness check.
Peer review can improve the scientific record by forcing those weaknesses to be repaired before publication.
But the amount of improvement varies. Some papers arrive strong. Some reviews add little. Some reviewer requests make a paper worse. The process has no automatic quality guarantee.
Its value comes from the quality of the criticism and the quality of the response.
28. Re-Review Asks Whether the Repair Actually Solved the Problem
After revision, an editor may decide independently or return the manuscript to some or all reviewers.
Re-review is useful when important technical concerns require specialist confirmation.
The question changes from:
What is wrong with this paper?
to:
Did the revision resolve the concern without creating a new one?
29. An Editorial Decision Is a Decision Under Uncertainty
Editors rarely know that a paper is “true”.
They decide whether the manuscript, after review and revision, meets the journal’s standards strongly enough to enter publication.
The decision can include scientific validity, significance, journal fit, clarity, policy compliance and the strength of unresolved concerns.
It is therefore more accurate to say:
The journal judged the manuscript publishable under its process.
That is a meaningful signal.
It is not an oracle.
30. Acceptance Does Not Mean Every Reviewer Became Convinced
A manuscript can be accepted even if one reviewer remains sceptical.
Perhaps the unresolved concern is not fatal. Perhaps another specialist explains why it is not applicable. Perhaps the authors narrow the claim so the concern no longer threatens the central inference.
Conversely, several favourable recommendations do not force acceptance if an editor identifies a serious problem.
The decision belongs to the editorial process, not to a referendum among reviewers.
31. Rejection Can Improve a Paper Without Proving the Paper Was Bad
A rejected manuscript may later be revised substantially and published elsewhere.
The rejected version and published version are not the same object.
Criticism can expose an issue, authors can repair it, new evidence can be added, claims can be narrowed and the paper can become stronger.
Peer review therefore produces information even when it does not produce acceptance.
32. Single-Anonymised Review Hides the Reviewer but Not the Author
In a common review model, reviewers know the authors’ identities while authors do not know the reviewers’ identities.
The intended benefit is reviewer candour without fear of retaliation.
The concern is that visible author identity, institution, country or reputation may influence judgement consciously or unconsciously.
No anonymity model eliminates every bias. It changes which information pathways remain open.
33. Double-Anonymised Review Hides Author Identity From Reviewers Where Possible
Double-anonymised review attempts to conceal authors’ identities from reviewers while also concealing reviewers’ identities from authors.
The aim is to reduce status and identity cues.
In specialised fields, however, reviewers may infer authorship from topic, datasets, citations, methods or preprints. Anonymisation can reduce some signals without making authorship unknowable.
Nature Portfolio currently offers double-anonymised options across parts of its journal family, illustrating that review models are design choices rather than fixed laws of science.
34. Open and Transparent Peer Review Expose More of the Route
Open peer review can mean several different things: reviewer identities may be disclosed, reports may be published, author responses may be public, or readers may be able to comment openly.
Transparent peer review commonly preserves a public record of reviewer comments, author rebuttals and sometimes editorial decision letters.
The benefit is inspectability. Readers can see some of the criticism that shaped the final paper.
The trade-offs include reviewer privacy, willingness to criticise powerful authors, and the additional social dynamics created when reports become public.
Transparency changes the evidence available about the review. It does not automatically make every review good.
35. Peer Review Can Be Biased Because Reviewers Are Human
Research on peer review has long examined possible biases related to prestige, institutional affiliation, nationality, language, gender, theoretical allegiance and the direction of results.
The evidence is heterogeneous across disciplines and review systems, so it is unwise to reduce the problem to one universal bias estimate.
The more durable lesson is structural: if irrelevant information can influence judgement, review design should ask whether that information needs to be visible, whether multiple reviewers can counterbalance one another, and whether editorial decisions can be audited.
36. Peer Review Is Not Designed to Detect Every Fraud
This boundary is essential.
Reviewers usually do not visit the laboratory, interview participants, inspect every raw file, repeat the experiment or recreate every analysis from scratch.
A review of the peer-review literature notes that reviewers may detect some misconduct or serious errors, but conventional peer review should not be treated as a reliable fraud-detection system.
Fraud detection may require data audits, image forensics, institutional investigation, whistle-blower evidence, replication or post-publication scrutiny.
Peer review can make deception harder.
It cannot make deception impossible.
37. A Reviewer Cannot Replicate the Study During Ordinary Review
If a paper reports a three-year field study, the reviewer cannot usually conduct another three-year field study before the journal makes a decision.
If a paper reports a clinical trial, reviewers cannot recruit another population and rerun it.
This is why peer review and replication must remain separate concepts.
Peer review challenges the reported route.
Replication creates a new evidence route.
A mature research system needs both.
38. Code and Data Review Can Increase What Reviewers Are Able to Check
Computational research creates an opportunity conventional papers did not always provide.
If appropriate data and code are available, reviewers or dedicated reproducibility teams can sometimes test whether reported tables and figures can be regenerated.
This can expose undocumented transformations, missing dependencies, coding errors or discrepancies between prose and computation.
But access has legitimate limits. Human-participant data may require controlled environments, de-identification or legal restrictions. Proprietary or security-sensitive datasets may not be openly distributable.
The correct goal is not “everything must always be public”.
It is maximum appropriate inspectability without violating ethical, legal or safety constraints.
39. Reproducibility Review and Scientific Replication Are Different
If reviewers can regenerate the same result from the same data and code, that is evidence that the computational route is recoverable.
It does not show that new data will produce the same scientific conclusion.
The National Academies distinction used across the eduKateSG research corridor is useful here:
- reproducibility concerns consistent results from the same data and computational methods;
- replicability concerns consistent conclusions from new data addressing the same scientific question.
Peer review can support reproducibility.
It cannot substitute for replication.
40. Preprints Separate Public Communication From Journal Certification
A preprint allows researchers to share a manuscript publicly before formal journal peer review has finished.
This can accelerate communication, establish priority, invite wider criticism and make negative or preliminary findings visible.
But readers must preserve the state label.
A preprint may later be revised substantially. It may never be accepted. It may contain errors that reviewers would catch.
The useful mental model is not “preprint bad, peer-reviewed good”.
It is a versioned evidence state: what scrutiny has this document undergone so far?
41. Registered Reports Move Peer Review Upstream
Conventional peer review often arrives when the study is already finished.
At that point, a reviewer can identify a weak design but cannot travel back in time and repair data collection.
Registered Reports change the sequence.
The research question, methods and analysis plan are reviewed before the relevant results are known. If the protocol meets the journal’s standards, the journal can grant in-principle acceptance. The completed study then returns for a second-stage review focused on whether the approved plan was followed and whether the conclusions are justified.
In 2026, Nature announced an expansion of Registered Reports across the fields it publishes, extending the format beyond its earlier, narrower use. The underlying design principle is powerful: judge the importance of the question and strength of the method before knowing whether the result is exciting.
42. Registered Reports Attack Publication Bias at a Different Point in the System
Traditional publishing can create incentives to favour surprising, positive or statistically significant outcomes.
If a journal makes an in-principle publication commitment before the outcomes are known, the result itself has less opportunity to determine whether the study becomes visible.
That does not eliminate every source of bias.
But it changes the reward structure from:
Did you get an exciting result?
toward:
Did you ask a worthwhile question with a design capable of answering it?
43. Post-Publication Peer Review Extends the Scrutiny Window
Publication increases the number of eyes on a paper dramatically.
Researchers may try to reproduce analyses, compare the result with their own data, notice a mathematical error, identify an image anomaly, question an assumption or conduct a replication.
Formal comments, letters, corrections, online discussion and dedicated post-publication review platforms can all continue the scrutiny process.
The scientific record should therefore be understood as editable under evidence, not frozen at the moment of publication.
44. Correction Is Not an Embarrassing Epilogue—It Is Part of the System
Published papers can require errata, corrections, expressions of concern or retractions.
The appropriate response depends on the problem’s severity and whether the central conclusions remain reliable.
A corrected record is stronger than an uncorrectable one.
This is the same governing principle that appears throughout eduKateSG’s research architecture: the quality of a knowledge system is not measured by whether it ever makes errors. It is measured partly by whether errors can be found, traced, bounded and repaired.
45. Retraction Does Not Mean “Everything by These Authors Is False”
A retraction applies to a particular published record and the reasons attached to its unreliability or publication history.
It should not be converted casually into a claim about every piece of work associated with an author, institution or field.
At the same time, serious misconduct can justifiably trigger wider audits.
The disciplined approach is provenance-based: identify which claims depend on which evidence and which evidence has been withdrawn or corrected.
46. Peer Review Does Not Replace the Reader
A peer-reviewed article still has to be read critically.
Ask what population was studied, how variables were measured, what comparison supports the claim, how large the effect is, what uncertainty remains and whether the conclusion matches the evidence.
Peer review raises the prior confidence that some informed scrutiny occurred.
It does not remove the need to inspect the paper itself.
47. Journal Prestige Is Not a Substitute for Reading the Methods
Prestigious journals reject many papers and select for criteria beyond basic validity, including significance and breadth of interest.
They also make mistakes.
A famous journal title can be a useful metadata signal about editorial selectivity and visibility.
It cannot replace the causal chain inside the paper.
Science would be fragile if truth changed according to the masthead above the abstract.
48. Peer Review Can Become a Bottleneck
Modern science produces enormous numbers of manuscripts.
Reviewing them consumes expert time. Reviewers are frequently unpaid, and the same active researchers may receive more requests than they can responsibly accept.
Capacity constraints can delay publication, narrow reviewer pools and increase fatigue.
This is a systems problem rather than simply a problem of individual goodwill.
A sustainable peer-review system needs enough qualified reviewers, appropriate recognition, sensible editorial triage and methods that avoid wasting specialist attention on questions that can be resolved earlier.
49. AI Changes Reviewer Workflow but Not Reviewer Responsibility
AI tools can summarise text, check language, inspect code, search literature and assist with parts of scholarly workflow.
But unpublished manuscripts can contain confidential, proprietary or sensitive information. Journals therefore increasingly specify rules for whether and how reviewers may use AI tools.
The governing principle should remain clear:
A tool can assist the review. It does not inherit the reviewer’s accountability.
The human reviewer remains responsible for the judgement they submit and for following the journal’s confidentiality and disclosure rules.
50. Specialist Review Is Often Better Than Pretending One Reviewer Knows Everything
A complicated manuscript may contain a domain claim, a statistical model, a software pipeline, a qualitative component and a specialised instrument.
Expecting one person to assess every layer creates false completeness.
Some journals therefore add statistical reviewers, methods reviewers, image-integrity checks, data editors or specialist reviewers for particular techniques.
A recent randomised trial in biomedical journals, for example, studied the effect of adding librarians and information specialists as methodological reviewers for systematic-review search methods.
The broader lesson is architectural: distribute scrutiny according to expertise rather than pretending “peer reviewed” describes one uniform inspection depth.
51. Interdisciplinary Papers Need Reviewers Who Can See Across Boundaries
Interdisciplinary work creates a special risk.
A manuscript can be competent inside each discipline while failing at the interface between them.
A machine-learning model may be technically excellent but use a clinically meaningless target. An archaeological inference may use sophisticated genetics while oversimplifying historical context. An education study may apply a statistical model correctly while misunderstanding the classroom mechanism.
Strong interdisciplinary review therefore needs both depth and bridge expertise.
52. Review Quality Depends on the Question the Reviewer Thinks They Are Answering
Is the reviewer judging:
- whether the study is valid?
- whether it is novel?
- whether it matters to this journal’s readers?
- whether the methods are reproducible?
- whether the writing is clear?
- whether the paper deserves scarce journal space?
Those are different jobs.
Peer-review systems become more coherent when journals tell reviewers which jobs they are performing and editors distinguish scientific criticism from editorial selection.
53. The Hostile Test: Three Reviewers Miss the Same Hidden Error
Imagine a manuscript with a subtle coding error in a preprocessing pipeline.
The domain science looks plausible. The statistical model is standard. The figures look convincing. Three reviewers read the paper carefully and none reruns the code because the code is unavailable.
The article is published.
Six months later, another group tries to reproduce the analysis, discovers the error and finds that the central effect is much smaller.
Did peer review fail?
At one level, yes: the paper entered the record with an important error.
At another level, the episode reveals why peer review cannot be the only gate. Code availability, reproducibility checks, replication and post-publication correction provide different forms of coverage.
No single checkpoint should be asked to carry the whole epistemic load.
54. The Second Hostile Test: Reviewers Demand a Result They Prefer
Imagine a careful study that finds little evidence for a popular effect.
A reviewer repeatedly asks for alternative analyses until one produces a positive result.
That review has stopped functioning as quality control and started creating outcome pressure.
A strong editor should distinguish legitimate robustness requests from requests that effectively search for a preferred conclusion.
Registered Reports are one structural response to this failure mode because key methodological judgements occur before the outcome is known.
55. The Third Hostile Test: A Famous Author and an Unknown Author Submit Equivalent Work
If the famous author is automatically trusted and the unknown author automatically scrutinised, identity has entered the evidence pathway.
Double-anonymised review attempts to reduce that pathway, although perfect anonymisation can be difficult.
The broader lesson reaches beyond publishing:
Evaluate the provenance that matters to the claim; minimise provenance that merely triggers status heuristics.
56. Peer Review Is a Network, Not a Single Pair of Eyes
The visible review report is only one layer in a larger network:
- authors design and report;
- editors triage and select expertise;
- reviewers interrogate;
- authors respond;
- editors adjudicate;
- publishers preserve versions and corrections;
- readers inspect;
- other researchers reproduce, replicate and synthesize;
- institutions investigate misconduct where necessary.
Trust becomes more defensible when these roles remain distinct enough to challenge one another.
57. Peer Review and Publishing Are Neighbours, Not Synonyms
How Publishing Works owns the larger route by which manuscripts become durable, identifiable, distributed and discoverable records.
Peer review owns a narrower mechanism inside or around that route:
independent scrutiny → criticism → revision → editorial judgement.
A book can be published without journal-style peer review. A preprint can be publicly distributed before journal review. A registered report can undergo peer review before conventional results exist.
Keeping the concepts separate prevents keyword and conceptual collision.
58. Peer Review and Verification Are Also Different
Verification asks whether specified claims, requirements or properties have been checked against appropriate evidence.
Peer review may contain verification activities, but it is broader and less uniform. A reviewer may assess conceptual novelty, methodological suitability, interpretation, clarity and journal relevance without independently verifying every underlying datum.
Calling peer review “verification” without specifying the scope exaggerates what occurred.
59. Peer Review and Evidence Synthesis Are Different Too
A peer reviewer judges one manuscript in context.
A systematic review or meta-analysis asks what a body of studies collectively shows.
A single peer-reviewed paper can be outweighed by a larger, more rigorous evidence base. Conversely, a new paper can reveal that an older consensus depended on a hidden assumption.
The unit of scientific confidence should therefore not be “peer-reviewed paper” alone.
It should be the evolving evidence landscape.
60. Primary School: Peer Review Begins as “Let Someone Else Check Your Reasoning”
Young students do not need journal editorial systems to learn the underlying habit.
A classmate can ask:
- What are you trying to show?
- Where is your evidence?
- Did you measure it fairly?
- Could something else explain the result?
- Can I follow your steps?
- What would make you change your answer?
The goal is not to make children imitate academic referees.
It is to teach that good work can become better when another mind tries to find the weak point.
61. Secondary School: Review the Claim, Not the Person
Secondary students can practise separating criticism of an argument from criticism of its author.
“This evidence does not support your conclusion yet” is scientifically useful.
“You are bad at science” is not.
Students can also learn that disagreement is not automatically disrespect. A serious challenge can be a form of intellectual cooperation when it makes the final work stronger.
62. JC and University: Learn to Write a Review That Another Person Can Act On
At higher levels, peer feedback should become specific and prioritised.
Instead of:
The methodology is weak.
write:
The causal conclusion depends on group comparability, but allocation was observational and the manuscript does not address baseline differences that could explain the outcome.
The second comment identifies the claim, the dependency, the missing evidence and the reason it matters.
That is what useful criticism looks like.
63. How to Read a Peer-Reviewed Paper
- What exact claim is being made?
- What study design supports it?
- Who or what was sampled?
- How were key variables measured?
- What comparison or counterfactual structure exists?
- How large is the effect or relationship?
- How uncertain is the estimate?
- Which analyses were pre-specified?
- What alternative explanations remain?
- What limitations do the authors acknowledge?
- Does the abstract overstate the Results section?
- Does the news headline overstate the abstract?
- Has the result been reproduced or replicated?
- How does it fit the wider literature?
- Have corrections or post-publication critiques appeared?
64. How to Write a Useful Peer Review
- State the paper’s main claim in your own words.
- Identify its strongest contribution.
- Identify the few issues that genuinely determine validity or interpretability.
- Separate major from minor concerns.
- Explain the reasoning behind each concern.
- Point to the relevant section, figure, method or assumption.
- Do not demand your preferred study if the authors’ design can answer their question.
- Do not ask for citations simply to increase citations to yourself or your network.
- Disclose conflicts.
- Respect confidentiality.
- Keep criticism directed at the work, not the authors.
- Distinguish essential repairs from interesting future work.
- Tell the editor what remains uncertain.
65. How an Editor Should Read Conflicting Reviews
- Which concern threatens the central claim?
- Which reviewer has the most relevant expertise for that concern?
- Are reviewers evaluating the same claim?
- Is disagreement substantive or terminological?
- Can clarification resolve it?
- Does the issue require additional specialist review?
- Is a requested analysis scientifically necessary or merely preferred?
- Would narrowing the claim solve the problem?
- What unresolved uncertainty remains after revision?
- Does publication still serve the journal’s readers and scientific community?
66. Where Peer Review Fits in the eduKateSG “How Works” Landscape
- How Scientific Research Works — the larger route from question to correctable knowledge.
- How Hypotheses Work — how candidate explanations become evidence-sensitive claims.
- How Sampling Works — how part of the world becomes eligible to support a larger inference.
- How Measurement Works — how observations become bounded comparable results.
- How Evidence Works — how results support or weaken claims.
- How Uncertainty Works — how limits remain visible.
- How Verification Works — independent checking against specified claims or requirements.
- How Publishing Works — the durable record, identifiers, distribution, discovery and correction infrastructure.
- How Citation Works — evidence lineage and recoverability.
Peer review sits at the junction where research meets informed external criticism before editorial commitment. It does not own the entire research process and it does not own the entire publishing process. Its specific job is to make a manuscript encounter competent resistance.
67. What This Article Does Not Claim
- Peer review does not prove that a paper is true.
- Peer review does not guarantee that fraud, coding errors or image manipulation will be detected.
- Reviewers do not necessarily inspect raw data or rerun analyses unless the review process specifically includes that work.
- Reviewer recommendations are advice to editors, not votes that mechanically determine publication.
- Reviewer disagreement does not automatically mean the system failed.
- Journal rejection does not automatically mean the central scientific claim is false.
- Journal acceptance does not automatically mean every reviewer agrees.
- Anonymity can reduce some information pathways without eliminating all bias.
- Open review increases transparency without guaranteeing better judgement in every case.
- Peer review cannot replace reproducibility checks, independent replication, evidence synthesis or post-publication correction.
68. A Compact Peer-Review Audit
- What type of peer review was used?
- Was there editorial screening before external review?
- What expertise did the manuscript require?
- How were reviewers selected?
- Were relevant conflicts disclosed?
- What was the scope of reviewer access—manuscript only, code, data, supplementary files?
- Did reviewers examine question, design, sampling, measurement and analysis?
- Were causal claims matched to causal evidence?
- Were effect sizes and uncertainty visible?
- Were exploratory and confirmatory analyses distinguished?
- Were major and minor concerns separated?
- Did authors respond transparently?
- Was specialist re-review needed?
- How did the editor resolve reviewer disagreement?
- Were review reports or decisions made transparent?
- Has the paper later been corrected, criticised, reproduced or replicated?
- Does the paper still fit the wider evidence landscape?
69. Frequently Asked Questions
What is peer review?
Peer review is a structured process in which editors obtain critique from independent experts with relevant knowledge so that a manuscript’s methods, reasoning, reporting and claims can be assessed before an editorial decision.
Does peer review mean a study is correct?
No. It means the manuscript passed a particular editorial and review process. Important errors can remain, and later evidence can overturn published conclusions.
How many reviewers examine a paper?
It varies by journal and manuscript. Nature states that manuscripts sent for formal review are typically assessed by two or three reviewers, sometimes more when specialised advice is required.
Who makes the final decision?
The journal editor does. Reviewer reports provide expert advice, but editorial decisions are not necessarily determined by majority recommendation.
What is double-anonymised peer review?
It is a model in which author identities are hidden from reviewers and reviewer identities are hidden from authors as far as the process allows, with the aim of reducing some status and identity cues.
What is transparent peer review?
It generally makes parts of the review history—such as reviewer reports, author responses or decision letters—available with the published article. Exact implementations differ by journal.
What is a Registered Report?
It is an article format in which the research question and study plan undergo peer review before the relevant results are known. Strong protocols can receive in-principle acceptance, after which the completed study is reviewed again against the approved plan.
Can reviewers detect fraud?
They sometimes identify suspicious features, but conventional peer review should not be treated as a reliable fraud-detection system. Investigation, data access, forensic checks and later replication can be necessary.
Can authors disagree with reviewers?
Yes. Authors can explain why a requested change is unsupported, inappropriate or outside scope, provided the response is reasoned and evidence-based. The editor decides whether the response resolves the concern.
70. Authoritative Research Corridor
- Nature — Peer Review: process, reviewer selection, editorial decisions, anonymity and transparency
- Nature Communications — Registered Reports
- Nature — Expanding Registered Reports across fields in 2026
- Nature Human Behaviour — Promoting reproducibility with Registered Reports
- Peer review: concepts, variants and controversies
- Reviewer recommendations and editorial outcomes: agreement and disagreement across review rounds
- Methodological specialist peer review: randomised trial involving librarians and information specialists
Final Thought: Peer Review Is Organised Resistance
A manuscript begins as an argument its authors already understand.
That is exactly why another mind is useful.
The reviewer has not lived inside the project for three years. They were not there when the instrument failed, when the analysis finally worked, when the theory started to feel obvious.
They arrive cold.
And cold eyes see different things.
Where did this claim come from?
Why this sample?
Why this measure?
What else could explain the result?
Why is the conclusion stronger than the evidence?
What would I need to see before I believe this?
Those questions do not make the reviewer the owner of truth.
They make the paper less protected from challenge.
Then the editor weighs the arguments.
The authors answer.
The manuscript changes.
And even after publication, reality keeps its right of reply.
Peer review is useful not because experts cannot be wrong, but because knowledge becomes stronger when important claims are forced to survive informed resistance.