Why English? | Reading ROB-MEN Network Meta-Analysis Bias Through Missing Evidence, Comparisons and Network Structure

People searching for ROB-MEN, publication bias in network meta-analysis or missing evidence in treatment networks need language that works at more than one scale. English matters because a missing outcome within a known trial is not the same as an unpublished trial, a direct comparison is not the same as a network estimate, and one absent study can influence several treatment contrasts through indirect evidence.

The official ROB-MEN web application, checked on 5 November 2026, describes ROB-MEN as a tool for risk of bias due to missing evidence in network meta-analysis and part of CINeMA. It requires long-format arm-level data and asks users to retain all studies identified in the review, including studies that did not report the outcome of interest. The official page cites the 2021 methodology article and 2023 guidance for the web application.

The University of Bern methods group explains that ROB-MEN examines unavailable results, unpublished studies, direct pairwise comparisons, study contributions, small-study effects and unobserved comparisons, producing low-risk, some-concerns or high-risk judgements for network estimates. This article owns that network-specific intent. ROB-ME owns missing evidence in a conventional synthesis result; neither tool replaces within-study bias appraisal, heterogeneity, incoherence or certainty assessment.

• Define the treatment network, outcome, time point and estimate.
• Keep known studies visible even when outcomes are unavailable.
• Assess suspected missing evidence in direct comparisons.
• Trace how those comparisons contribute to each network estimate.
• Report the judgement, likely direction and uncertainty without accusing or promising.

---

Define the Network Meta-Analysis Estimate

Tie the assessment to one outcome, time point, treatment contrast and network model.

Practical reading move for Define the Network Meta-Analysis Estimate

Map every treatment node and direct comparison before judging missing evidence. The same missing study can influence several network estimates through different contribution paths. Under Define the Network Meta-Analysis Estimate, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Define the Network Meta-Analysis Estimate

Suppose treatments A, B and C form a triangle, but small unfavourable A-versus-C studies appear unpublished. An A-versus-B estimate may still be affected if the network model borrows strongly from the A-C and B-C paths. Apply the example to define the network meta-analysis estimate by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

ROB-MEN addresses bias due to missing evidence in network meta-analysis; ROB-ME has a related but separate owner for conventional meta-analytic synthesis results. For define the network meta-analysis estimate, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Distinguish ROB-MEN From ROB-ME

ROB-MEN focuses on network meta-analysis estimates, while ROB-ME owns conventional synthesis-result missing evidence.

Practical reading move for Distinguish ROB-MEN From ROB-ME

Separate unpublished studies from selectively unavailable outcomes within known studies. Then ask whether the reason for absence could depend on the direction, magnitude or statistical significance of results. Under Distinguish ROB-MEN From ROB-ME, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Distinguish ROB-MEN From ROB-ME

A trial registry lists pain and function, yet the article reports only function. The pain result is a known missing outcome; its influence depends on eligibility, comparison and contribution to the network estimate. Apply the example to distinguish rob-men from rob-me by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

A judgement concerns a particular network estimate and outcome, not the honesty of every investigator or the truth of an entire field. For distinguish rob-men from rob-me, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Map Treatments as Nodes and Studies as Edges

The geometry shows where evidence exists, which comparisons are direct and which estimates borrow indirectly.

Practical reading move for Map Treatments as Nodes and Studies as Edges

Create a comparison-level table for suspected missing evidence and a network-level table for how direct evidence contributes to each estimate. This prevents one broad publication-bias label from covering unlike problems. Under Map Treatments as Nodes and Studies as Edges, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Map Treatments as Nodes and Studies as Edges

An empty comparison in the network is not automatically evidence of suppression. The team asks whether trials were plausible, conducted, registered or unavailable because findings were less attractive. Apply the example to map treatments as nodes and studies as edges by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

The ROB-MEN web application supports the process, but uploaded data, comparison logic and interpretation still require trained human review. For map treatments as nodes and studies as edges, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

List All Identified Studies Including Missing Outcomes

Known studies remain in the data structure even when the outcome of interest was not reported.

Practical reading move for List All Identified Studies Including Missing Outcomes

Use the study protocol, registry, conference record and all reports to identify outcomes that were planned but not available. Do not convert a blank cell into an assumed zero effect. Under List All Identified Studies Including Missing Outcomes, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for List All Identified Studies Including Missing Outcomes

If small studies consistently favour a commercially sponsored treatment, record the pattern and alternative explanations. The judgement should not accuse investigators without evidence about the missingness process. Apply the example to list all identified studies including missing outcomes by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

Missing evidence is not the same as heterogeneity, incoherence or within-study risk of bias, although all may affect confidence in a network estimate. For list all identified studies including missing outcomes, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Separate Missing Results From Missing Studies

Selective outcome reporting and non-publication are related but different missing-evidence processes.

Practical reading move for Separate Missing Results From Missing Studies

Inspect small-study effects as one clue among several. Network geometry, heterogeneity, design differences and chance can also produce patterns that require explanation. Under Separate Missing Results From Missing Studies, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Separate Missing Results From Missing Studies

A large three-arm trial can contribute to several contrasts at once. Missing one eligible outcome from that trial may matter more than several tiny two-arm studies with minimal network contribution. Apply the example to separate missing results from missing studies by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

A symmetrical pattern does not prove absence of missing evidence, and an asymmetrical pattern does not prove publication bias. For separate missing results from missing studies, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Ask Whether Missingness Depends on Results

Consider whether P value, magnitude or direction plausibly influenced availability.

Practical reading move for Ask Whether Missingness Depends on Results

For each network estimate, describe which direct comparisons carry the most weight and which of those have suspected missing evidence. Contribution matters more than simply counting concerns. Under Ask Whether Missingness Depends on Results, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Ask Whether Missingness Depends on Results

The web application requires arm-level long-format data and includes studies that did not report the outcome. This design keeps unavailable results visible instead of silently dropping their studies. Apply the example to ask whether missingness depends on results by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

The framework does not rescue a poorly specified network question; treatments, outcomes, time points and eligible designs must be defined first. For ask whether missingness depends on results, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Assess Each Direct Pairwise Comparison

Suspected missing evidence is first located in the direct comparisons that build the network.

Practical reading move for Assess Each Direct Pairwise Comparison

Write a free-text rationale before accepting the proposed low-risk, some-concerns or high-risk category. The rationale should name the missing-evidence mechanism and likely direction of distortion. Under Assess Each Direct Pairwise Comparison, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Assess Each Direct Pairwise Comparison

Students can draw a subway-style treatment network and shade links with suspected missing evidence, then trace which journeys to an estimate pass through those links. Apply the example to assess each direct pairwise comparison by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

Network estimates can be useful while uncertain. Communicate the judgement as calibrated caution, not a promise that treatment rankings will or will not hold. For assess each direct pairwise comparison, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Examine Unavailable Results in Known Studies

Protocols, registries and companion reports can reveal outcomes that were measured but not published.

Practical reading move for Examine Unavailable Results in Known Studies

Map every treatment node and direct comparison before judging missing evidence. The same missing study can influence several network estimates through different contribution paths. Under Examine Unavailable Results in Known Studies, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Examine Unavailable Results in Known Studies

Suppose treatments A, B and C form a triangle, but small unfavourable A-versus-C studies appear unpublished. An A-versus-B estimate may still be affected if the network model borrows strongly from the A-C and B-C paths. Apply the example to examine unavailable results in known studies by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

ROB-MEN addresses bias due to missing evidence in network meta-analysis; ROB-ME has a related but separate owner for conventional meta-analytic synthesis results. For examine unavailable results in known studies, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Examine Unpublished Studies

Registries, conference records and research context may identify completed work absent from journals.

Practical reading move for Examine Unpublished Studies

Separate unpublished studies from selectively unavailable outcomes within known studies. Then ask whether the reason for absence could depend on the direction, magnitude or statistical significance of results. Under Examine Unpublished Studies, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Examine Unpublished Studies

A trial registry lists pain and function, yet the article reports only function. The pain result is a known missing outcome; its influence depends on eligibility, comparison and contribution to the network estimate. Apply the example to examine unpublished studies by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

A judgement concerns a particular network estimate and outcome, not the honesty of every investigator or the truth of an entire field. For examine unpublished studies, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Consider Small-Study Effects

Patterns related to study size may inform concern while retaining alternative explanations.

Practical reading move for Consider Small-Study Effects

Create a comparison-level table for suspected missing evidence and a network-level table for how direct evidence contributes to each estimate. This prevents one broad publication-bias label from covering unlike problems. Under Consider Small-Study Effects, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Consider Small-Study Effects

An empty comparison in the network is not automatically evidence of suppression. The team asks whether trials were plausible, conducted, registered or unavailable because findings were less attractive. Apply the example to consider small-study effects by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

The ROB-MEN web application supports the process, but uploaded data, comparison logic and interpretation still require trained human review. For consider small-study effects, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Identify Unobserved Comparisons

A missing edge can matter when trials were plausible yet evidence is unavailable for result-related reasons.

Practical reading move for Identify Unobserved Comparisons

Use the study protocol, registry, conference record and all reports to identify outcomes that were planned but not available. Do not convert a blank cell into an assumed zero effect. Under Identify Unobserved Comparisons, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Identify Unobserved Comparisons

If small studies consistently favour a commercially sponsored treatment, record the pattern and alternative explanations. The judgement should not accuse investigators without evidence about the missingness process. Apply the example to identify unobserved comparisons by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

Missing evidence is not the same as heterogeneity, incoherence or within-study risk of bias, although all may affect confidence in a network estimate. For identify unobserved comparisons, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Use Contribution to Move Across the Network

Direct evidence contributes unequally to different network estimates and should be weighted conceptually.

Practical reading move for Use Contribution to Move Across the Network

Inspect small-study effects as one clue among several. Network geometry, heterogeneity, design differences and chance can also produce patterns that require explanation. Under Use Contribution to Move Across the Network, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Use Contribution to Move Across the Network

A large three-arm trial can contribute to several contrasts at once. Missing one eligible outcome from that trial may matter more than several tiny two-arm studies with minimal network contribution. Apply the example to use contribution to move across the network by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

A symmetrical pattern does not prove absence of missing evidence, and an asymmetrical pattern does not prove publication bias. For use contribution to move across the network, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Judge the Direction of Potential Bias

State whether missing evidence could favour one treatment or remain directionally uncertain.

Practical reading move for Judge the Direction of Potential Bias

For each network estimate, describe which direct comparisons carry the most weight and which of those have suspected missing evidence. Contribution matters more than simply counting concerns. Under Judge the Direction of Potential Bias, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Judge the Direction of Potential Bias

The web application requires arm-level long-format data and includes studies that did not report the outcome. This design keeps unavailable results visible instead of silently dropping their studies. Apply the example to judge the direction of potential bias by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

The framework does not rescue a poorly specified network question; treatments, outcomes, time points and eligible designs must be defined first. For judge the direction of potential bias, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Keep Outcome and Time Point Stable

Availability can differ across outcomes and follow-up times within the same network.

Practical reading move for Keep Outcome and Time Point Stable

Write a free-text rationale before accepting the proposed low-risk, some-concerns or high-risk category. The rationale should name the missing-evidence mechanism and likely direction of distortion. Under Keep Outcome and Time Point Stable, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Keep Outcome and Time Point Stable

Students can draw a subway-style treatment network and shade links with suspected missing evidence, then trace which journeys to an estimate pass through those links. Apply the example to keep outcome and time point stable by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

Network estimates can be useful while uncertain. Communicate the judgement as calibrated caution, not a promise that treatment rankings will or will not hold. For keep outcome and time point stable, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Prepare Long-Format Arm-Level Data

The official web application expects each treatment arm as a row and retains studies with unavailable outcomes.

Practical reading move for Prepare Long-Format Arm-Level Data

Map every treatment node and direct comparison before judging missing evidence. The same missing study can influence several network estimates through different contribution paths. Under Prepare Long-Format Arm-Level Data, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Prepare Long-Format Arm-Level Data

Suppose treatments A, B and C form a triangle, but small unfavourable A-versus-C studies appear unpublished. An A-versus-B estimate may still be affected if the network model borrows strongly from the A-C and B-C paths. Apply the example to prepare long-format arm-level data by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

ROB-MEN addresses bias due to missing evidence in network meta-analysis; ROB-ME has a related but separate owner for conventional meta-analytic synthesis results. For prepare long-format arm-level data, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Check Treatment Codes and Study Identity

A clean input file prevents duplicated studies, merged treatments or broken multi-arm structures.

Practical reading move for Check Treatment Codes and Study Identity

Separate unpublished studies from selectively unavailable outcomes within known studies. Then ask whether the reason for absence could depend on the direction, magnitude or statistical significance of results. Under Check Treatment Codes and Study Identity, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Check Treatment Codes and Study Identity

A trial registry lists pain and function, yet the article reports only function. The pain result is a known missing outcome; its influence depends on eligibility, comparison and contribution to the network estimate. Apply the example to check treatment codes and study identity by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

A judgement concerns a particular network estimate and outcome, not the honesty of every investigator or the truth of an entire field. For check treatment codes and study identity, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Interpret Low Risk Carefully

Low risk means little evidence of result-dependent missingness for that estimate, not proof of complete knowledge.

Practical reading move for Interpret Low Risk Carefully

Create a comparison-level table for suspected missing evidence and a network-level table for how direct evidence contributes to each estimate. This prevents one broad publication-bias label from covering unlike problems. Under Interpret Low Risk Carefully, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Interpret Low Risk Carefully

An empty comparison in the network is not automatically evidence of suppression. The team asks whether trials were plausible, conducted, registered or unavailable because findings were less attractive. Apply the example to interpret low risk carefully by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

The ROB-MEN web application supports the process, but uploaded data, comparison logic and interpretation still require trained human review. For interpret low risk carefully, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Interpret Some Concerns

Intermediate concern records a credible issue whose likely impact is limited or uncertain.

Practical reading move for Interpret Some Concerns

Use the study protocol, registry, conference record and all reports to identify outcomes that were planned but not available. Do not convert a blank cell into an assumed zero effect. Under Interpret Some Concerns, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Interpret Some Concerns

If small studies consistently favour a commercially sponsored treatment, record the pattern and alternative explanations. The judgement should not accuse investigators without evidence about the missingness process. Apply the example to interpret some concerns by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

Missing evidence is not the same as heterogeneity, incoherence or within-study risk of bias, although all may affect confidence in a network estimate. For interpret some concerns, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Interpret High Risk

High risk requires a strong, estimate-specific rationale about missing evidence and potential distortion.

Practical reading move for Interpret High Risk

Inspect small-study effects as one clue among several. Network geometry, heterogeneity, design differences and chance can also produce patterns that require explanation. Under Interpret High Risk, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Interpret High Risk

A large three-arm trial can contribute to several contrasts at once. Missing one eligible outcome from that trial may matter more than several tiny two-arm studies with minimal network contribution. Apply the example to interpret high risk by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

A symmetrical pattern does not prove absence of missing evidence, and an asymmetrical pattern does not prove publication bias. For interpret high risk, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Write the Free-Text Reason

A category becomes useful when the pathway from missing evidence to the network estimate is explicit.

Practical reading move for Write the Free-Text Reason

For each network estimate, describe which direct comparisons carry the most weight and which of those have suspected missing evidence. Contribution matters more than simply counting concerns. Under Write the Free-Text Reason, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Write the Free-Text Reason

The web application requires arm-level long-format data and includes studies that did not report the outcome. This design keeps unavailable results visible instead of silently dropping their studies. Apply the example to write the free-text reason by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

The framework does not rescue a poorly specified network question; treatments, outcomes, time points and eligible designs must be defined first. For write the free-text reason, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Use the Tool Within CINeMA

ROB-MEN forms part of the wider Confidence in Network Meta-Analysis framework rather than replacing all domains.

Practical reading move for Use the Tool Within CINeMA

Write a free-text rationale before accepting the proposed low-risk, some-concerns or high-risk category. The rationale should name the missing-evidence mechanism and likely direction of distortion. Under Use the Tool Within CINeMA, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Use the Tool Within CINeMA

Students can draw a subway-style treatment network and shade links with suspected missing evidence, then trace which journeys to an estimate pass through those links. Apply the example to use the tool within cinema by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

Network estimates can be useful while uncertain. Communicate the judgement as calibrated caution, not a promise that treatment rankings will or will not hold. For use the tool within cinema, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Did You Know? One Study Can Affect Many Contrasts

A multi-arm or highly connected study can contribute information across several network estimates.

Practical reading move for Did You Know? One Study Can Affect Many Contrasts

Map every treatment node and direct comparison before judging missing evidence. The same missing study can influence several network estimates through different contribution paths. Under Did You Know? One Study Can Affect Many Contrasts, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Did You Know? One Study Can Affect Many Contrasts

Suppose treatments A, B and C form a triangle, but small unfavourable A-versus-C studies appear unpublished. An A-versus-B estimate may still be affected if the network model borrows strongly from the A-C and B-C paths. Apply the example to did you know? one study can affect many contrasts by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

ROB-MEN addresses bias due to missing evidence in network meta-analysis; ROB-ME has a related but separate owner for conventional meta-analytic synthesis results. For did you know? one study can affect many contrasts, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Did You Know? The App Is Open Source

The official site states that the web application is distributed under the GNU AGPL v3 licence.

Practical reading move for Did You Know? The App Is Open Source

Separate unpublished studies from selectively unavailable outcomes within known studies. Then ask whether the reason for absence could depend on the direction, magnitude or statistical significance of results. Under Did You Know? The App Is Open Source, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Did You Know? The App Is Open Source

A trial registry lists pain and function, yet the article reports only function. The pain result is a known missing outcome; its influence depends on eligibility, comparison and contribution to the network estimate. Apply the example to did you know? the app is open source by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

A judgement concerns a particular network estimate and outcome, not the honesty of every investigator or the truth of an entire field. For did you know? the app is open source, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Avoid Counting Studies as the Whole Judgement

Size, precision, direction and contribution matter more than a simple number of suspected omissions.

Practical reading move for Avoid Counting Studies as the Whole Judgement

Create a comparison-level table for suspected missing evidence and a network-level table for how direct evidence contributes to each estimate. This prevents one broad publication-bias label from covering unlike problems. Under Avoid Counting Studies as the Whole Judgement, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Avoid Counting Studies as the Whole Judgement

An empty comparison in the network is not automatically evidence of suppression. The team asks whether trials were plausible, conducted, registered or unavailable because findings were less attractive. Apply the example to avoid counting studies as the whole judgement by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

The ROB-MEN web application supports the process, but uploaded data, comparison logic and interpretation still require trained human review. For avoid counting studies as the whole judgement, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Explain the Network to Students

A route map helps learners see how indirect evidence links treatments that were never compared head to head.

Practical reading move for Explain the Network to Students

Use the study protocol, registry, conference record and all reports to identify outcomes that were planned but not available. Do not convert a blank cell into an assumed zero effect. Under Explain the Network to Students, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Explain the Network to Students

If small studies consistently favour a commercially sponsored treatment, record the pattern and alternative explanations. The judgement should not accuse investigators without evidence about the missingness process. Apply the example to explain the network to students by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

Missing evidence is not the same as heterogeneity, incoherence or within-study risk of bias, although all may affect confidence in a network estimate. For explain the network to students, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Help Families Read Treatment Rankings

A rank is conditional on included evidence, assumptions and uncertainty and should not become an individual prescription.

Practical reading move for Help Families Read Treatment Rankings

Inspect small-study effects as one clue among several. Network geometry, heterogeneity, design differences and chance can also produce patterns that require explanation. Under Help Families Read Treatment Rankings, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Help Families Read Treatment Rankings

A large three-arm trial can contribute to several contrasts at once. Missing one eligible outcome from that trial may matter more than several tiny two-arm studies with minimal network contribution. Apply the example to help families read treatment rankings by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

A symmetrical pattern does not prove absence of missing evidence, and an asymmetrical pattern does not prove publication bias. For help families read treatment rankings, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Connect Missing Evidence to Sensitivity Work

Alternative assumptions can show whether treatment effects or rankings are fragile.

Practical reading move for Connect Missing Evidence to Sensitivity Work

For each network estimate, describe which direct comparisons carry the most weight and which of those have suspected missing evidence. Contribution matters more than simply counting concerns. Under Connect Missing Evidence to Sensitivity Work, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Connect Missing Evidence to Sensitivity Work

The web application requires arm-level long-format data and includes studies that did not report the outcome. This design keeps unavailable results visible instead of silently dropping their studies. Apply the example to connect missing evidence to sensitivity work by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

The framework does not rescue a poorly specified network question; treatments, outcomes, time points and eligible designs must be defined first. For connect missing evidence to sensitivity work, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Write a Responsible Network Conclusion

Name the affected estimate, missing-evidence concern, likely direction and remaining uncertainty.

Practical reading move for Write a Responsible Network Conclusion

Write a free-text rationale before accepting the proposed low-risk, some-concerns or high-risk category. The rationale should name the missing-evidence mechanism and likely direction of distortion. Under Write a Responsible Network Conclusion, write the operative noun, the comparison or synthesis it belongs to, the relevant source location and the exact reason the information changes interpretation.

Worked example for Write a Responsible Network Conclusion

Students can draw a subway-style treatment network and shade links with suspected missing evidence, then trace which journeys to an estimate pass through those links. Apply the example to write a responsible network conclusion by separating what the report states, what the reader can reasonably infer and what remains unknown.

Boundary, next step and transfer

Network estimates can be useful while uncertain. Communicate the judgement as calibrated caution, not a promise that treatment rankings will or will not hold. For write a responsible network conclusion, end with a bounded sentence: what the evidence supports, which limitation matters most, and what document, analysis or local question would reduce uncertainty. Students can practise the same move in projects; parents can use it when reading evidence claims; researchers and professionals can use it to make handovers and decisions auditable.

---

Frequently Asked Questions

What does ROB-MEN stand for?

Risk Of Bias due to Missing Evidence in Network meta-analysis.

What problem does it address?

Evidence missing because studies were unpublished or outcomes were selectively unavailable.

Is ROB-MEN part of CINeMA?

Yes.

Why include studies that did not report the outcome?

Their absence is part of the evidence needed to assess selective non-reporting.

What are the judgement categories?

Low risk, some concerns and high risk.

How is ROB-MEN different from ROB-ME?

ROB-MEN is designed for network meta-analysis estimates; ROB-ME addresses missing evidence in synthesis results more generally.

Does a missing network edge prove publication bias?

No. The reason for absence and plausibility of conducted but unavailable studies must be assessed.

Can one trial affect several estimates?

Yes, especially in multi-arm or highly connected networks.

Does a treatment ranking give an individual prescription?

No. It is conditional on the network, model, evidence and clinical context.

How can students understand contribution?

Draw the network and trace which direct comparisons supply information to a chosen treatment contrast.

---

Useful Next Reading

• Use the official ROB-MEN web application and documentation.
• Read the University of Bern ROB-MEN methods overview.
• Read conventional synthesis missing-evidence judgements with ROB-ME.
• Understand effect sizes, weighting, heterogeneity and missing evidence.
• Build a transparent source-to-synthesis writing trail.

Closing thought: ROB-MEN turns a vague concern about publication bias into an estimate-specific account of what may be missing, where it enters the network and how it could change interpretation.