Why English? | Using MMAT 2018 to Appraise Qualitative, Quantitative and Mixed-Methods Studies Without Hiding the Evidence

MMAT 2018 mixed-methods appraisal helps reviewers work with evidence that does not all use the same design. People searching for the Mixed Methods Appraisal Tool, MMAT checklist or mixed-methods quality assessment often need to appraise interviews, trials, non-randomised studies, surveys and genuinely integrated mixed-methods studies within one systematic mixed-studies review.

The official MMAT version 2018 user guide, checked on 5 November 2026, provides two screening questions and five criteria for each of five study categories: qualitative research, randomised controlled trials, quantitative non-randomised studies, quantitative descriptive studies and mixed-methods studies. The guide advises at least two independent reviewers, discourages an overall numerical score and recommends criterion-level reporting.

English is what keeps the design logic visible. A survey is not a trial. Interviews do not become mixed methods merely because a paper includes percentages. A mixed-methods study must integrate qualitative and quantitative components for a reason. Precise language lets reviewers say what each strand contributes, where the strands meet, why they diverge and which methodological limitation matters to a particular conclusion.

Use this guide beside the eduKate explanations of SRQR qualitative reporting, CONSORT 2025 trial reporting and STROBE observational reporting. Reporting guidance and critical appraisal are complementary, not interchangeable.

---

Begin With the Two Screening Questions

Confirm that the research question is clear and the collected data can address it before deeper appraisal.

What the MMAT criterion asks

Start with the two screening questions: whether research questions are clear and whether collected data can address them. For begin with the two screening questions, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

A survey and interview study asks one integrated question, but the report presents two parallel result sections with no explanation of how findings meet; integration is the issue. Read that example through begin with the two screening questions by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

The official MMAT version 2018 guide is designed for systematic mixed-studies reviews containing qualitative, quantitative and mixed-methods primary research. The boundary is decisive for begin with the two screening questions. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Classify the Study Design From Methods, Not Labels

Use what researchers actually did rather than accepting a title or abstract label uncritically.

What the MMAT criterion asks

Choose among qualitative, randomised quantitative, non-randomised quantitative, quantitative descriptive and mixed-methods categories. For classify the study design from methods, not labels, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

A qualitative study uses rich interviews but its stated grounded-theory aim does not match a purely descriptive analysis; the appraisal traces that coherence gap. Read that example through classify the study design from methods, not labels by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

The guide says MMAT is not for non-empirical papers such as reviews or theoretical papers and is not suited to some designs including economic and diagnostic-accuracy studies. The boundary is decisive for classify the study design from methods, not labels. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Know What MMAT Cannot Appraise

Non-empirical papers, reviews, theoretical work and certain specialised designs need other approaches.

What the MMAT criterion asks

For qualitative studies, connect approach, data collection, derivation of findings, substantiation and coherence. For know what mmat cannot appraise, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

A randomised classroom trial describes allocation but outcome assessors are teachers who know group assignment; the relevant criterion remains visible without condemning every aspect. Read that example through know what mmat cannot appraise by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

The developers advise at least two reviewers to appraise independently and recommend relevant training or expertise. The boundary is decisive for know what mmat cannot appraise. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Appraise With at Least Two Independent Reviewers

Plan training, independent notes and a fair consensus process before the final table is produced.

What the MMAT criterion asks

For randomised trials, examine randomisation, baseline comparability, outcome completeness, assessor blinding and adherence. For appraise with at least two independent reviewers, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

A non-randomised programme comparison measures baseline achievement and attendance but not prior support, leaving an important confounding route. Read that example through appraise with at least two independent reviewers by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

The guide discourages an overall score because it hides which methodological features are problematic and weights criteria mechanically. The boundary is decisive for appraise with at least two independent reviewers. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 1: Qualitative Approach and Question

Judge whether the qualitative approach fits the question and the kind of understanding sought.

What the MMAT criterion asks

For non-randomised studies, consider representativeness, measurement, complete outcomes, confounding and whether exposure or intervention occurred as intended. For category 1: qualitative approach and question, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

An online survey samples through one voluntary social-media group; the reader distinguishes a useful descriptive snapshot from representation of all parents. Read that example through category 1: qualitative approach and question by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

A Can’t tell response signals insufficient information, not a midpoint between Yes and No. The boundary is decisive for category 1: qualitative approach and question. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 1: Data Collection and Derivation

Trace whether data collection is adequate and findings are genuinely derived from the data.

What the MMAT criterion asks

For descriptive studies, inspect sampling relevance, representativeness, measurement, nonresponse and analysis. For category 1: data collection and derivation, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

Interview findings explain why average test scores improved for some learners but not others; the mixed-methods interpretation shows exactly how strands connect. Read that example through category 1: data collection and derivation by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

Mixed methods means purposeful integration of qualitative and quantitative components, not merely placing two methods in one paper. The boundary is decisive for category 1: data collection and derivation. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 1: Substantiation and Coherence

Check whether interpretations are supported and data sources, collection, analysis and interpretation cohere.

What the MMAT criterion asks

For mixed-methods studies, appraise each component and then the rationale, integration, interpretation, divergences and adherence to each tradition. For category 1: substantiation and coherence, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

Quantitative and qualitative findings conflict; the report examines timing and participant differences instead of choosing the more convenient strand. Read that example through category 1: substantiation and coherence by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

Methodological quality is separate from topic importance, ethical value and local applicability. The boundary is decisive for category 1: substantiation and coherence. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 2: Randomisation and Baseline Comparability

Examine allocation methods and whether groups begin sufficiently comparable for the intended contrast.

What the MMAT criterion asks

Use Can’t tell when reporting is insufficient, and document what information would be needed to answer Yes or No. For category 2: randomisation and baseline comparability, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

Two reviewers record criterion-level notes and later compare patterns in sensitivity analysis rather than converting five answers into a percentage score. Read that example through category 2: randomisation and baseline comparability by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

Automatic exclusion of lower-quality studies is usually discouraged; criterion-level reporting and sensitivity analysis can preserve more information. The boundary is decisive for category 2: randomisation and baseline comparability. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 2: Outcome Completeness and Blinding

Read missing outcomes, assessor knowledge and their plausible influence rather than ticking labels.

What the MMAT criterion asks

Start with the two screening questions: whether research questions are clear and whether collected data can address them. For category 2: outcome completeness and blinding, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

A survey and interview study asks one integrated question, but the report presents two parallel result sections with no explanation of how findings meet; integration is the issue. Read that example through category 2: outcome completeness and blinding by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

The official MMAT version 2018 guide is designed for systematic mixed-studies reviews containing qualitative, quantitative and mixed-methods primary research. The boundary is decisive for category 2: outcome completeness and blinding. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 2: Adherence to Assigned Intervention

Ask what participants actually received and whether departures change the estimated effect.

What the MMAT criterion asks

Choose among qualitative, randomised quantitative, non-randomised quantitative, quantitative descriptive and mixed-methods categories. For category 2: adherence to assigned intervention, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

A qualitative study uses rich interviews but its stated grounded-theory aim does not match a purely descriptive analysis; the appraisal traces that coherence gap. Read that example through category 2: adherence to assigned intervention by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

The guide says MMAT is not for non-empirical papers such as reviews or theoretical papers and is not suited to some designs including economic and diagnostic-accuracy studies. The boundary is decisive for category 2: adherence to assigned intervention. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 3: Representativeness and Measurement

For non-randomised studies, inspect target-population fit and measurement of intervention, exposure and outcome.

What the MMAT criterion asks

For qualitative studies, connect approach, data collection, derivation of findings, substantiation and coherence. For category 3: representativeness and measurement, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

A randomised classroom trial describes allocation but outcome assessors are teachers who know group assignment; the relevant criterion remains visible without condemning every aspect. Read that example through category 3: representativeness and measurement by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

The developers advise at least two reviewers to appraise independently and recommend relevant training or expertise. The boundary is decisive for category 3: representativeness and measurement. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 3: Confounding and Intended Exposure

Judge whether important confounders are addressed and whether exposure or intervention occurred as intended.

What the MMAT criterion asks

For randomised trials, examine randomisation, baseline comparability, outcome completeness, assessor blinding and adherence. For category 3: confounding and intended exposure, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

A non-randomised programme comparison measures baseline achievement and attendance but not prior support, leaving an important confounding route. Read that example through category 3: confounding and intended exposure by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

The guide discourages an overall score because it hides which methodological features are problematic and weights criteria mechanically. The boundary is decisive for category 3: confounding and intended exposure. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 4: Sampling Strategy and Representation

For descriptive studies, connect the sampling route to the population about which the report speaks.

What the MMAT criterion asks

For non-randomised studies, consider representativeness, measurement, complete outcomes, confounding and whether exposure or intervention occurred as intended. For category 4: sampling strategy and representation, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

An online survey samples through one voluntary social-media group; the reader distinguishes a useful descriptive snapshot from representation of all parents. Read that example through category 4: sampling strategy and representation by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

A Can’t tell response signals insufficient information, not a midpoint between Yes and No. The boundary is decisive for category 4: sampling strategy and representation. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 4: Nonresponse and Measurement

Examine who did not respond, whether measurement is appropriate and how both shape the description.

What the MMAT criterion asks

For descriptive studies, inspect sampling relevance, representativeness, measurement, nonresponse and analysis. For category 4: nonresponse and measurement, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

Interview findings explain why average test scores improved for some learners but not others; the mixed-methods interpretation shows exactly how strands connect. Read that example through category 4: nonresponse and measurement by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

Mixed methods means purposeful integration of qualitative and quantitative components, not merely placing two methods in one paper. The boundary is decisive for category 4: nonresponse and measurement. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 4: Statistical Analysis

Check whether estimates and analyses answer the descriptive question with suitable uncertainty.

What the MMAT criterion asks

For mixed-methods studies, appraise each component and then the rationale, integration, interpretation, divergences and adherence to each tradition. For category 4: statistical analysis, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

Quantitative and qualitative findings conflict; the report examines timing and participant differences instead of choosing the more convenient strand. Read that example through category 4: statistical analysis by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

Methodological quality is separate from topic importance, ethical value and local applicability. The boundary is decisive for category 4: statistical analysis. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 5: Rationale for Mixed Methods

Require a reason that combining qualitative and quantitative evidence adds value to the research question.

What the MMAT criterion asks

Use Can’t tell when reporting is insufficient, and document what information would be needed to answer Yes or No. For category 5: rationale for mixed methods, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

Two reviewers record criterion-level notes and later compare patterns in sensitivity analysis rather than converting five answers into a percentage score. Read that example through category 5: rationale for mixed methods by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

Automatic exclusion of lower-quality studies is usually discouraged; criterion-level reporting and sensitivity analysis can preserve more information. The boundary is decisive for category 5: rationale for mixed methods. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 5: Effective Integration

Locate where components are connected in design, sampling, data collection, analysis or interpretation.

What the MMAT criterion asks

Start with the two screening questions: whether research questions are clear and whether collected data can address them. For category 5: effective integration, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

A survey and interview study asks one integrated question, but the report presents two parallel result sections with no explanation of how findings meet; integration is the issue. Read that example through category 5: effective integration by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

The official MMAT version 2018 guide is designed for systematic mixed-studies reviews containing qualitative, quantitative and mixed-methods primary research. The boundary is decisive for category 5: effective integration. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 5: Interpret the Product of Integration

Ask what new understanding emerged from the connection rather than restating two separate result lists.

What the MMAT criterion asks

Choose among qualitative, randomised quantitative, non-randomised quantitative, quantitative descriptive and mixed-methods categories. For category 5: interpret the product of integration, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

A qualitative study uses rich interviews but its stated grounded-theory aim does not match a purely descriptive analysis; the appraisal traces that coherence gap. Read that example through category 5: interpret the product of integration by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

The guide says MMAT is not for non-empirical papers such as reviews or theoretical papers and is not suited to some designs including economic and diagnostic-accuracy studies. The boundary is decisive for category 5: interpret the product of integration. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 5: Address Divergence and Inconsistency

Treat disagreement between strands as evidence to investigate, not a nuisance to remove.

What the MMAT criterion asks

For qualitative studies, connect approach, data collection, derivation of findings, substantiation and coherence. For category 5: address divergence and inconsistency, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

A randomised classroom trial describes allocation but outcome assessors are teachers who know group assignment; the relevant criterion remains visible without condemning every aspect. Read that example through category 5: address divergence and inconsistency by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

The developers advise at least two reviewers to appraise independently and recommend relevant training or expertise. The boundary is decisive for category 5: address divergence and inconsistency. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Category 5: Respect Each Methodological Tradition

A mixed-methods label cannot repair a weak qualitative or quantitative component.

What the MMAT criterion asks

For randomised trials, examine randomisation, baseline comparability, outcome completeness, assessor blinding and adherence. For category 5: respect each methodological tradition, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

A non-randomised programme comparison measures baseline achievement and attendance but not prior support, leaving an important confounding route. Read that example through category 5: respect each methodological tradition by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

The guide discourages an overall score because it hides which methodological features are problematic and weights criteria mechanically. The boundary is decisive for category 5: respect each methodological tradition. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Use Can’t Tell Honestly

Record missing or unclear information and identify companion papers or author questions that could resolve it.

What the MMAT criterion asks

For non-randomised studies, consider representativeness, measurement, complete outcomes, confounding and whether exposure or intervention occurred as intended. For use can’t tell honestly, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

An online survey samples through one voluntary social-media group; the reader distinguishes a useful descriptive snapshot from representation of all parents. Read that example through use can’t tell honestly by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

A Can’t tell response signals insufficient information, not a midpoint between Yes and No. The boundary is decisive for use can’t tell honestly. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Did You Know? Mixed Methods Is About Integration

Show why one interview quotation beside one chart is not automatically a mixed-methods design.

What the MMAT criterion asks

For descriptive studies, inspect sampling relevance, representativeness, measurement, nonresponse and analysis. For did you know? mixed methods is about integration, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

Interview findings explain why average test scores improved for some learners but not others; the mixed-methods interpretation shows exactly how strands connect. Read that example through did you know? mixed methods is about integration by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

Mixed methods means purposeful integration of qualitative and quantitative components, not merely placing two methods in one paper. The boundary is decisive for did you know? mixed methods is about integration. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Report Criteria Instead of an Overall Score

Present the Yes, No and Can’t tell pattern with explanations so readers see where strengths and concerns lie.

What the MMAT criterion asks

For mixed-methods studies, appraise each component and then the rationale, integration, interpretation, divergences and adherence to each tradition. For report criteria instead of an overall score, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

Quantitative and qualitative findings conflict; the report examines timing and participant differences instead of choosing the more convenient strand. Read that example through report criteria instead of an overall score by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

Methodological quality is separate from topic importance, ethical value and local applicability. The boundary is decisive for report criteria instead of an overall score. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Finish With an MMAT-Informed Synthesis Plan

Explain how appraisal shapes sensitivity analysis, interpretation and future research without automatic deletion.

What the MMAT criterion asks

Use Can’t tell when reporting is insufficient, and document what information would be needed to answer Yes or No. For finish with an mmat-informed synthesis plan, identify the study design before choosing criteria. The MMAT version 2018 does not invite a reader to rate every row for every paper. It asks the team to use the relevant design category, preserve Yes, No and Can’t tell responses, and explain the source passage or missing detail that supports each answer.

Mixed-study example

Two reviewers record criterion-level notes and later compare patterns in sensitivity analysis rather than converting five answers into a percentage score. Read that example through finish with an mmat-informed synthesis plan by separating the qualitative component, the quantitative component and—when the study truly uses a mixed-methods design—the integration between them. Good English makes the connecting logic visible: what one strand contributes, where the strands meet, what happens when they disagree and how the final interpretation is warranted.

Use, limit and next action

Automatic exclusion of lower-quality studies is usually discouraged; criterion-level reporting and sensitivity analysis can preserve more information. The boundary is decisive for finish with an mmat-informed synthesis plan. Do not turn the five criteria into an unsupported overall percentage or use Can’t tell as a polite Yes. Record the criterion-level pattern, consider companion papers or author contact, and explain how the appraisal informs synthesis. The developers discourage automatic exclusion of studies solely because methodological quality is low; thoughtful sensitivity analysis and transparent discussion are often more informative.

---

Frequently Asked Questions

What is MMAT?

MMAT is the Mixed Methods Appraisal Tool. Version 2018 supports critical appraisal of qualitative, quantitative and mixed-methods primary studies in systematic mixed-studies reviews.

How many study categories does MMAT 2018 include?

Five: qualitative, quantitative randomised controlled trials, quantitative non-randomised, quantitative descriptive and mixed methods.

How many criteria are used for each category?

Five criteria for the relevant category, after two initial screening questions.

Should every paper be rated against all five categories?

No. Choose the appropriate category from the methods. A genuine mixed-methods study requires appraisal of its qualitative and quantitative components as well as mixed-methods integration.

Why is an overall MMAT score discouraged?

A total hides which criteria are problematic and gives equal numerical weight to different issues. Criterion-level reporting is more informative.

What does Can’t tell mean?

The report lacks enough clear information to answer Yes or No. It is an information-gap judgement, not a neutral pass.

Should studies with low methodological quality be excluded?

The official guide says automatic exclusion is usually discouraged. Reviewers should follow a prespecified plan and may use sensitivity analysis and cautious interpretation.

Can students use MMAT thinking outside a systematic review?

Yes. The design-classification and evidence-linked-question habits help students avoid judging interviews, experiments and surveys by the wrong standards.

---

Useful Next Reading

• Read the official MMAT version 2018 user guide.
• Read the MMAT 2018 methods paper.
• Read interview and focus-group reports with COREQ.
• Read statistical methods and results with SAMPL.

Closing thought: MMAT 2018 shows why English is a bridge across methods: it keeps each design’s logic intact while making the connections, disagreements and limits of a mixed evidence base visible.