Why English? | Reading PROBAST+AI Prediction Models Through Development Quality, Evaluation Bias, Applicability and Fairness

PROBAST+AI prediction model appraisal matters because a model can look impressive while its data, outcome, validation or intended use remains unclear. People searching for AI model bias, prediction model risk of bias or PROBAST guidance need to separate the process of developing a model from the process of evaluating its performance. Those are different questions with different failure modes.

The open PROBAST+AI statement, published on 24 March 2025 and checked on 5 November 2026, says the update extends and replaces PROBAST-2019. It has two parts. Model development uses 16 targeted signalling questions to assess quality and applicability. Model evaluation uses 18 signalling questions to assess risk of bias and applicability. Both parts cover participants and data sources, predictors, outcome and analysis; fairness and algorithmic bias are considered throughout.

English is a safety mechanism here. Training, tuning, internal validation, external validation and implementation do not mean the same thing. Discrimination is not calibration. Average performance is not fairness. An algorithmic score is not a decision. Precise writing keeps the target population, moment of prediction, model version, data split, outcome definition, performance estimate and uncertainty connected.

Readers should pair this appraisal guide with the existing TRIPOD+AI reporting guide. TRIPOD+AI helps authors disclose model studies; PROBAST+AI helps readers assess quality, risk of bias and applicability. The course and career information guide is a useful reminder that a prediction should inform a decision, not replace personal evidence and human judgement.

---

Start With the Intended Use and Decision Moment

Define who will use the model, for whom, to predict what, at which point and with what consequence.

Development or evaluation?

The 2025 update uses two parts: 16 signalling questions for development quality and applicability, and 18 for evaluation risk of bias and applicability. For start with the intended use and decision moment, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A model is trained and evaluated on the same records and reports excellent accuracy; apparent performance may be optimistic because evaluation is not independent. Apply the example to start with the intended use and decision moment by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

PROBAST+AI was published on 24 March 2025 and the authors state that it extends and replaces PROBAST-2019. This boundary protects start with the intended use and decision moment from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Identify the Exact Model Version

Preserve algorithm, coefficients, preprocessing, thresholds and software version rather than relying on a product name.

Development or evaluation?

Both parts organise questions across participants and data sources, predictors, outcome and analysis. For identify the exact model version, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A random data split leaks repeated records from one patient into both sets, so the evaluation does not represent performance on genuinely new people. Apply the example to identify the exact model version by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

The update covers prediction models developed with regression or artificial-intelligence and machine-learning approaches. This boundary protects identify the exact model version from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Separate Development From Performance Evaluation

Map which data and methods created the model and which data estimated how well it performs.

Development or evaluation?

Define the intended use, target population and moment of prediction before judging any model report. For separate development from performance evaluation, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A school-risk model uses end-of-term attendance to predict an outcome that staff need to address mid-term; the predictor is unavailable at the decision point. Apply the example to separate development from performance evaluation by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

Development quality and evaluation risk of bias are not interchangeable; a well-developed model can still be evaluated with biased methods. This boundary protects separate development from performance evaluation from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Choose the Correct PROBAST+AI Part

Use development questions for model quality and evaluation questions for risk of bias without blending the judgements.

Development or evaluation?

Separate apparent performance from internal validation and external validation; they answer different questions about optimism and transport. For choose the correct probast+ai part, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

An outcome label comes from billing codes that differ across hospitals; apparent portability may reflect coding practice rather than the underlying condition. Apply the example to choose the correct probast+ai part by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

High discrimination does not establish good calibration, fairness, clinical usefulness or benefit after implementation. This boundary protects choose the correct probast+ai part from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Domain 1: Participants and Data Sources

Assess setting, eligibility, sampling, time period, data provenance and fit with the intended population.

Development or evaluation?

Check whether predictor values would really be available at the intended decision moment and measured consistently in the target setting. For domain 1: participants and data sources, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

Average performance is acceptable, but calibration is poor for an under-represented group; the appraisal keeps subgroup evidence and uncertainty visible. Apply the example to domain 1: participants and data sources by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

External validation in one dataset does not guarantee transport to every population, year or workflow. This boundary protects domain 1: participants and data sources from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Check for Duplicates, Clustering and Leakage

Keep repeated patients, sites, images and time-linked records together when splitting data.

Development or evaluation?

Inspect outcome definition, timing and assessor knowledge because a convenient label can be an unstable target. For check for duplicates, clustering and leakage, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

External validation occurs in a different country but the report gives little information about prevalence, care pathways and predictor measurement; applicability remains uncertain. Apply the example to check for duplicates, clustering and leakage by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

Applicability asks whether participants, predictors and outcomes match the intended use; it is distinct from internal risk of bias. This boundary protects check for duplicates, clustering and leakage from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Read Missingness as a Modelling Decision

Inspect why data are missing, how missing values are handled and whether the strategy is available in practice.

Development or evaluation?

Examine sample size, missing data, model complexity, validation, calibration, discrimination and handling of performance uncertainty together. For read missingness as a modelling decision, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A complex machine-learning model uses many predictors with few outcome events; internal validation cannot fully rescue an unstable development process. Apply the example to read missingness as a modelling decision by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

Fairness requires appropriate data and analysis, but no checklist can decide social values or acceptable trade-offs on its own. This boundary protects read missingness as a modelling decision from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Domain 2: Predictors

Check definition, timing, measurement, availability, blinding and stability of every important predictor.

Development or evaluation?

Look for fairness and algorithmic-bias concerns throughout the four domains rather than confining them to one demographic table. For domain 2: predictors, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A student project predicts course completion and compares discrimination with calibration, missingness, fairness and practical consequences instead of celebrating one accuracy percentage. Apply the example to domain 2: predictors by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

PROBAST+AI is an appraisal framework, not a reporting guideline, product approval or permission to automate consequential decisions. This boundary protects domain 2: predictors from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Preserve the Moment of Prediction

Exclude information that becomes available only after the decision the model is supposed to support.

Development or evaluation?

The 2025 update uses two parts: 16 signalling questions for development quality and applicability, and 18 for evaluation risk of bias and applicability. For preserve the moment of prediction, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A model is trained and evaluated on the same records and reports excellent accuracy; apparent performance may be optimistic because evaluation is not independent. Apply the example to preserve the moment of prediction by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

PROBAST+AI was published on 24 March 2025 and the authors state that it extends and replaces PROBAST-2019. This boundary protects preserve the moment of prediction from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Handle Predictor Selection and Complexity Carefully

Examine data-driven selection, effective sample size, interactions, nonlinear terms and regularisation.

Development or evaluation?

Both parts organise questions across participants and data sources, predictors, outcome and analysis. For handle predictor selection and complexity carefully, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A random data split leaks repeated records from one patient into both sets, so the evaluation does not represent performance on genuinely new people. Apply the example to handle predictor selection and complexity carefully by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

The update covers prediction models developed with regression or artificial-intelligence and machine-learning approaches. This boundary protects handle predictor selection and complexity carefully from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Domain 3: Outcome

Assess definition, measurement, timing, assessor knowledge and whether outcome labels match the intended prediction.

Development or evaluation?

Define the intended use, target population and moment of prediction before judging any model report. For domain 3: outcome, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A school-risk model uses end-of-term attendance to predict an outcome that staff need to address mid-term; the predictor is unavailable at the decision point. Apply the example to domain 3: outcome by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

Development quality and evaluation risk of bias are not interchangeable; a well-developed model can still be evaluated with biased methods. This boundary protects domain 3: outcome from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Treat Proxy Labels With Caution

Administrative codes and convenient surrogates may encode workflow, access and historical inequity rather than the target condition.

Development or evaluation?

Separate apparent performance from internal validation and external validation; they answer different questions about optimism and transport. For treat proxy labels with caution, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

An outcome label comes from billing codes that differ across hospitals; apparent portability may reflect coding practice rather than the underlying condition. Apply the example to treat proxy labels with caution by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

High discrimination does not establish good calibration, fairness, clinical usefulness or benefit after implementation. This boundary protects treat proxy labels with caution from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Domain 4: Analysis in Model Development

Inspect model fitting, overfitting control, tuning, resampling, optimism and presentation of the final model.

Development or evaluation?

Check whether predictor values would really be available at the intended decision moment and measured consistently in the target setting. For domain 4: analysis in model development, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

Average performance is acceptable, but calibration is poor for an under-represented group; the appraisal keeps subgroup evidence and uncertainty visible. Apply the example to domain 4: analysis in model development by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

External validation in one dataset does not guarantee transport to every population, year or workflow. This boundary protects domain 4: analysis in model development from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Appraise Apparent Performance

Recognise estimates produced on the development data as especially vulnerable to optimism.

Development or evaluation?

Inspect outcome definition, timing and assessor knowledge because a convenient label can be an unstable target. For appraise apparent performance, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

External validation occurs in a different country but the report gives little information about prevalence, care pathways and predictor measurement; applicability remains uncertain. Apply the example to appraise apparent performance by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

Applicability asks whether participants, predictors and outcomes match the intended use; it is distinct from internal risk of bias. This boundary protects appraise apparent performance from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Appraise Internal Validation

Ask whether bootstrapping, cross-validation or splitting preserves independence and reflects the full development process.

Development or evaluation?

Examine sample size, missing data, model complexity, validation, calibration, discrimination and handling of performance uncertainty together. For appraise internal validation, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A complex machine-learning model uses many predictors with few outcome events; internal validation cannot fully rescue an unstable development process. Apply the example to appraise internal validation by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

Fairness requires appropriate data and analysis, but no checklist can decide social values or acceptable trade-offs on its own. This boundary protects appraise internal validation from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Appraise External Validation

Check whether evaluation uses genuinely separate data and whether differences in setting, time and measurement are reported.

Development or evaluation?

Look for fairness and algorithmic-bias concerns throughout the four domains rather than confining them to one demographic table. For appraise external validation, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A student project predicts course completion and compares discrimination with calibration, missingness, fairness and practical consequences instead of celebrating one accuracy percentage. Apply the example to appraise external validation by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

PROBAST+AI is an appraisal framework, not a reporting guideline, product approval or permission to automate consequential decisions. This boundary protects appraise external validation from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Read Discrimination and Calibration Together

A model may rank people well while systematically overpredicting or underpredicting absolute risk.

Development or evaluation?

The 2025 update uses two parts: 16 signalling questions for development quality and applicability, and 18 for evaluation risk of bias and applicability. For read discrimination and calibration together, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A model is trained and evaluated on the same records and reports excellent accuracy; apparent performance may be optimistic because evaluation is not independent. Apply the example to read discrimination and calibration together by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

PROBAST+AI was published on 24 March 2025 and the authors state that it extends and replaces PROBAST-2019. This boundary protects read discrimination and calibration together from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Examine Classification Thresholds and Consequences

Connect sensitivity, specificity and decision thresholds to false positives, false negatives and intended action.

Development or evaluation?

Both parts organise questions across participants and data sources, predictors, outcome and analysis. For examine classification thresholds and consequences, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A random data split leaks repeated records from one patient into both sets, so the evaluation does not represent performance on genuinely new people. Apply the example to examine classification thresholds and consequences by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

The update covers prediction models developed with regression or artificial-intelligence and machine-learning approaches. This boundary protects examine classification thresholds and consequences from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Keep Performance Uncertainty Visible

Require confidence intervals or other uncertainty information and avoid celebrating small numerical differences.

Development or evaluation?

Define the intended use, target population and moment of prediction before judging any model report. For keep performance uncertainty visible, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A school-risk model uses end-of-term attendance to predict an outcome that staff need to address mid-term; the predictor is unavailable at the decision point. Apply the example to keep performance uncertainty visible by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

Development quality and evaluation risk of bias are not interchangeable; a well-developed model can still be evaluated with biased methods. This boundary protects keep performance uncertainty visible from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Assess Applicability Separately

Compare participants, predictors and outcome with the intended use even when internal methods appear sound.

Development or evaluation?

Separate apparent performance from internal validation and external validation; they answer different questions about optimism and transport. For assess applicability separately, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

An outcome label comes from billing codes that differ across hospitals; apparent portability may reflect coding practice rather than the underlying condition. Apply the example to assess applicability separately by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

High discrimination does not establish good calibration, fairness, clinical usefulness or benefit after implementation. This boundary protects assess applicability separately from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Make Fairness and Algorithmic Bias Explicit

Examine representation, measurement, performance and consequences across relevant groups with adequate uncertainty.

Development or evaluation?

Check whether predictor values would really be available at the intended decision moment and measured consistently in the target setting. For make fairness and algorithmic bias explicit, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

Average performance is acceptable, but calibration is poor for an under-represented group; the appraisal keeps subgroup evidence and uncertainty visible. Apply the example to make fairness and algorithmic bias explicit by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

External validation in one dataset does not guarantee transport to every population, year or workflow. This boundary protects make fairness and algorithmic bias explicit from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Did You Know? More Data Can Repeat Old Bias

Explain why a large historical dataset may reproduce uneven access, labels or decisions at greater scale.

Development or evaluation?

Inspect outcome definition, timing and assessor knowledge because a convenient label can be an unstable target. For did you know? more data can repeat old bias, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

External validation occurs in a different country but the report gives little information about prevalence, care pathways and predictor measurement; applicability remains uncertain. Apply the example to did you know? more data can repeat old bias by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

Applicability asks whether participants, predictors and outcomes match the intended use; it is distinct from internal risk of bias. This boundary protects did you know? more data can repeat old bias from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Translate Model Findings for Students and Parents

Describe what a score can support, what it cannot decide and which human evidence should remain in the conversation.

Development or evaluation?

Examine sample size, missing data, model complexity, validation, calibration, discrimination and handling of performance uncertainty together. For translate model findings for students and parents, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A complex machine-learning model uses many predictors with few outcome events; internal validation cannot fully rescue an unstable development process. Apply the example to translate model findings for students and parents by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

Fairness requires appropriate data and analysis, but no checklist can decide social values or acceptable trade-offs on its own. This boundary protects translate model findings for students and parents from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Finish With a PROBAST+AI Decision Audit

Document domain evidence, development quality, evaluation bias, applicability, fairness and the next validation needed before use.

Development or evaluation?

Look for fairness and algorithmic-bias concerns throughout the four domains rather than confining them to one demographic table. For finish with a probast+ai decision audit, first decide whether the report describes how a model was developed, how its performance was evaluated, or both. PROBAST+AI deliberately separates development quality from risk of bias in performance evaluation. English matters because words such as training, tuning, internal validation, external validation and deployment refer to different stages and should not be blended into one claim of accuracy.

Prediction-model example

A student project predicts course completion and compares discrimination with calibration, missingness, fairness and practical consequences instead of celebrating one accuracy percentage. Apply the example to finish with a probast+ai decision audit by tracing participants and data sources, predictors, outcome and analysis. Ask which population the model targets, when predictor values are available, how the outcome is defined, which performance estimate is reported and whether evaluation data are genuinely separate. A polished dashboard cannot repair a weak temporal split, an unstable outcome label or a small effective sample.

Fairness, applicability and decision boundary

PROBAST+AI is an appraisal framework, not a reporting guideline, product approval or permission to automate consequential decisions. This boundary protects finish with a probast+ai decision audit from overclaiming. Keep methodological quality, risk of bias, applicability and fairness as related but distinct judgements. A model can look accurate on average yet fail in an under-represented group or a new setting. Finish with the supported use, the main uncertainty and the next check before anyone treats the model as a decision-maker rather than one input into responsible human judgement.

---

Frequently Asked Questions

What is PROBAST+AI?

It is the 2025 update of PROBAST for appraising prediction models developed with regression or AI and machine-learning methods. The authors state that it replaces PROBAST-2019.

How is model development different from model evaluation?

Development constructs the model. Evaluation estimates its performance. PROBAST+AI assesses development quality and evaluation risk of bias separately.

How many signalling questions are there?

The development part has 16 targeted questions; the evaluation part has 18. Both use four domains.

What are the four domains?

Participants and data sources, predictors, outcome, and analysis.

Is high accuracy enough to deploy a model?

No. Accuracy may hide class imbalance and says nothing by itself about calibration, fairness, applicability, consequences or benefit.

What is data leakage?

Leakage occurs when evaluation data or future information influence development, producing performance that may not hold for genuinely new cases.

Does external validation prove universal usefulness?

No. It supports performance in the tested setting and period. Other populations, workflows, years and measurement systems may differ.

Can a model replace teachers, clinicians or advisers?

PROBAST+AI does not grant that authority. A model should be one appropriately validated input within accountable human decision-making.

---

Useful Next Reading

• Read the open 2025 PROBAST+AI statement.
• Read the PROBAST+AI PubMed record and publication details.
• Read prediction-model reports with TRIPOD+AI.
• Read statistical methods, assumptions and results with SAMPL.

Closing thought: PROBAST+AI shows why English belongs inside responsible modelling: it separates construction from evaluation, accuracy from usefulness, averages from fairness and an algorithmic estimate from the human decision that follows.