Why English? | Reporting SaML Machine Learning With Complex Survey Data Through Sampling Design, Weights, Clusters, Splits, Calibration and Population Validity

ORIENTATION

What this guide covers

Current fact check

SaML is a proposed minimum reporting statement, not a final checklist. The official record cites large health surveys such as NHANES, BRFSS and NHIS and identifies risks of biased population estimates, understated uncertainty and unreliable fairness assessment when design features are ignored. PRICSSA recommends reporting survey dates, mode, response, sample design, weights, missingness, variance estimation and code.

What this guide does not cover

This article focuses on reporting machine-learning studies that use complex probability-sample survey data. It does not prescribe one algorithm, guarantee unbiased inference, replace survey statisticians or machine-learning specialists, or invent final SaML items.

Worked example

Every chapter returns to a fictional model using a national youth survey to predict persistent disengagement while respecting school clusters, unequal selection probabilities and population weights. It is fictional, low-stakes and educational; it does not report a real school, clinical decision or guaranteed outcome.

Warm up with eduKate’s guides to explaining scientific evidence and asking better questions in class.

CHAPTER 1 OF 16

1. State the Prediction or Modelling Task

Back to contents

Why this move matters

Define outcome, prediction horizon, unit of prediction, intended use and whether the goal is individual prediction, population description, causal estimation or exploration.

Connect the machine-learning pipeline to the survey design

Define outcome, prediction horizon, unit of prediction, intended use and whether the goal is individual prediction, population description, causal estimation or exploration. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: The model predicts persistent disengagement at the next wave, not the effect of a school programme. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

Machine learning does not turn prediction into causation. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “State the Prediction or Modelling Task”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 2 OF 16

2. Name the Survey and Target Population

Back to contents

Why this move matters

Report survey owner, cycles, geography, eligibility, data-collection mode, response and the population the sample was designed to represent.

Connect the machine-learning pipeline to the survey design

Report survey owner, cycles, geography, eligibility, data-collection mode, response and the population the sample was designed to represent. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: The report states which young people in the country are inside and outside the survey frame. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

A national label can hide institutional or geographic exclusions. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Name the Survey and Target Population”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 3 OF 16

3. Describe Stratification, Clustering and Selection

Back to contents

Why this move matters

Explain strata, primary sampling units, stages, oversampling, unequal probabilities and variables required to reproduce the design.

Connect the machine-learning pipeline to the survey design

Explain strata, primary sampling units, stages, oversampling, unequal probabilities and variables required to reproduce the design. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: Schools are primary sampling units and selected classes form the next sampling stage. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

Public-use files may mask or collapse design variables. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Describe Stratification, Clustering and Selection”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 4 OF 16

4. Identify Weights and What They Represent

Back to contents

Why this move matters

Name base, nonresponse, calibration, replicate or longitudinal weights, their cycle and how they relate observations to the target population.

Connect the machine-learning pipeline to the survey design

Name base, nonresponse, calibration, replicate or longitudinal weights, their cycle and how they relate observations to the target population. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: A student with weight 520 represents a different population count from one with weight 85. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

Extreme weights can increase variance and influence training. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Identify Weights and What They Represent”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 5 OF 16

5. Define the Analytic Cohort and Outcome Construction

Back to contents

Why this move matters

Show eligibility, linkage, exclusions, event timing, label source, censoring, leakage risks and denominators before and after preprocessing.

Connect the machine-learning pipeline to the survey design

Show eligibility, linkage, exclusions, event timing, label source, censoring, leakage risks and denominators before and after preprocessing. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: The outcome is constructed only from the future wave and excluded from predictor creation. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

Administrative labels may encode access rather than need. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Define the Analytic Cohort and Outcome Construction”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 6 OF 16

6. Report Features, Coding and Preprocessing

Back to contents

Why this move matters

List predictors, measurement timing, categorical encoding, scaling, transformations, feature selection and whether operations were learned inside training folds.

Connect the machine-learning pipeline to the survey design

List predictors, measurement timing, categorical encoding, scaling, transformations, feature selection and whether operations were learned inside training folds. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: Scaling and category levels are fitted within each training resample. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

Whole-dataset preprocessing can leak information. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Report Features, Coding and Preprocessing”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 7 OF 16

7. Handle Missing Data With Design and Time in View

Back to contents

Why this move matters

Report missingness by variable and group, imputation models, indicators, number of imputations, fold boundaries and variables omitted from public files.

Connect the machine-learning pipeline to the survey design

Report missingness by variable and group, imputation models, indicators, number of imputations, fold boundaries and variables omitted from public files. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: Imputation is fitted within training data and includes design variables where justified. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

Complete-case analysis can alter the represented population. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Handle Missing Data With Design and Time in View”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 8 OF 16

8. Split Data Without Breaking Clusters or Time

Back to contents

Why this move matters

Explain training, validation and test partitions, cluster grouping, repeated people, survey waves, geographic separation, random seeds and leakage checks.

Connect the machine-learning pipeline to the survey design

Explain training, validation and test partitions, cluster grouping, repeated people, survey waves, geographic separation, random seeds and leakage checks. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: All students from one sampled school remain in the same partition. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

A random row split can place near-neighbours on both sides. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Split Data Without Breaking Clusters or Time”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 9 OF 16

9. Explain How Weights Enter Model Training

Back to contents

Why this move matters

State whether weights affect loss, resampling, class balance, feature selection or not at all, and justify the choice relative to the target task.

Connect the machine-learning pipeline to the survey design

State whether weights affect loss, resampling, class balance, feature selection or not at all, and justify the choice relative to the target task. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: Sampling weights enter the loss while class weighting addresses outcome imbalance separately. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

Different weight types solve different problems. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Explain How Weights Enter Model Training”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 10 OF 16

10. Report Tuning and Resampling Under the Survey Design

Back to contents

Why this move matters

Describe search space, folds, nested validation, cluster grouping, weighted metrics, stopping rules, seeds and computational budget.

Connect the machine-learning pipeline to the survey design

Describe search space, folds, nested validation, cluster grouping, weighted metrics, stopping rules, seeds and computational budget. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: Cluster-grouped cross-validation uses a weighted calibration criterion. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

Tuning on the final test set invalidates honest evaluation. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Report Tuning and Resampling Under the Survey Design”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 11 OF 16

11. Evaluate Discrimination and Error for the Target Population

Back to contents

Why this move matters

Report design-aware and unweighted metrics when informative, confidence intervals, class prevalence, thresholds and the population interpretation of each metric.

Connect the machine-learning pipeline to the survey design

Report design-aware and unweighted metrics when informative, confidence intervals, class prevalence, thresholds and the population interpretation of each metric. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: Weighted area under the curve and threshold errors are reported alongside raw sample results. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

A single metric hides important trade-offs. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Evaluate Discrimination and Error for the Target Population”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 12 OF 16

12. Report Calibration, Thresholds and Decision Consequences

Back to contents

Why this move matters

Show calibration plots or summaries, recalibration, threshold selection, false-positive and false-negative consequences and who chose the operating point.

Connect the machine-learning pipeline to the survey design

Show calibration plots or summaries, recalibration, threshold selection, false-positive and false-negative consequences and who chose the operating point. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: A support threshold is selected with educators and evaluated across plausible resource capacity. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

Good ranking does not guarantee accurate probabilities. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Report Calibration, Thresholds and Decision Consequences”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 13 OF 16

13. Quantify Uncertainty With the Design Structure

Back to contents

Why this move matters

Explain replicate weights, Taylor linearisation, cluster bootstrap or other methods, including how model fitting and tuning uncertainty are handled.

Connect the machine-learning pipeline to the survey design

Explain replicate weights, Taylor linearisation, cluster bootstrap or other methods, including how model fitting and tuning uncertainty are handled. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: Bootstrap resamples primary sampling units rather than individual rows. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

One uncertainty method may not capture every pipeline choice. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Quantify Uncertainty With the Design Structure”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 14 OF 16

14. Audit Fairness With Representation and Measurement in View

Back to contents

Why this move matters

Report group definitions, sample and population counts, weighting, measurement quality, uncertainty, intersectionality and the limits of parity metrics.

Connect the machine-learning pipeline to the survey design

Report group definitions, sample and population counts, weighting, measurement quality, uncertainty, intersectionality and the limits of parity metrics. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: Error rates are compared with weighted denominators and wide intervals for smaller groups. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

A fairness metric can hide exclusion from the sampling frame. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Audit Fairness With Representation and Measurement in View”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 15 OF 16

15. Test Temporal, Geographic and Survey-Cycle Transport

Back to contents

Why this move matters

Describe validation in later cycles or held-out regions, changes in instruments, policy, coding, prevalence and recalibration needs.

Connect the machine-learning pipeline to the survey design

Describe validation in later cycles or held-out regions, changes in instruments, policy, coding, prevalence and recalibration needs. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: The model is tested in the next survey cycle after questionnaire changes are mapped. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

Internal validation cannot establish future stability. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Test Temporal, Geographic and Survey-Cycle Transport”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

CHAPTER 16 OF 16

16. Publish Code, Versions and a Bounded Population Claim

Back to contents

Why this move matters

Share permissible code, variable dictionaries, software and seeds; state what the model supports, what it cannot represent and what monitoring or validation is next.

Connect the machine-learning pipeline to the survey design

Share permissible code, variable dictionaries, software and seeds; state what the model supports, what it cannot represent and what monitoring or validation is next. Name the target population, strata, clusters, selection probabilities, weights, waves and design variables before explaining how data were split, trained, tuned and evaluated.

Replay one design-aware modelling pathway

Worked example: The report supports population-aware evaluation in one survey and does not claim universal individual prediction. Trace the survey sample, eligible analytic set, cluster-respecting split, weighted loss or alternative, tuning, calibration, uncertainty and population-level interpretation.

Separate predictive performance from population validity

A model can score well on an unweighted test set while misrepresenting the population the survey was designed to describe. Discrimination, calibration, fairness and uncertainty are separate reporting jobs.

Respect the development boundary

Public-use restrictions may prevent full row-level sharing. SaML was registered on 18 May 2026 and is under development. PRICSSA and TRIPOD+AI are neighbouring guidance, not substitutes for survey-aware pipeline reporting.

Student, parent and career transfer

Make one evidence card for “Publish Code, Versions and a Bounded Population Claim”: source location, plain-language meaning, limit and next question. A parent or teacher can ask for one everyday explanation and one specialist explanation; both must preserve the same conditions. This habit supports data science, statistics, survey methodology, public health, education research, economics, social policy, machine learning, responsible AI, evaluation and scientific communication without promising a grade, admission, funding decision or job.

PRACTICE LAB

Build a SaML survey-aware machine-learning reporting decision trail

Student route

1. Choose one original report inside this guide’s scope.
2. Copy its objective and underline actor, action, context, evidence and intended decision.
3. Draw the specific reasoning chain used in this article.
4. Mark each step reported, inferred, contested or unknown.
5. Rewrite the conclusion so it preserves the narrowest important limitation.

Parent or teacher route

Ask the learner to explain how English helps us make a machine-learning analysis of complex survey data reproducible from population sampling and design variables to training, evaluation and population-level interpretation twice: once to a classmate and once to a method specialist. Praise a precise correction and a well-stated unknown; both show disciplined attention to evidence.

Early-career route

Create a one-page SaML survey-aware machine-learning reporting audit with source link, date checked, scope, method logic, decisive wording, contrary evidence, limit and next action. A colleague should be able to continue without guessing.

Continue with eduKate’s guides to reading course and career information and giving useful feedback in group projects.

USEFUL NEXT READING

Continue with a related guide

Read the Full SBB data-literacy and statistical-claims guide for a related but different reader job. Return here when the question is specifically how to make a machine-learning analysis of complex survey data reproducible from population sampling and design variables to training, evaluation and population-level interpretation. Use the linked guide for its separate reader task, and this guide for the question defined here.

For English practice across research, school and career communication, visit the Singapore English Tuition Centre learning site.

FREQUENTLY ASKED QUESTIONS

Questions students and parents often ask

Is SaML final?

No. EQUATOR lists the statement as under development after registration on 18 May 2026.

What is a complex survey?

It commonly uses stratification, clusters and unequal selection probabilities rather than a simple random sample.

Why do survey weights matter?

They connect sampled observations to the target population and may affect estimates, training and evaluation.

Can I randomly split rows?

Not automatically. Repeated people or sampled clusters may need to remain together to avoid leakage.

Is high accuracy enough?

No. Calibration, uncertainty, representation, subgroup performance and intended use also matter.

How is SaML different from TRIPOD+AI?

TRIPOD+AI covers prediction-model reporting broadly; SaML focuses on the survey-design features that interact with machine-learning pipelines.