eduKateSG · Genetics · The Gold Standard of Education
Evaluate an ancestry-aware polygenic score as a prediction model, not a genetic verdict
Choose the explanation you need, or follow all twelve chapters in order.
Full chapter index · Independent practice · Genetics subject guide
A polygenic score combines many genetic variants using weights estimated in a discovery study. It can rank people by one component of predicted risk, but its accuracy and calibration depend on the population, phenotype, genotyping system, environment and clinical pathway in which it is used.
This guide follows a score from discovery weights to an independent target cohort. It separates discrimination, calibration and decision value; explains why performance can change along an ancestry continuum; and shows what must be checked before a score informs real care.
Continue from The Gold Standard of Genetics within The Gold Standard of Education: learning that transfers into life and the How X Works learning library.
Full chapter index
See the mechanism · Chapters 1–2
Build the worked model · Chapters 3–5
Test what changes · Chapters 6–8
Check evidence and limits · Chapters 9–10
Practise and continue · Chapters 11–12
State the outcome, time horizon, target population and decision. Predicting ten-year disease risk for screening is different from explaining biological mechanism or ranking a continuous trait in a research cohort.
Write the score formula: PGS = the sum across variants of allele dosage multiplied by an estimated weight. Record genome build, effect allele, imputation, weights and whether the score is standardised. A percentile has meaning only relative to a declared reference distribution.
Define the baseline model without the score. Age, sex, family history, clinical measurements and social or environmental variables may already predict the outcome. The relevant question is what the score adds in the intended pathway.
Keep genetic ancestry, race and ethnicity separate. Genetic ancestry is inferred from genomic relationships; race and ethnicity are social and self-identified categories that can correlate with exposures and access but are not substitutes for genotype.
Evidence check: for Define the prediction target and intended use, preserve the observation before the interpretation. Record outcome, horizon, population and clinical use. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Discovery effect estimates are shaped by the sample, phenotype definition, linkage disequilibrium, allele frequencies, genotyping array, imputation panel and statistical model. A target cohort can differ at every point.
A tag variant may predict well in one ancestry background because it correlates with a causal variant there. Different linkage disequilibrium can weaken that correlation elsewhere. Allele-frequency differences change score variance and the number of informative copies.
Environmental and clinical context also matter. If treatment, diet, diagnosis or surveillance differs, the same genetic liability may map to a different observed risk. Phenotype definitions that look identical in code can hide different ascertainment.
Transferability is therefore not a single group property. Performance can vary continuously among individuals and within broad labels.
Evidence check: for Trace where transferability can change, preserve the observation before the interpretation. Record score formula, alleles and reference distribution. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Keep discovery, tuning and evaluation data independent. Using target outcomes to choose variants or tune hyperparameters and then reporting performance in the same people creates optimism.
Align alleles and genome builds carefully. Remove or resolve palindromic variants when strand is uncertain. Match variant identifiers, inspect allele frequencies and document missing variants or substitutions.
Use a linkage-disequilibrium reference that represents the modelling method and target as well as possible. If multiple ancestry-specific or multi-ancestry summary statistics are combined, preserve how their weights were estimated.
Freeze the complete pipeline before final validation. A score is the weights plus variant handling, standardisation, reference, missing-data rule and outcome model—not just a downloadable file.
Evidence check: for Build the score without leakage, preserve the observation before the interpretation. Record discovery, tuning and evaluation independence. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Imagine a five-year outcome with an observed rate of 10% in an independent cohort. A baseline clinical model assigns 100 people to a predicted-risk group averaging 20%. Twenty events would be expected, but only twelve occur. The model is overpredicting in that group.
Adding a polygenic score improves the area under the receiver-operating curve from 0.68 to 0.72. Discrimination improved: pairs with and without the outcome are ordered somewhat better. Yet the combined model still predicts twenty events where twelve occur.
Recalibrating the intercept and slope in a separate calibration sample may bring predicted and observed risk closer. It does not prove that treatment based on the score improves outcomes.
The example shows why one attractive discrimination statistic cannot replace calibration and decision analysis.
| Measure | Baseline model | With polygenic score | Meaning |
|---|---|---|---|
| Five-year outcome rate | 10% | 10% | Observed cohort prevalence |
| AUC | 0.68 | 0.72 | Discrimination improves |
| Predicted events in group | 20 | 20 | Model expectation |
| Observed events in group | 12 | 12 | Overprediction remains |
Evidence check: for Work a fictional risk example, preserve the observation before the interpretation. Record every fictional prediction and event count. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 5 OF 12 · Build the worked model
5. Measure discrimination on the right scale
For binary outcomes, report the area under the curve with uncertainty, but also sensitivity, specificity and predictive values at relevant thresholds. Predictive values depend on outcome prevalence.
For time-to-event outcomes, use concordance and time-specific measures that handle censoring. For quantitative traits, report explained variance and prediction error. Do not compare unlike metrics as if they were the same accuracy.
Assess incremental performance over the intended baseline. A statistically significant association can add little useful ranking when established predictors are strong.
Stratify carefully. Subgroup estimates need adequate events and honest intervals. A noisy higher point estimate is not evidence that one group benefits more.
Evidence check: for Measure discrimination on the right scale, preserve the observation before the interpretation. Record discrimination on the correct scale. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 6 OF 12 · Test what changes
6. Check calibration before communicating absolute risk
Calibration asks whether predicted probabilities agree with observed frequencies. Inspect calibration-in-the-large, slope and curves across the range. A model can discriminate well while systematically over- or underpredicting.
Recalibration may adjust intercept and slope, or refit a broader model when relationships changed. It should use data separate from final evaluation. Recalibration cannot repair missing predictors or unstable biology by arithmetic alone.
Check calibration across relevant ancestry, sex, age and clinical subgroups without treating broad categories as homogeneous. Small cells can create misleading curves.
Communicate absolute risk only from a validated risk model. A PGS percentile by itself is not a five-year probability.
Evidence check: for Check calibration before communicating absolute risk, preserve the observation before the interpretation. Record calibration-in-the-large, slope and curves. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 7 OF 12 · Test what changes
7. Use ancestry-aware methods without erasing people
Multi-ancestry methods may combine summary statistics, ancestry-specific effects, functional annotations or local ancestry. They aim to improve prediction while borrowing information across populations.
Compare a naive transferred score, a target-specific score and the ancestry-aware method in the same held-out data. Report both average performance and performance along genetic-distance or ancestry continua.
Avoid forcing every individual into one discrete ancestry bin. Admixed genomes contain ancestry mosaics, and individual prediction accuracy can differ within conventional groups.
Ancestry adjustment that centres score distributions can make means look similar while leaving calibration or accuracy unequal. State what the transformation changes.
Evidence check: for Use ancestry-aware methods without erasing people, preserve the observation before the interpretation. Record ancestry-aware methods without rigid boxes. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Repeat evaluation across sites, genotyping platforms, imputation panels and phenotype definitions. A score that works only after site-specific exclusions may not be operationally portable.
Use temporal validation where clinical practice changes. Test missingness, medication, family history and population shifts. Monitor performance after deployment because the people offered testing may differ from the research cohort.
Report excluded individuals and failed genotypes. Technical failure that varies with ancestry can create inequity before the model makes a prediction.
External validation is strongest when conducted independently with a locked pipeline. A new cohort used repeatedly for method development is no longer a clean test.
Evidence check: for Test robustness and transport, preserve the observation before the interpretation. Record external, technical and temporal transport. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
A score is useful only if it changes an action that improves outcomes relative to harms and costs. Define thresholds, available interventions and the consequence of false positives and false negatives.
Use decision-curve analysis or another explicit framework to compare net benefit across plausible thresholds. Include the baseline clinical model and simple strategies such as treat all or treat none.
Consider downstream capacity. If many people are reclassified but confirmatory services are unavailable, the score can create anxiety and delay without benefit.
Audit differential access, communication and follow-up. Equal model accuracy does not guarantee equitable care.
| Layer | Question | Evidence | Failure avoided |
|---|---|---|---|
| Association | Does the score relate to outcome? | Independent coefficient | Noisy discovery |
| Discrimination | Does it rank cases higher? | AUC or concordance | Poor ordering |
| Calibration | Do probabilities match events? | Calibration curve | Wrong absolute risk |
| Decision value | Will using it help? | Net benefit and pathway | Unhelpful reclassification |
Evidence check: for Evaluate decision value and harms, preserve the observation before the interpretation. Record decision thresholds, harms and net benefit. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
CHAPTER 10 OF 12 · Check evidence and limits
10. State ethical and interpretive limits
A polygenic score is probabilistic and incomplete. It does not determine destiny, explain a person’s identity or capture all family, environmental and social causes.
Consent should cover data use, return of results and family implications. Genetic information can affect relatives and may carry privacy or discrimination concerns.
Do not use ancestry-aware modelling to justify biological race categories. The scientific problem is transfer across genetic, environmental and healthcare contexts, not a ranking of peoples.
Clinical implementation requires professional governance, transparent reporting and evidence that the pathway benefits the target population. Consumer availability is not clinical validation.
Evidence check: for State ethical and interpretive limits, preserve the observation before the interpretation. Record consent, identity and implementation limits. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Case A is the fictional cohort: discrimination improves, but absolute risk remains overpredicted. Recalibration is required before risk communication.
Case B uses the same score in a cohort with different imputation quality. Missing variants reduce variance and accuracy. Standardising the score hides the technical loss but does not restore information.
Case C has similar discrimination across two groups but worse calibration in one because baseline incidence differs. Equal AUC does not mean equal risk estimates.
Case D shows good calibration but no positive net benefit at actionable thresholds because the intervention is burdensome and the baseline model already identifies most high-risk people. Transfer means carrying the whole decision pathway, not only the score.
Evidence check: for Run a changed-context transfer test, preserve the observation before the interpretation. Record the changed-imputation context. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.
Archive the outcome definition, time horizon, discovery data, summary statistics, variant list, alleles, genome build, linkage-disequilibrium reference, imputation, standardisation, tuning, baseline model, validation split, subgroup counts, discrimination, calibration, decision analysis, failures, governance and exact claim.
Common questions: Is a high score a diagnosis? No. Does equal mean score imply equal performance? No. Can recalibration fix weak discrimination? No. Is ancestry a fixed set of boxes? No. Does better AUC prove clinical benefit? No. Should a score be used outside its validated population and pathway? Not without new evidence.
Continue with local-ancestry association and replication, haplotype sharing and ancestry tracts, and migration, selection and local adaptation. Read the open review of principles for transferring polygenic scores, the study of accuracy across an ancestry continuum, and evidence that diverse discovery variants improve transferability.
Evidence check: for Keep the evidence route and continue, preserve the observation before the interpretation. Record the narrowest supported prediction claim. Then state the inference, one credible alternative, and the additional observation that would distinguish them. This compact audit keeps a fluent explanation tied to evidence that another reader can inspect.
Changed-context transfer: repeat this chapter’s decision after changing one consequential condition while holding the others fixed. Calculate or predict what should change, name the threshold or mechanism, and compare with the original case. If the conclusion survives, state the range tested; if it reverses, explain why. Transfer is demonstrated by the changed case, not asserted from the first example.
Practical decision record: write one sentence for what is known, one for what remains uncertain, and one for the next safe action. Include the evidence owner, the latest useful decision time, and the condition that would stop or reverse the action. This turns a technical result into a reviewable choice without pretending uncertainty has disappeared.