GOLD STANDARD · PROBABILITY · HOW X WORKS
Find the decision hidden inside the model
A linkage pipeline cannot estimate the population it never allows itself to see. Blocking-recall audits sample the excluded pair space, while cost-sensitive review puts limited human attention where it most improves the zero-truncated population decision.
This guide connects the Gold Standard of Probability, Gold Standard of Education, and How X Works hub.
Chapter contents and reading routes
Choose the route closest to your present task. Every chapter remains expanded, so the article also reads naturally from start to finish.
Orient the audit problem
Allocate review effort
Propagate findings to totals
Transfer and govern
Use evidence and next routes
CHAPTER 1 OF 12 · ORIENT THE DECISION
1. The pairs outside a block still matter
Blocking makes record linkage computationally possible by deciding which pairs will be scored. A true link excluded at that stage becomes a false negative before the classifier starts. Reviewing only candidate pairs can estimate precision inside the block, but it cannot measure how many true links the block missed.
A blocking-recall audit deliberately samples from the excluded pair space using wider keys, trusted identifiers or a labelled validation frame. The result is an estimate of candidate-generation recall, with uncertainty, rather than a comforting assumption that the block found everything important.
CHAPTER 2 OF 12 · ORIENT THE DECISION
2. Keep three review goals distinct
One review can estimate global linkage error, another can calibrate score bands, and another can search for missed links in excluded regions. These objectives imply different sampling frames and allocations. A sample optimised for high-score precision may contribute little evidence about blocking recall.
Write the estimand first: false-match rate among accepted links, missed-match rate among excluded pairs, subgroup recall or movement in the final population total. Then design the clerical sample around that quantity instead of asking reviewers to inspect an undifferentiated queue.
CHAPTER 3 OF 12 · BUILD THE METHOD
3. A worked allocation across score bands
Suppose the candidate set has 2,000 high-score pairs, 8,000 middle-score pairs and 90,000 low-score or excluded pairs. A uniform sample of 300 pairs would spend 270 reviews in the huge low band even if almost none are plausible. Equal allocation of 100 per band improves comparison but ignores different uncertainty and decision value.
A cost-sensitive design might allocate 60 to high scores, 160 to the ambiguous middle and 80 to the excluded search frame. The allocation is not automatically optimal; it expresses that middle-band classification and excluded-pair recall are most likely to change overlap and the unseen-zero estimate.
| Review stratum | Pairs in frame | Illustrative reviews |
|---|---|---|
| Accepted high scores | 2,000 | 60 |
| Ambiguous middle | 8,000 | 160 |
| Low or excluded search frame | 90,000 | 80 |
CHAPTER 4 OF 12 · BUILD THE METHOD
4. Sample excluded pairs intelligently without hiding the design
Pure simple random sampling from millions of excluded pairs can find almost no true links. Use broader blocking passes, agreement patterns, record ambiguity and demographic strata to enrich plausible misses, but retain known inclusion probabilities for estimation.
A targeted hunt can discover failure modes, while a probability sample supports an error estimate. Combining them is useful only when the discovery sample and estimation sample remain labelled. Otherwise the audit can improve the system yet produce a biased performance number.
CHAPTER 5 OF 12 · BUILD THE METHOD
5. Turn reviewer effort into an explicit loss function
A false accepted link can inflate overlap and push a population total downward. A missed link can reduce overlap and push the total upward. Review time also has a cost, and errors in a small policy-relevant subgroup may matter more than the same number elsewhere.
Define losses before seeing labels: total-size movement, decision-threshold crossing, subgroup harm and review minutes. Cost-sensitive linkage does not mean putting a price on people; it means allocating finite evidence collection to the mistakes that change the stated decision.
CHAPTER 6 OF 12 · TEST THE RESULT
6. Weight audit results back to the linkage frame
If middle-score pairs are reviewed at a higher rate, their raw match proportion cannot be treated as the overall rate. Use inverse inclusion probabilities or a design-based estimator to recover frame-level quantities. Preserve stratum sizes, selection probabilities, nonresponse and adjudication rules.
Report reviewer disagreement separately. A pair with uncertain truth is not safely converted into a certain nonmatch. Dual review, adjudication and an indeterminate category make the evidence more honest.
CHAPTER 7 OF 12 · TEST THE RESULT
7. Propagate blocking recall into the GIGP total
Suppose the current linkage identifies 240 overlaps and a weighted excluded-pair audit estimates 12 missed links, with a plausible range of 6 to 20. Rebuilding the capture frequencies with 246, 252 and 260 overlaps gives three compatible inputs to the zero-truncated GIGP fit.
If the corresponding fitted totals are 1,590, 1,545 and 1,490, the audit shows that blocking uncertainty can move the total by 100 units. These are illustrative values, not a shortcut formula. Refit every scenario because repaired links alter the full observed-count table.
| Added missed links | Illustrative overlap | Illustrative fitted total |
|---|---|---|
| 6 | 246 | 1,590 |
| 12 | 252 | 1,545 |
| 20 | 260 | 1,490 |
CHAPTER 8 OF 12 · TRANSFER AND GOVERN
8. Changed-context transfer: patient registers to species lists
In a patient register, missed links may arise from changed names or addresses. In a biodiversity inventory, they may arise from noisy images, tag loss or movement between sites. The stable problem is a candidate generator that excludes most pairs and a finite review budget.
Transfer succeeds when the audit frame, unit and duplicate rules are rebuilt. A species encounter can legitimately repeat, while a person-level register may require one-to-one matching within the target period.
CHAPTER 9 OF 12 · TRANSFER AND GOVERN
9. Choose a stopping rule before review begins
Review can continue indefinitely near an ambiguous threshold. Stop when the uncertainty around recall or the population decision is small enough for its purpose, when the expected value of more review falls below its cost, or when a pre-specified budget is exhausted.
Monitor whether new labels still change calibration, blocking rules or the GIGP sensitivity range. Do not stop merely because the latest batch supports the preferred result.
CHAPTER 10 OF 12 · TRANSFER AND GOVERN
10. Limits and common misconceptions
An enriched audit is not automatically representative. High reviewer agreement does not prove the identifiers are sufficient, and a narrow confidence interval can exclude systematic misses created by the search frame itself.
Blocking-recall correction also does not repair source dependence, migration, frame undercoverage or a misspecified GIGP mixture. Keep the audit conclusion tied to the linkage mechanism it actually observes.
CHAPTER 11 OF 12 · CHECK AND CONTINUE
11. Questions linkage teams often ask
Must reviewers inspect obvious nonmatches? A probability sample needs some evidence from the low or excluded space, but efficient enrichment can reduce waste while preserving weights. Should the review label feed the matcher? It may, provided evaluation remains separate or uses honest cross-validation.
What if clerical truth is not reliable? Use repeated review, adjudication and sensitivity to disputed labels. The final report should distinguish model uncertainty, sampling uncertainty and truth-label uncertainty.
CHAPTER 12 OF 12 · CHECK AND CONTINUE
12. Evidence, specialist routes and the next check
The design-based foundation is Design-Based Estimation with Record-Linked Administrative Files and a Clerical Review Sample. A Simple Sampling Method for Estimating the Accuracy of Large Scale Record Linkage Projects samples pairs above and below the acceptance cut-off, while Designing an Ambiguity-Aware Clerical Review Strategy makes workload, precision and ambiguity coverage explicit.
Continue with blocking-scheme and duplicate-constraint sensitivity, match-uncertainty propagation, and dual-frame calibration. The next practical check is to sample one excluded-pair stratum and refit the total with design weights.
Contents · Previous chapter · Return to the Probability root