GOLD STANDARD · RADAR · HOW X WORKS
Find the decision hidden inside the model
Radar truth collection can fail after selection, after field deployment or at adjudication. A gold-standard conditional-extremes CFAR validation records each response stage, calibrates the observed truth sample to known operational totals and shows how reference dropout could move the false-alarm conclusion.
This guide connects the Gold Standard of Radar, Gold Standard of Education, and How X Works hub.
Chapter contents and reading routes
Choose the route closest to your present task. Every chapter remains expanded, so the article also reads naturally from start to finish.
Orient multistage nonresponse
Build calibration weights
Bound reference dropout
Transfer and govern
Use evidence and next routes
CHAPTER 1 OF 12 · ORIENT TRUTH LOSS
1. Selected for truth is not the same as truth observed
A phase-two unit may be selected because its clutter regime or detector score is informative, yet never receive a usable reference label. Access can fail, a sensor can drop out, an image can be ambiguous or an adjudication panel can remain divided.
Treat selection, contact, measurement and adjudication as separate stages. Combining them into one response flag hides whether a repair needs field logistics, sensor redundancy or a clearer truth definition.
CHAPTER 2 OF 12 · ORIENT TRUTH LOSS
2. Define the validation estimand before weighting
The target may be false-alarm probability across all operational cells, tail calibration within supported covariate regimes or detection performance at a declared threshold. The denominator and exposure unit must be explicit.
A weighted respondent estimate answers the target only if the response model and auxiliary totals connect respondents to the full operational population. Weighting cannot repair a reference standard that measures a different event.
CHAPTER 3 OF 12 · BUILD THE WEIGHTS
3. Start with design and response probabilities
The base weight is the inverse phase-two selection probability. Multiply it by inverse probabilities for later response stages when those probabilities are estimated from information available for all selected units.
Keep probability factors and trimming decisions in the audit trail. A final weight alone cannot reveal whether instability arose from rare design strata, field nonresponse or reference dropout.
CHAPTER 4 OF 12 · BUILD THE WEIGHTS
4. Work through a two-stage response example
Suppose 2,000 operational cells are represented by 200 selected truth cases. In a difficult coastal regime, 40 cases were selected, 30 produced reference data and 24 reached final adjudication. The adjudication response among measured cases is 24/30 = 0.80.
If each selected coastal case initially represents 10 cells, a simple final-stage weight is 10/(0.75 × 0.80) = 16.67, where 0.75 is measurement response. The 24 adjudicated cases then represent about 400 cells, but only under conditional response assumptions.
| Stage | Cases remaining | Stage response | Cumulative response |
|---|---|---|---|
| Selected | 40 | 1.00 | 1.00 |
| Reference measured | 30 | 0.75 | 0.75 |
| Adjudicated | 24 | 0.80 | 0.60 |
CHAPTER 5 OF 12 · BUILD THE WEIGHTS
5. Calibrate weights to operational totals
Known totals for range band, sea state, scan mode or detector-score group can anchor the respondent sample to the operational frame. Calibration chooses adjusted weights that reproduce those totals while staying near the base nonresponse weights.
Use variables observed consistently for respondents and nonrespondents and related to response or the outcome. Adding many sparse margins can create extreme weights and apparent precision without genuine information.
CHAPTER 6 OF 12 · BOUND DROPOUT
6. Diagnose weight concentration and residual imbalance
Report weight ranges, effective sample size, influential units and balance before and after calibration. Inspect tail outcomes by response stage because a balanced covariate table can coexist with outcome-dependent dropout.
If trimming is required, show the estimand change or bias sensitivity it creates. Trimming protects variance; it does not make the rare, high-weight regime representative.
CHAPTER 7 OF 12 · BOUND DROPOUT
7. Bound what missing reference labels could change
Let 120 adjudicated cases contain 9 false alarms and 20 unresolved reference cases remain. The observed false-alarm fraction is 7.5%. If none of the unresolved cases are false alarms, the all-case fraction is 9/140 = 6.4%; if all are, it is 29/140 = 20.7%.
Those worst-case bounds are often wide. Tighten them only with credible restrictions, such as audited error rates by reason for dropout. If a release decision changes inside the allowed range, obtain additional truth or use a conservative threshold.
| Unresolved-case assumption | False alarms | Rate among 140 |
|---|---|---|
| None are false alarms | 9 | 6.4% |
| Same as respondents | about 11 | about 7.9% |
| All are false alarms | 29 | 20.7% |
CHAPTER 8 OF 12 · TRANSFER AND GOVERN
8. Changed-context transfer: medical test verification
Move from radar truth to disease verification. The stable structure is two-phase selection, multistage response, a costly reference standard and verification bias. Patient consent, clinical pathways and harms change.
Transfer succeeds when every response stage and reference definition is rebuilt. Radar range bands cannot simply become age bands; the auxiliary variables must explain clinical selection, response or outcome in the new setting.
CHAPTER 9 OF 12 · TRANSFER AND GOVERN
9. Design follow-up by uncertainty and decision value
Recontact or remeasure cases that drive the bound, not merely the easiest missing labels. Preserve positive follow-up probability in every critical operational regime so later inference remains possible.
A targeted wave should record its selection probability and cost. Otherwise adaptive recovery creates a new convenience sample layered on top of the first.
CHAPTER 10 OF 12 · TRANSFER AND GOVERN
10. Limits and common misconceptions
Calibration reproduces chosen margins, not unobserved truth. Missing at random is an assumption conditional on the recorded variables, not a property conferred by software. Reference labels can also be wrong among respondents.
A narrow weighted interval can coexist with wide nonresponse bounds. Present both. Sampling variance describes repeated designs under a model; sensitivity bounds describe movement under allowed departures from the response assumption.
CHAPTER 11 OF 12 · CHECK AND CONTINUE
11. Questions validation teams often ask
Model response in stages when different mechanisms and remedies operate at selection, measurement and adjudication. Combine stages only when their predictors and consequences are genuinely indistinguishable.
Can calibration replace inverse-response modelling?
Calibration may reduce bias and variance when good auxiliary totals exist, but it still relies on a link between those variables, response and the target outcome.
When is dropout small enough to ignore?
Judge its weighted share and decision impact, not its raw count. A few missing cases in a rare extreme regime can dominate a tail-performance conclusion.
CHAPTER 12 OF 12 · CHECK AND CONTINUE
12. Evidence, specialist routes and the next check
research on two-phase nonresponse follow-up shows how later subsampling can recover information while preserving design probabilities. two-phase validation methods explain weighted analysis with selectively observed truth, while the FDA diagnostic-test reporting guidance discusses verification and reference-standard bias.
Continue through adaptive phase-two truth allocation, two-phase verification-bias correction, and reference-standard error and adjudication. The next check is to publish the response tree, calibration margins and dropout bounds beside the CFAR estimate.