Find your next learning step
A ratio-guided hybrid policy sends easy histories to greedy selection and difficult histories to lookahead. Before deployment, the gate itself needs evaluation: it may route the wrong stratum, rely on sparse logged actions or look strong only because historical operators preferred easier cases. Off-policy evaluation estimates how the proposed hybrid would have performed using logged audits, while support diagnostics and conservative bounds define where deployment is justified. The safe answer can be selective deployment, not an all-or-nothing launch.
Open the chapter contents · Remote Sensing root · Gold Standard of Education · How X Works
Chapter contents
Choose the route closest to the decision, or read from Chapter 1. The chapters remain open; these grouped routes are a navigable index.
Route 1 · Define the decision
Route 2 · Work the ledgers
Route 3 · Transfer the method
Route 4 · Check limits and evidence
Route 5 · Ask and continue
CHAPTER 1 OF 12 · DEFINE THE DECISION
1. Treat the gate as part of the policy
The gate, greedy branch, lookahead branch and budget-transfer rule jointly determine value. Evaluating only the branch algorithms while treating routing as fixed misses the most consequential source of error.
Define the target policy completely: features available at decision time, route probabilities, actions within each branch and what happens when information is missing.
CHAPTER 2 OF 12 · DEFINE THE DECISION
2. Understand what logged audits can identify
Off-policy evaluation requires overlap between actions taken historically and actions the target policy would take. If rare flood histories never received field checks, logs cannot reliably estimate a policy that sends all such histories to the field.
Show support by stratum and action. A large overall log can still provide almost no evidence for a narrow high-consequence route.
CHAPTER 3 OF 12 · WORK THE LEDGERS
3. A routing-support ledger
Historical audits came from a mixed human policy. The proposed gate is compared with logged action coverage before value estimation.
| Stratum | Logged histories | Target lookahead rate | Effective supported cases | Support decision |
|---|---|---|---|---|
| Clear agriculture | 2,400 | 0.08 | 1,980 | Adequate |
| Cloud-edge forest | 820 | 0.74 | 390 | Monitor |
| Urban boundary | 1,050 | 0.36 | 610 | Adequate |
| Rare flood onset | 140 | 0.91 | 28 | Insufficient |
Effective support reflects propensity weights and action overlap, not the raw number of scenes alone.
CHAPTER 4 OF 12 · WORK THE LEDGERS
4. Worked doubly robust policy comparison
A doubly robust estimator combines a model-based outcome prediction with propensity-weighted residual correction. It can remain consistent if one of those components is correctly specified under the method’s assumptions, but it is not immune to poor overlap.
| Policy | Estimated utility | Standard error | Conservative lower bound | Deployment result |
|---|---|---|---|---|
| Historical baseline | 6.10 | 0.12 | 5.86 | Reference |
| All greedy | 6.22 | 0.18 | 5.87 | No clear gain |
| All lookahead | 6.58 | 0.41 | 5.78 | Too uncertain |
| Selective hybrid | 6.49 | 0.20 | 6.09 | Pilot supported strata |
Use an interval procedure appropriate to the data dependence and selection process. The displayed subtraction is a teaching example, not a universal safety certificate.
CHAPTER 5 OF 12 · WORK THE LEDGERS
5. Cross-fit nuisance models and audit weights
Estimate propensities and outcome models on folds separate from the records they score. Cross-fitting reduces overfitting bias but does not repair omitted confounding or actions absent from the logs.
Inspect extreme weights, effective sample size and stratum balance. Weight clipping trades variance for bias and must be reported with sensitivity results.
CHAPTER 6 OF 12 · TRANSFER THE METHOD
6. Deploy selectively with a fallback
Activate the hybrid only in strata with adequate support and a conservative value result. Route unsupported histories to the historical baseline, expert review or a controlled data-collection protocol.
Specify rollback triggers, budget caps and a shadow period in which recommendations are logged but not executed. Safe deployment is a state machine, not a one-time approval.
CHAPTER 7 OF 12 · TRANSFER THE METHOD
7. Changed-context transfer: coastal bloom audits
Transfer the evaluation to coastal bloom confirmation, where vessel sampling historically favoured accessible waters. Propensity models must account for access, weather and tide using information available at the original decision time.
The transfer check withholds offshore or storm-linked strata lacking overlap, even if the global hybrid estimate is positive. New coverage is collected through a governed pilot rather than extrapolation.
CHAPTER 8 OF 12 · TRANSFER THE METHOD
8. Connect hybrid routing, ratios and lookahead
Use the hybrid-routing guide to define the target policy. The ratio-diagnostics guide supplies routing evidence.
The lookahead guide defines the expensive branch and its reference-collection costs.
CHAPTER 9 OF 12 · CHECK LIMITS AND EVIDENCE
9. Limits and common misconceptions
Doubly robust does not mean robust to every failure. If both nuisance models are wrong, overlap is weak or unobserved factors drive historical actions and outcomes, the estimate can still be biased.
A positive global value does not justify every route. Local support and consequence matter, and selective deployment may be the only evidence-aligned claim.
CHAPTER 10 OF 12 · CHECK LIMITS AND EVIDENCE
10. Practical evidence checklist
Archive the target and logging policies, gate features, propensities, outcomes, fold assignments, support tables, weight diagnostics, effective sample sizes, uncertainty method, thresholds and fallback policy.
During shadow and pilot phases, log route recommendations, actual routes, costs, delays and outcomes. Re-estimate value on untouched periods and review any stratum where route frequency drifts.
CHAPTER 11 OF 12 · ASK AND CONTINUE
11. Frequently asked questions
Can we evaluate an action never taken historically?
Not from those logs alone. Use a controlled pilot, a defensible structural model or keep the baseline in that region.
Should extreme importance weights be deleted?
They diagnose poor overlap. Clipping may stabilise estimation but introduces bias, so report the rule and a sensitivity analysis.
When is full deployment appropriate?
Only after every intended stratum has adequate support, conservative value and monitored operational safeguards; otherwise deploy selectively.
CHAPTER 12 OF 12 · ASK AND CONTINUE
12. Sources and next reading
Jiang and Li develop doubly robust off-policy value evaluation. Thomas, Theocharous and Ghavamzadeh propose high-confidence off-policy evaluation. Kallus and colleagues study doubly robust distributionally robust evaluation and learning. The routing ledgers here are original teaching examples.
Continue through the Remote Sensing root, the Gold Standard of Education and How X Works.
Top · Contents · Previous · How X Works