Find your next learning step
Two-phase soil sampling measures inexpensive auxiliary information widely, then collects costly laboratory or field truth on a probability subsample. The verification data can correct a map efficiently, provided the inclusion design, residual model and limits of double robustness remain visible.
Open the chapter contents · Soil Science root · Gold Standard of Education · How X Works
Chapter contents
Choose the route closest to the question, or read from Chapter 1 for the complete argument. The teaching chapters remain open; these grouped routes form the index.
Route 1 · Design the two phases
Route 2 · Work the correction ledger
Route 3 · Build robust map checks
Route 4 · Check limits and evidence
Route 5 · Ask and continue
CHAPTER 1 OF 12 · DESIGN THE TWO PHASES
1. Why soil mapping benefits from two phases
Collecting terrain, satellite, land-use and legacy-map variables across a region is often inexpensive compared with laboratory chemistry, hydraulic measurements or difficult field access. Two-phase sampling uses that difference deliberately: phase one builds the wide auxiliary frame, and phase two selects a probability subsample for costly verification.
The design is more than “train on some points and predict the rest.” Every second-phase inclusion probability belongs in the analysis. Verification sites are evidence about map error across the phase-one frame, not merely extra rows in a machine-learning table.
This article extends the Soil Science root, spatially balanced sampling and informative nonresponse. It asks how a designed verification subsample can correct a map without pretending that model predictions are ground truth.
CHAPTER 2 OF 12 · DESIGN THE TWO PHASES
2. Define the frame, target and verification variable first
Phase one needs a finite frame or a clearly defined spatial population: pixels, field segments, grid cells or management units. Record auxiliary variables for every unit and state the target estimand, such as regional mean carbon, area above a threshold or a corrected cell-level surface.
Phase two measures the target variable or a higher-quality reference on selected units. The inclusion probability πᵢ must be positive wherever the target population has support. Units with πᵢ = 0 cannot be repaired by weighting; this is a positivity failure.
Keep three quantities separate: the phase-one map prediction mᵢ, the verified value yᵢ, and the residual eᵢ = yᵢ−mᵢ. A verification design can target residual uncertainty while still preserving probability-based inference.
CHAPTER 3 OF 12 · WORK THE CORRECTION LEDGER
3. Build strata around expected map error
Useful phase-one strata may combine predicted property level, terrain, accessibility, land use and a measure of extrapolation. The purpose is to place verification effort where errors or decisions may differ, not to make equal-looking groups.
Within every stratum h, record the population size Nₕ, selected sample nₕ and inclusion probability nₕ/Nₕ for simple random sampling. Spatially balanced selection within strata can improve geographic spread while the inclusion design remains explicit.
Oversample rare, high-risk strata when useful. The analysis then restores population representation with weights. An unweighted mean of the verification sites answers the composition of the sample, not the composition of the region.
CHAPTER 4 OF 12 · WORK THE CORRECTION LEDGER
4. Worked four-stratum correction ledger
Take a phase-one frame of 100 cells with a baseline map mean of 6.40 units. A twenty-cell verification sample is allocated across four strata. Mean verified residuals are 0.2, −0.4, 1.0 and 1.8.
The design-weighted residual correction is ΣNₕēₕ/ΣNₕ = {30(0.2)+25(−0.4)+25(1.0)+20(1.8)}/100 = 0.57. The corrected regional mean is 6.40+0.57 = 6.97.
| Stratum | Frame cells Nₕ | Verified nₕ | Mean residual ēₕ | Weighted contribution Nₕēₕ/100 |
|---|---|---|---|---|
| Low prediction, stable access | 30 | 6 | 0.2 | 0.060 |
| High prediction, stable access | 25 | 5 | −0.4 | −0.100 |
| Low prediction, difficult access | 25 | 5 | 1.0 | 0.250 |
| High prediction, difficult access | 20 | 4 | 1.8 | 0.360 |
| Total | 100 | 20 | — | 0.570 |
The last two strata reverse the comforting conclusion from accessible sites. This is why phase-one accessibility information belongs in the design and correction.
CHAPTER 5 OF 12 · WORK THE CORRECTION LEDGER
5. Design uncertainty belongs beside the corrected mean
Suppose stratum residual variances are 0.36, 0.64, 1.00 and 1.44. For simple random sampling within strata, an illustrative variance estimator for the weighted residual mean is ΣWₕ²(1−fₕ)sₕ²/nₕ, where Wₕ=Nₕ/N and fₕ=nₕ/Nₕ.
| Stratum | Wₕ | fₕ | Residual variance sₕ² | Variance contribution |
|---|---|---|---|---|
| Low, stable | 0.30 | 0.20 | 0.36 | 0.00432 |
| High, stable | 0.25 | 0.20 | 0.64 | 0.00640 |
| Low, difficult | 0.25 | 0.20 | 1.00 | 0.01000 |
| High, difficult | 0.20 | 0.20 | 1.44 | 0.01152 |
| Total | — | — | — | 0.03224 |
The standard error is √0.03224 = 0.180. A simple normal interval for the corrected mean is about 6.97 ± 1.96(0.180), or 6.62 to 7.32. Spatial dependence and a more complex selection design may require replication weights or a design-specific variance estimator instead.
CHAPTER 6 OF 12 · BUILD ROBUST MAP CHECKS
6. Difference estimation uses the map without surrendering design control
The corrected mean is a difference estimator: take the known phase-one average prediction, then add a probability-weighted estimate of average residual. When the map predicts well, residuals vary less than raw measurements, improving precision.
Design protection comes from the weighted residual term. Even if the map is biased, a valid probability sample can estimate the average error over the frame. This does not make every cell prediction correct; it protects the specified population summary under the sampling design.
For a corrected surface, model residuals using the verification data and add predicted residuals to the phase-one map. Keep a separate design-based regional estimate as an anchor. A visually smooth corrected map should reconcile with the population total it claims to represent.
CHAPTER 7 OF 12 · BUILD ROBUST MAP CHECKS
7. Double-robust correction combines two working models
The standard term is doubly robust. One model describes the probability that a unit receives verification or yields a usable observation. The other predicts the verified soil outcome or residual from phase-one variables.
An augmented inverse-probability estimator combines weighted observed residuals with predictions for all units. Under its regularity conditions, it can remain consistent if either the verification/response model or the outcome model is correctly specified, even when the other is wrong.
“Either model” does not mean “any analysis.” The estimator still needs positivity, correct target definition, compatible data, and a valid treatment of spatial or design dependence. If both working models are wrong in the same region, double robustness offers no rescue.
CHAPTER 8 OF 12 · BUILD ROBUST MAP CHECKS
8. Changed context: access failure appears after selection
Suppose all twenty verification sites were selected with known probabilities, but steep wet slopes fail more often in the field. The realised observations are now a second sample within the planned phase-two sample.
Model response using variables available for every selected site, preserve the original phase-two weight and multiply by an appropriate response adjustment. Then use the outcome model to augment missing verified values. Run a sensitivity analysis for residual dependence on the unobserved soil value.
This is the bridge to preferential sampling and informative nonresponse. Planned inclusion, realised access and spatial prediction are different mechanisms and need separate ledgers.
Map correction needs spatial transfer checks
Partition verification points in spatial blocks, not random rows, when evaluating map transfer. Random cross-validation leaks nearby information and can make residual correction look more stable than it is.
Map residuals, weights, leverage and area of applicability. A correction driven by a few high-weight sites should be reported with that dependence. Compare regional totals from the corrected map with design-weighted estimates and investigate material disagreement.
Use spatial cross-validation and buffers and area-of-applicability masks to mark where the residual model transfers. An uncertainty map should widen outside the verified covariate domain.
CHAPTER 9 OF 12 · CHECK LIMITS AND EVIDENCE
9. Limits and common misconceptions
Two-phase sampling is not the same as convenience validation. Without known or defensible inclusion probabilities, design weights cannot reconstruct the frame.
More phase-one pixels do not substitute for phase-two truth. Auxiliary coverage can be nearly complete while the laboratory sample remains too small for rare domains or flexible residual models.
Double robustness is not triple certainty. It does not protect against positivity violations, measurement error in the reference method, an incorrectly defined target population, interference between units or both working models being wrong.
CHAPTER 10 OF 12 · CHECK LIMITS AND EVIDENCE
10. Evidence and reproducibility route
Neyman’s Contribution to the Theory of Sampling Human Populations established foundational probability-sampling and two-phase ideas. Bang and Robins formalised doubly robust estimation for missing-data and causal-inference settings in Doubly Robust Estimation in Missing Data and Causal Inference Models.
Archive the phase-one frame, target definition, strata, selection algorithm, inclusion probabilities, replacement rules, field response, laboratory quality control, outcome model, verification model and variance method. Preserve site identifiers and coordinate privacy controls consistently.
Recalculate the 0.57 residual correction and 0.03224 variance from the four-stratum ledger. A second implementation should recover the same weighted totals before a complex spatial model is accepted.
CHAPTER 11 OF 12 · ASK AND CONTINUE
11. Frequently asked questions
Why not verify the twenty most uncertain pixels?
That may help debugging, but without positive known inclusion probabilities it does not support design-based correction for the whole frame.
Does the verification sample have to be equal in every stratum?
No. Allocate for precision, rare domains and cost, then analyse with the actual design weights.
What is corrected: the map or the regional mean?
They are separate outputs. A design-weighted residual can protect a regional mean; a corrected surface additionally needs a transferable residual model.
When is an estimator doubly robust?
Under stated assumptions, it is consistent when either the response/inclusion model or the outcome model is correct. The claim must be tied to the exact estimator and target.
CHAPTER 12 OF 12 · ASK AND CONTINUE
12. Final transfer check and connected reading
Rebuild the four-stratum correction from Nₕ and ēₕ, then compute its standard error and interval. Move half of the difficult-access verification sites into nonresponse and explain which probabilities, predictions and sensitivity parameters must change.
Continue through the Soil Science root, spatially balanced sampling, preferential sampling, informative nonresponse and area-of-applicability masks. Together they distinguish planned verification, failed observation and predictive transfer.
Top · Contents · Previous · How X Works