Two forecasts can assign different probabilities to exactly the same rain event and still each be calibrated. The reason is that each forecast may sort the possible cases using different information. Calibration within one sorting does not require agreement with a forecast using another sorting.
The short answer is to inspect the joint record. Compare the event definition and issue time first; then cross-tabulate both probabilities and the outcome. Averaging the two probabilities creates a third forecasting rule. Its calibration and usefulness need their own checks.
This article constructs an original forty-ticket probability model, derives both forecasts from it, and checks their average, scores and information limits. Every number is fictional. The rain event is a mathematical label, not a current forecast or an activity-safety rule. The Weather Forecasting subject guide supplies the wider observation-to-prediction chain.
eduKateSG · Weather Forecasting
Find your next learning step
Trace two information sets, check their shared outcome table, and evaluate a combined forecast on its own terms.
ROUTE 1 · CHAPTERS 1–5
Define the event and information
Build a complete ticket model and verify each component forecast.
ROUTE 2 · CHAPTERS 6–10
Inspect joint calibration
Compare conditional risks, disagreement and the averaged probability.
ROUTE 3 · CHAPTERS 11–15
Work scores and information gains
Reconstruct squared-error identities and distinguish expectation from one outcome.
ROUTE 4 · CHAPTERS 16–20
Transfer and verify
Change the case mix, retain dependence and design independent evaluation.
ROUTE 5 · CHAPTERS 21–27
Practise and follow the evidence
Test a changed table, identify missing joint structure and reproduce the ticket audit.
Chapter contents
Define the event and information · Chapters 1–5
1. Hold the predicted event fixed
2. Build a complete forty-ticket world
3. Give the first forecaster only A
Inspect joint calibration · Chapters 6–10
6. Distinguish calibration from complete information
7. Check disagreement after conditioning on the other forecast
8. Form the average and identify its groups
Work scores and information gains · Chapters 11–15
11. Compare P, Q and their average
12. Derive the averaging score identity
13. Test whether equal weighting is best
14. Recover the value of additional information
15. Separate better expectation from a better single outcome
Transfer and verify · Chapters 16–20
16. Change the case mix deliberately
17. Preserve joint information when combining forecasts
18. Do not multiply evidence without a dependence assumption
Practise and follow the evidence · Chapters 21–27
21. Repair, stabilise and extend the learning
22. Try a changed table before reading the answer
23. Ask what separate reliability plots cannot identify
24. Locate the stronger agreement requirement
25. Reconstruct the result with an independent ticket audit
CHAPTER 1 OF 27 · Define the event and information
1. Hold the predicted event fixed
Write one event sentence before comparing probabilities: rain above a specified teaching threshold, at one specified place, during one specified interval. Let Y equal one if that event occurs and zero otherwise. Both forecasts in this lesson concern that same Y.
A forecast for rain somewhere in a district and a forecast for rain at one station need not agree because they predict different events. An evening forecast and a later nowcast may have different evidence. Those are useful questions, but they are not the puzzle being isolated here.
We therefore hold the target, valid period, observation rule and nominal issue time fixed. The forecasters differ only in which signal they see. Their signals are named A and B, each with a low and a high category. These names do not claim that any real weather variable has only two states.
This boundary keeps the reasoning clean. If disagreement remains after the event contract is aligned, examine the information and probability model rather than immediately declaring that one forecast must be unreliable.
CHAPTER 2 OF 27 · Define the event and information
2. Build a complete forty-ticket world
Imagine forty equally likely tickets. Each carries an A label, a B label and an outcome Y. There are ten tickets in each of four A–B combinations. In the low–low combination, one ticket has Y = 1. The corresponding event counts in low–high, high–low and high–high are five, three and nine.
| A category | B category | Tickets | Event tickets | Event fraction within the cell |
|---|---|---|---|---|
| Low | Low | 10 | 1 | 0.10 |
| Low | High | 10 | 5 | 0.50 |
| High | Low | 10 | 3 | 0.30 |
| High | High | 10 | 9 | 0.90 |
Choose a ticket uniformly. That choice defines an exact probability distribution; the table is not an estimate obtained from forty operational forecasts. If we instead collected forty real observations, their observed fractions would have sampling uncertainty and would not establish these as known population rates.
There are eighteen event tickets, so the unconditional event probability is 18/40 = 0.45. Keep both the four cell fractions and that overall fraction. They answer different conditioning questions and will let us reconstruct every subsequent calculation.
CHAPTER 3 OF 27 · Define the event and information
3. Give the first forecaster only A
Forecaster P sees A but not B. When A is low, twenty tickets remain possible. Six carry the event: one from the low–low cell and five from low–high. P therefore reports 6/20 = 0.30.
When A is high, the event count is three plus nine, or twelve among twenty. P reports 0.60. Its complete forecasting rule is consequently P = 0.30 for low A and P = 0.60 for high A.
Nothing has been averaged across the wrong denominator. P is using the event fraction among all tickets compatible with the information it receives. Within each A category, the two B categories are equally represented in this particular world.
A learner can check the rule without knowing a formula for conditional expectation. Cover the B column, gather tickets sharing the visible A label, and divide event tickets by all remaining tickets. The symbolic notation P = E[Y given A] is a compact description of that operation.
CHAPTER 4 OF 27 · Define the event and information
4. Give the second forecaster only B
Forecaster Q sees B but not A. For low B, four of twenty tickets carry the event, giving Q = 0.20. For high B, fourteen of twenty carry it, giving Q = 0.70.
Thus the same low-A, high-B ticket receives P = 0.30 and Q = 0.70. Their forty-percentage-point disagreement is substantial. It is also fully consistent with the single complete probability table we specified.
P groups tickets by rows, while Q groups them by columns. A ticket can belong to a relatively low-risk row and a relatively high-risk column at once. Neither grouping changes the ticket’s eventual outcome; each changes the collection against which its probability is calculated.
Notice what the example does not assume. P and Q are not noisy reports of one already-known cell fraction. Each is the exact conditional event probability for its own supplied information. Disagreement arises even before estimation error or imperfect meteorological models are introduced.
CHAPTER 5 OF 27 · Define the event and information
5. Verify calibration separately for both forecasts
For this binary problem, calibration means E[Y given P] = P. Among tickets receiving a particular probability, the event fraction equals that probability. The equality refers to the distribution, not to a promise about an individual draw.
P’s 0.30 group contains six events in twenty tickets; its 0.60 group contains twelve in twenty. Q’s 0.20 group contains four in twenty; its 0.70 group contains fourteen in twenty. Both forecasts satisfy the definition exactly in the constructed world.
Their average probabilities also equal the overall event rate. For P, (0.30 + 0.60)/2 = 0.45. For Q, (0.20 + 0.70)/2 = 0.45. This overall agreement is necessary here, but it is weaker than the group-by-group checks.
Do not reverse that implication. A forecast can average to the right event rate while giving the wrong conditional probabilities. We will shortly construct precisely that situation by averaging P and Q ticket by ticket.
CHAPTER 6 OF 27 · Inspect joint calibration
6. Distinguish calibration from complete information
The low–high cell has an event fraction of 0.50. P reports 0.30 there because it cannot separate that cell from low–low. Q reports 0.70 because it cannot separate it from high–high. Both leave potentially useful distinctions unresolved.
Define R as the forecast that sees both A and B. It reports the four cell probabilities 0.10, 0.50, 0.30 and 0.90. In this model, R uses all the signal information we have specified.
That is not the same as knowing Y. Even R = 0.90 leaves one non-event ticket among ten in its cell. The outcome still has uncertainty after the signals are observed. Perfect use of a stated information set should not be confused with perfect prediction of every realization.
This distinction matters when someone calls a probability the true forecast. Ask true conditional on what information and under which model. P, Q and R are different conditional probabilities inside one coherent construction, rather than three incompatible claims about a single unconditioned fraction.
CHAPTER 7 OF 27 · Inspect joint calibration
7. Check disagreement after conditioning on the other forecast
Take only P’s 0.30 group. Q divides it into two equally sized parts. In the Q = 0.20 part, the event fraction is 0.10. In the Q = 0.70 part, it is 0.50. Those fractions average to P’s 0.30, but neither equals it.
Therefore E[Y given P,Q] is not generally P. The other forecast reveals additional structure within P’s probability group. The corresponding stronger check also fails for Q: within Q = 0.20, the two P groups have event fractions 0.10 and 0.30.
Researchers study such questions under cross-calibration, with definitions appropriate to different forecast formats. Here the exact binary condition being tested is visible in the table. We do not infer a general distributional result from a reliability curve alone.
The practical question is whether another available prediction helps explain residual outcome differences inside your forecast groups. Answering it requires paired records. Two separate reliability plots discard the pairing needed for this particular check.
For a reproducible joint check, retain the two original probabilities even when a dashboard displays only their average. A reviewer needs to distinguish a forecast pair that was actually issued from a pair reconstructed later. Preserve timestamps for both records; otherwise an apparent gain from combining them may depend on information that arrived after the required decision.
CHAPTER 8 OF 27 · Inspect joint calibration
8. Form the average and identify its groups
Let M = (P + Q)/2. In table order, its four values are 0.25, 0.50, 0.40 and 0.65. Each value identifies one cell because these four averages are distinct.
The event fractions among those groups remain 0.10, 0.50, 0.30 and 0.90. Averaging probabilities has not changed the ticket outcomes. Consequently the 0.25 group occurs at 0.10, the 0.40 group at 0.30, and the 0.65 group at 0.90.
M is not calibrated in this model. It is correct in its 0.50 group and wrong in the other three. This is a direct counterexample to the assertion that the average of calibrated forecasts must itself be calibrated.
The wording must stay proportionate. We have shown that averaging can lose calibration, not that every average necessarily does. If two calibrated forecasts agree everywhere, averaging them changes nothing. Combination is a new forecasting method whose properties depend on the joint distribution.
CHAPTER 9 OF 27 · Inspect joint calibration
9. Explain why the right overall mean does not rescue it
M’s mean is (0.25 + 0.50 + 0.40 + 0.65)/4 = 0.45. That matches the overall event probability exactly. A report comparing only the mean forecast with the total event fraction would miss the conditional mismatches.
The errors of M relative to the cell probabilities are +0.15, zero, +0.10 and −0.25. They sum to zero under equal cell weights. Positive and negative conditional errors compensate in the aggregate.
This does not make the overall comparison useless. It remains a valid check of one property, often described as calibration in the large or mean agreement. Its limitation is that it cannot establish the stronger group-by-group equality we defined earlier.
A useful teaching repair is to ask which table columns were retained. A single total preserves the average but removes the pairing between forecast level and outcome frequency. Restoring those groups reveals the problem without changing any observations or accusing the arithmetic total of being incorrect.
CHAPTER 10 OF 27 · Inspect joint calibration
10. Calculate expected Brier scores from the cells
For a probability f and binary outcome Y, the squared error is (f − Y)². If the event probability in a cell is r, its expected value is r(1 − f)² + (1 − r)f².
Expand this expression to obtain r(1 − r) + (f − r)². The first term is the remaining outcome variability within that cell. The second is the penalty for reporting a probability different from its event fraction.
For R, f equals r in every cell, so its expected Brier score is the average of 0.09, 0.25, 0.21 and 0.09. The result is 0.16. The forty-ticket construction gives the same result by scoring each ticket individually and averaging.
This is an exact expectation under the declared model. It is not a fitted verification score for a real weather service, and it does not require us to pretend that a short archive demonstrates long-run calibration. Keep the model calculation separate from the empirical task of estimating comparable quantities.
CHAPTER 11 OF 27 · Work scores and information gains
11. Compare P, Q and their average
Because P is calibrated, its expected score can also be computed by averaging p(1 − p) over its own forecast groups. This gives (0.21 + 0.24)/2 = 0.225. For Q, the result is (0.16 + 0.21)/2 = 0.185.
For M, use its differences from R. The extra squared penalty is (0.15² + 0² + 0.10² + 0.25²)/4 = 0.02375. Adding the irreducible cell contribution 0.16 gives 0.18375.
M therefore has a lower expected Brier score than either P or Q in this example, even though M is not calibrated. There is no contradiction: a score combines aspects of forecast quality, while calibration is one specific conditional property.
The difference between M and Q is only 0.00125. It is exact in the fictional model, but a real finite archive would need an uncertainty analysis before supporting such a small performance claim. The model makes a logical distinction; it does not supply a practical significance threshold.
CHAPTER 12 OF 27 · Work scores and information gains
12. Derive the averaging score identity
For any particular outcome y and weight w between zero and one, expand the square of wP + (1 − w)Q − y. The result is w(P − y)² + (1 − w)(Q − y)² − w(1 − w)(P − Q)².
Average over cases to obtain the same identity for mean scores. The mixture score is no greater than the weighted mean of the two component scores because the last term is nonnegative and is subtracted.
That result does not say the mixture beats the better component. A weighted mean of two scores can lie above the smaller score by more than the disagreement correction removes. Nor does the identity certify calibration; it is an algebraic statement about squared error.
In our table, E[(P − Q)²] = (0.01 + 0.16 + 0.16 + 0.01)/4 = 0.085. At equal weights, the score is 0.205 − 0.25 × 0.085 = 0.18375, matching the independent cell calculation.
CHAPTER 13 OF 27 · Work scores and information gains
13. Test whether equal weighting is best
Keep the joint table fixed and write Fw = wP + (1 − w)Q. Substituting our component scores and disagreement gives expected score 0.185 − 0.045w + 0.085w².
Complete the square, or differentiate this quadratic. Its minimum over the allowed interval occurs at w = 0.045/0.17 = 9/34, approximately 0.265. The corresponding expected score is approximately 0.179044, lower than the equal-weight result.
This optimal weight belongs to our known teaching distribution. It is not a recommended weight for combining real forecast providers. Estimating a weight from historical observations introduces another learning problem, with overfitting and changing conditions to consider.
It also does not restore calibration automatically. The weighted rule gives approximately 0.226, 0.594, 0.306 and 0.674 in the four cells, whose event fractions remain 0.10, 0.50, 0.30 and 0.90. Optimizing one proper score within a restricted combination family need not recover the full conditional forecast.
CHAPTER 14 OF 27 · Work scores and information gains
14. Recover the value of additional information
R improves on P’s expected score by 0.225 − 0.16 = 0.065. Compute the average squared difference between R and P: (0.20² + 0.20² + 0.30² + 0.30²)/4 = 0.065.
Similarly, R improves on Q by 0.025. Its squared differences from Q are 0.10², 0.20², 0.10² and 0.20², averaging to 0.025. These equalities follow because P and Q are the appropriate coarser conditional probabilities of R.
The result gives a precise sense in which the additional signal is useful for expected squared probability error. It does not price the cost of collecting that signal, prove timely availability or settle the value of every downstream decision.
It also does not rank the usefulness of A and B universally. Under this table, B alone gives the lower expected Brier score. Change the cell probabilities or weights and that ranking may change. Information value is a property of a specified outcome, distribution and evaluation question.
CHAPTER 15 OF 27 · Work scores and information gains
15. Separate better expectation from a better single outcome
In the low-A, high-B cell, P = 0.30 and R = 0.50. If the sampled ticket is a non-event, P’s squared error is 0.09 while R’s is 0.25. The forecast using more information scores worse on that particular realization.
If an event ticket is drawn in the same cell, P scores 0.49 while R scores 0.25. The cell contains equally many event and non-event tickets, so the expected scores are 0.29 for P and 0.25 for R.
Thus an expected improvement does not require improvement on every ticket. Looking only at a memorable dry case can reverse the conclusion one would draw from the defined distribution. Conversely, one successful update does not prove that additional information is consistently being used well.
For a learning check, ask the student to calculate both outcomes before averaging them. This prevents the phrase better forecast from silently changing between a realized score, an expected score and a wider operational judgement.
Now construct a second population. Give high A probability 0.75 and low A probability 0.25, while B remains equally likely low or high within either A category. Keep all four conditional event fractions unchanged.
P’s row probabilities remain 0.30 and 0.60 because the within-row B mixture is unchanged. Q’s column probabilities change. For low B, the event probability is 0.25 × 0.10 + 0.75 × 0.30 = 0.25. For high B, it is 0.25 × 0.50 + 0.75 × 0.90 = 0.80.
The new overall event probability is 0.25 × 0.30 + 0.75 × 0.60 = 0.525. Q’s old 0.20 and 0.70 values are no longer calibrated to the new population, despite unchanged cell-level event mechanisms.
This is a controlled transfer exercise, not proof that real forecasting changes follow this pattern. It identifies the exact assumption that moved: the weights of possible A categories within Q’s information groups. A historical calibration claim must name the population in which it was checked.
CHAPTER 17 OF 27 · Transfer and verify
17. Preserve joint information when combining forecasts
In the original table, the pair (P,Q) identifies the cell and therefore R. A combination method with access to the pair and the known table can recover 0.10, 0.50, 0.30 or 0.90 directly. A plain arithmetic mean applies a much more restrictive rule.
In this particular example even M identifies the cell, because its four values are distinct. A correctly learnt mapping from M to the corresponding cell probability could therefore recover R. Averaging has not destroyed information here; the uncorrected numerical interpretation is the problem.
That observation has a boundary. In another distribution, two different forecast pairs can have the same average while having different conditional event risks. A correction using only the average cannot separate those pairs. Retaining both original probabilities allows an analyst to investigate that possibility.
Do not equate keeping two columns with having a validated combination model. The joint record preserves an opportunity to learn. Adequate observations, a suitable method and independent evaluation are still needed to turn that opportunity into dependable forecast performance.
CHAPTER 18 OF 27 · Transfer and verify
18. Do not multiply evidence without a dependence assumption
Calibration alone does not tell us whether the signals are conditionally independent given Y. In the original table, A and B are independent before observing Y because all four cells have equal weights. That does not make them independent within event tickets.
Among the eighteen event tickets, nine have both high A and high B. The conditional joint fraction is 9/18 = 1/2. The product of the separate high-A and high-B fractions is (12/18)(14/18) = 14/27, which is different.
A Bayesian combination formula that multiplies two likelihood ratios requires its relevant factorization assumptions. Using two calibrated probabilities as though they came from independent evidence can therefore produce another probability different from R.
The repair is to write the joint model or inspect a suitable joint archive. Do not infer evidence independence from different provider names, different signal labels or the fact that each separate reliability diagram looks satisfactory. Shared observations and model components can create further dependencies in real systems.
CHAPTER 19 OF 27 · Transfer and verify
19. Design an empirical check with the right records
A practical archive needs each original probability, issue time, valid period, event specification, location rule, forecast version and observed outcome. Keep both forecasts on the same eligible cases, including missingness flags. Record a combination rule before evaluating its future performance.
First examine each forecast’s reliability with meaningful sample counts. Next examine paired forecast groups, asking whether the second probability separates outcomes inside the first forecast’s groups. Compare the combined prediction with both components and a suitable baseline on matched cases.
Our exact four-cell table avoids estimation uncertainty by construction. Real probabilities are often nearly continuous, making exact pairs sparse. Grouping, smoothing or fitting a conditional model requires judgement and introduces uncertainty. A highly detailed display can hide very small denominators.
Consecutive times, overlapping intervals and nearby locations can share weather systems. The amount of independent evidence may be much smaller than the number of rows. A validation design should reflect those dependencies and retain changes in observing or forecasting systems rather than treating all rows as exchangeable.
CHAPTER 20 OF 27 · Transfer and verify
20. Recalibrate without learning from the final test
Suppose an analyst fits a correction from M to event frequencies. The original table says what an ideal correction would be in our teaching population; an operational analyst does not receive that table as known truth.
Use development data to choose the correction and preserve a later evaluation sample that did not determine it. Freeze the selected rule before applying it to that sample. Report performance for the combined rule itself, rather than inheriting the calibration labels of P and Q.
Choosing among many probability bins, model forms or weights after looking at the final outcomes can make the resulting verification too optimistic. Keep the sequence of decisions and the evaluation boundary visible. If a forecast system changes, a previously adequate correction may need a new assessment.
Calibration is also target-specific. A correction for one rain threshold at one lead does not automatically calibrate another threshold, a rainfall amount distribution or a joint event across locations. Preserve the forecast question when moving from a convenient example to an operational product.
CHAPTER 21 OF 27 · Practise and follow the evidence
21. Repair, stabilise and extend the learning
For a repair route, begin with the forty tickets and ask for the overall event fraction. Then ask for one row fraction and one column fraction. If these are confused, stay with visible counts before introducing expectations or scores.
For stabilisation, cover alternating columns and ask which probability can be calculated from the remaining information. The learner should explain why low A can coexist with high B and why P = 0.30 is not a claim that every B subgroup has event rate 0.30.
For extension, calculate M’s calibration errors and derive the averaging score identity. Then change the row weights while retaining the cell fractions. Ask which forecasts remain calibrated and require the answer to refer to the new conditional denominators.
A useful independent test gives a new four-cell table without naming the method. The learner must decide whether the task asks for a conditional fraction, a score, a combination rule or evidence of calibration. Selecting the right question is part of the understanding being assessed.
CHAPTER 22 OF 27 · Practise and follow the evidence
22. Try a changed table before reading the answer
Keep four equally likely cells, but replace their event fractions with 0.20, 0.40, 0.60 and 0.80 in the same order. Find the A-only forecast, the B-only forecast and their equally weighted average in every cell.
The A-only values are 0.30 and 0.70. The B-only values are 0.40 and 0.60. Their averages are 0.35, 0.45, 0.55 and 0.65. Each average again identifies a cell, whose event fractions are different from those averages. Both components are calibrated, while their average is not.
The cell-aware forecast has expected Brier score (0.16 + 0.24 + 0.24 + 0.16)/4 = 0.20. The average has an additional mean squared conditional error of 0.0125, giving 0.2125. Check those values independently rather than transferring the earlier score unchanged.
Finally change the target from rain to a fictional demand event. The conditional-probability arithmetic remains valid after a new model is specified, but the weather interpretation, observation process and available signals require replacement. Mathematical transfer preserves relationships, not unsupported domain claims.
CHAPTER 23 OF 27 · Practise and follow the evidence
23. Ask what separate reliability plots cannot identify
Suppose we are given only the original equal cell weights and the calibrated P and Q values. Their row and column sums constrain the cell event probabilities but do not determine them. Let the low–low event fraction be a. The other three fractions must then be 0.60 − a, 0.40 − a and 0.80 + a.
Every a between zero and 0.20 keeps all four fractions between zero and one. Every such table gives the same P values 0.30 and 0.60 and the same Q values 0.20 and 0.70. Our original table chooses a = 0.10, but it is not forced by those separate calibration statements.
For a = 0, the low–high cell has event fraction 0.60. For a = 0.20, it has event fraction 0.40. Both worlds preserve each component’s full probability-group reliability. A proposed joint probability of 0.50 therefore requires evidence beyond those separate plots.
This is an identification limit, not merely a warning about noisy data. Even exact knowledge of the marginal reliability relationships leaves the joint cell mechanism unresolved. Paired forecast–outcome observations can investigate that missing structure; unpaired summaries cannot recover it by more precise arithmetic.
CHAPTER 24 OF 27 · Practise and follow the evidence
24. Locate the stronger agreement requirement
Imagine that P and Q both claimed to equal E[Y given P,Q] on every forecast pair with positive probability. Within any such pair there is only one conditional event fraction. It cannot simultaneously equal two different numbers.
Therefore those stronger claims would imply P = Q except on events of probability zero. Our table contains positive-probability pairs with P different from Q, so both forecasts cannot satisfy that stronger joint-information requirement. Separate calibration makes no such claim and remains intact.
The phrase using the same information needs comparable care. If two rules both equal the conditional event probability under the very same information set and probability model, they agree almost surely. If they merely draw on similarly named datasets, their processing, models or actual available information can still differ.
For a practical explanation, state which level has been established. Each forecast being calibrated separately is one result. Neither retaining outcome information beyond the other is a stronger result. Both being ideal for a common full information set is stronger again. Moving between those statements without additional evidence creates an apparent contradiction that the definitions do not contain.
CHAPTER 25 OF 27 · Practise and follow the evidence
25. Reconstruct the result with an independent ticket audit
An independent checker can reproduce the model without any specialist forecasting software. Create four groups of ten numbered tickets. Mark respectively one, five, three and nine tickets as events. Assign the row probability to every ticket in its row and the column probability to every ticket in its column.
Next compute the average probability for each ticket. Group tickets by that average and count the marked events. The 0.25, 0.50, 0.40 and 0.65 groups should reveal event fractions 0.10, 0.50, 0.30 and 0.90. If they do not, check the labels and denominator before changing a formula.
Finally calculate every ticket’s squared error for P, Q, M and R. The totals across forty tickets should be 9.00, 7.40, 7.35 and 6.40. Dividing by forty recovers 0.225, 0.185, 0.18375 and 0.16. This route uses direct outcome records rather than the expectation decomposition.
Change one event label and repeat. The original calibration equalities will generally change because the declared population changed. That is a useful diagnostic: the method should respond to an altered input rather than merely reproduce memorised totals. Keep the original and changed versions separate so another learner can identify exactly which assumption or observation moved.
CHAPTER 26 OF 27 · Practise and follow the evidence
26. Questions about disagreement and combination
Does disagreement prove one forecast is uncalibrated?
No. P and Q are an explicit counterexample. Each matches outcome frequencies within its own probability groups, while their differing information partitions give different probabilities on individual tickets.
Is an average safer because it is less extreme?
Its position between two numbers does not establish calibration or decision quality. In the original table M improves expected Brier score but loses calibration. A real combined method needs direct evaluation for its intended use.
Must the forecast using more information win every time?
No. R minimizes expected squared error among forecasts using the joint signals in our model, yet can receive a worse realized score than P on a particular ticket. The distinction between expected and realized performance is essential.
Can these probabilities determine weather safety decisions?
These are fictional binary teaching models. They do not specify hazard severity, public-warning criteria, organisational obligations or appropriate actions for people. Consult the responsible meteorological service and the relevant current procedures for actual conditions.
CHAPTER 27 OF 27 · Practise and follow the evidence
27. Follow the connected reading routes
For the wider forecast chain, return to Weather Forecasting. For definitions and matched evaluations, use Weather Forecast Verification and Forecast Reliability, Sharpness and Decision Thresholds. For learning a distributional correction from ensemble output, continue through Weather Ensemble Forecast Calibration.
Different forecasting questions need different constructions. Combining Rain-Occurrence Probabilities Across Time Windows concerns several events and their overlap. Here we combine predictions of one event. Comparing Displaced Rainfall Patterns with Neighbourhood Fractions changes the spatial verification question. Allocating Limited Protection Across Simultaneous Forecast Decisions adds a constrained consequence model. None of those questions is settled by averaging two probabilities.
The primary research routes are Gneiting and Ranjan’s Combining Predictive Distributions and Strähl and Ziegel’s Cross-calibration of Probabilistic Forecasts. They provide the broader combination and cross-calibration frameworks. Sources were checked on 25 October 2026. The tickets, tables, score calculations and transfer examples here are original constructions, not reported meteorological results from those papers.
Connect the learning to Learning That Transfers into Life and the How X Works hub. A useful final explanation states the event, identifies what each forecast knows, reconstructs the conditional denominator and tests the combined rule on its own terms.