The Gold Standard of Wisdom | Using an Outside View Without Ignoring Case-Specific Evidence

A plan describes intended actions: who will do what, with which resources and checkpoints. A forecast estimates what will happen. A detailed plan can improve execution, yet detail alone does not prove that the predicted finish date or outcome is realistic. The inside view concentrates on the present case’s steps and scenarios. The outside view compares outcomes from a class of relevant past cases.

Daniel Kahneman and Dan Lovallo described how an inside view can anchor people on scenarios and contribute to overly optimistic forecasts. The outside view, later called reference-class forecasting in project settings, begins with actual outcomes of comparable ventures. It does not forbid knowledge of the case. It changes the order: start with a distribution, then ask what justifies movement within or beyond it.

This order protects against a familiar conversation. Someone lists five reasons the present effort should finish early, yet no one asks how often similar efforts finished early even when participants also felt prepared. The base record is not destiny. It is a calibration surface against which the special-case reasons must earn their weight.

Contents · Next chapter

“Success” is too flexible for a reference class. It may mean finishing by a date, staying within a cost, achieving a quality threshold or producing a result that people can use. Define the event and measurement window first. Otherwise, cases will be included because they feel similar after their outcomes are known.

Our central example concerns a student research team forecasting calendar weeks from approved question to submission of a complete first draft that meets a stated internal rubric. A pause longer than two weeks remains inside calendar duration. Scope reduction after approval counts as a recorded change, not as proof that the original scope finished quickly.

These rules make cases comparable. A team that submitted an outline is not counted as a completed first draft. A project whose clock started after data collection cannot enter without adjustment. Definition comes before selection so that good and poor outcomes face the same boundary. Wisdom begins by preventing the answer from choosing its own evidence.

Contents · Previous chapter · Next chapter

Suppose 12 past teams completed comparable projects in 9, 10, 10, 11, 11, 12, 12, 13, 14, 15, 17 and 20 weeks. The median is the average of the sixth and seventh ordered values: 12 weeks. Nine of 12 took at least 10 weeks. Only one finished in 9 weeks, and none finished in the eight weeks forecast by the current team.

The current inside plan says eight weeks: one for design, three for evidence collection, two for analysis and two for writing. The plan is arithmetically complete, but it contains no allowance for recruitment delay, revision after feedback or overlap limits. The outside view does not prove eight weeks impossible. It shows that eight lies beyond every observed completion in this small class.

A careful first judgement is therefore: the unadjusted reference class centres near 12 weeks, with observed outcomes from 9 to 20, while the inside estimate is 8. The disagreement is a decision signal. The team should test why it expects to outperform all 12 cases, not merely repeat that its schedule contains eight labelled boxes.

Contents · Previous chapter · Next chapter

A reference class can be too broad or too narrow. “All projects” combines unlike work. “Projects by teams exactly like us using this exact question” may leave only one case, eliminating the distribution. Choose factors that affect the outcome and could have been specified before seeing results: task type, required approvals, team experience, data access and quality threshold.

Suppose the 12 cases include eight projects with external participant recruitment and four using existing public data. The current project uses public data. A narrower class of the four may be more relevant if data access materially changes duration, but four cases provide less stable evidence. Show both distributions instead of hiding the choice.

Reference-class selection is a judgement, so document alternatives. If a favourable narrow class was created only after noticing it finishes faster, treat that as a warning. The goal is not to find the average closest to the preferred plan. It is to find a defensible comparison that preserves enough cases to reveal variation.

Contents · Previous chapter · Next chapter

The median of 12 weeks is useful, but the ordered outcomes reveal more. One team took 20 weeks, and three took 15 or more. If a late outcome has serious consequences, the upper portion matters. A decision may need a date with a chosen chance of being met, not merely the most central date.

With only 12 observations, avoid pretending to know a smooth probability curve. Report empirical counts. For example, 3 of 12 finished by 10 weeks, 7 of 12 by 12 weeks and 10 of 12 by 15 weeks. The exact past proportions do not guarantee future probabilities, but they make the risk appetite visible.

If the team promises a stakeholder a 10-week date, it should know that only a quarter of the supplied cases met that threshold. A 15-week planning date covered ten of 12. Choosing between them depends on consequences, flexibility and the quality of the reference class. Wisdom does not make the choice by arithmetic alone; it uses arithmetic to prevent the choice from hiding its exposure.

Contents · Previous chapter · Next chapter

Rejecting the eight-week forecast does not mean discarding the plan. The task sequence can identify dependencies, resource needs and early warning points. The outside view calibrates the total; the inside plan helps explain and manage the path.

Rework the plan against 12 weeks. Keep design at one week, evidence preparation at two, collection and cleaning at three, analysis at two, drafting at two and feedback and revision at two. Some activities may overlap, but overlap is a claim about capacity and dependency. Mark which work can truly occur in parallel.

The combined approach is stronger. The reference class says what comparable cases achieved. The inside account says what must happen here and where the current case differs. If the steps sum to eight only because revision was omitted, the comparison has exposed the missing work. If a new tool genuinely removes a repeated bottleneck, the plan identifies the mechanism for a justified adjustment.

Contents · Previous chapter · Next chapter

List case-specific differences before assigning numbers. The current team uses an existing dataset, has one experienced analyst and faces a fixed two-week ethics review. The first two factors may reduce duration; the review adds a minimum dependency. Evidence matters: “we are motivated” has weak directional value if past teams were also motivated.

Use a transparent adjustment ledger. Begin at the 12-week median. If records from similar public-data projects support a two-week reduction, move to ten. If the fixed review can overlap with preparation, do not add it twice; if it cannot, include its actual critical-path effect. Avoid stacking several optimistic adjustments that all describe the same advantage.

The final forecast might be 10–13 weeks rather than a false precise 10.7. Preserve the base: 12-week median, 9–20 observed range, four-case public-data subset, and stated adjustments. Another reader can then disagree with the adjustment without losing the original evidence.

Contents · Previous chapter · Next chapter

Suppose “experienced analyst,” “faster cleaning” and “fewer coding errors” all arise from one person’s prior experience. Treating them as three independent reductions exaggerates the case difference. Draw a dependency map: analyst experience may influence cleaning speed and error rate, which influence duration. Adjust for the downstream effect once using evidence.

The same rule applies to risks. “Recruitment difficulty,” “low response” and “extended recruitment period” may be one pathway, not three unrelated reasons for delay. A long checklist can create false confidence if correlated items are added like separate votes.

This connects to the Wisdom branch’s existing article on evaluating advice when the track record leaves recommendations out. That guide preserves missing cases and checks whether repeated testimonials describe independent experience. Here the focus is forecasting: dependencies among case-specific reasons should remain visible when adjusting from a reference class.

Contents · Previous chapter · Next chapter

Assume the four public-data projects finished in 9, 10, 11 and 12 weeks. Their median is 10.5. The eight recruitment projects therefore account for most longer outcomes. Using the narrower class makes the current public-data project look closer to 10 or 11 weeks than 12.

Now ask whether the current quality threshold matches those four. If two used a lighter rubric, exclude or adjust them prospectively. The remaining two cases are too few for a stable distribution. The broader class regains importance, perhaps with data-source type treated as an adjustment rather than a gate.

Report sensitivity: broad-class median 12; narrow public-data median 10.5; narrowest compatible evidence only two cases. A wise forecast may use 11–13 weeks, not because averaging medians is magically correct, but because the evidence supports an improvement while remaining too thin for certainty. Showing sensitivity is more honest than presenting one carefully selected class as inevitable.

Contents · Previous chapter · Next chapter

The 20-week case influences the mean more than the median. Investigate it before deleting it. If delay came from a condition the current project cannot face, such as a laboratory closure, an adjusted class may exclude it with a documented rule. If it reflects ordinary approval or coordination risk, it is part of the outcome distribution.

Removing an uncomfortable observation because it makes the forecast look bad is outcome shopping. Keeping every case regardless of definition can also mislead. The decision depends on comparability established by facts, not by the desired statistic.

Run both versions. With the 20-week case, the mean of the 12 durations is 12.83 weeks. Without it, the mean is about 12.18. The median remains 12 in both. This shows why robust summaries can help, while the tail still matters for contingency planning. An outlier can be unusual and decision-relevant at the same time.

Contents · Previous chapter · Next chapter

A team may forecast a 12-week median outcome yet commit to a 15-week external deadline. The commitment includes a service level and consequences; it need not equal the central forecast. Conversely, announcing the optimistic target as the forecast hides risk.

Keep three dates if useful: an internal target, a calibrated forecast and an external commitment. The target motivates coordinated work. The forecast represents current evidence. The commitment reflects reliability, contingency and stakeholder needs. Label each date so that pressure cannot silently convert one into another.

If the evidence changes, update the forecast. Changing it responsibly is not betrayal. The existing guide on caring for people who relied on a decision that must change owns the duties created by reliance: communicate, protect affected people and repair foreseeable harm. Calibration and care belong together.

Contents · Previous chapter · Next chapter

An outside view estimates outcomes; it does not decide which outcome matters most. A 70% chance of finishing by a date may be acceptable for an internal draft and unacceptable for a public promise that affects another person’s visa, care or income. Wisdom connects the distribution to consequences and values.

Create a consequence table. For each candidate commitment, record what happens if it is met, slightly missed or badly missed; who bears the cost; and which recovery options exist. A later date may reduce risk but also delay a benefit. An earlier date may be reversible if framed as a provisional checkpoint.

Avoid using “the data say” to hide a value judgement. The data may show that ten of 12 past cases finished by 15 weeks. Choosing 15 as the promise reflects a desired reliability and the costs of error. State both. Evidence informs responsibility; it does not replace it.

Contents · Previous chapter · Next chapter

Reference classes can help with personal planning, but people are not interchangeable project units. Sensitive decisions involve rights, dignity, preferences and unique constraints. A group average should not become a sentence imposed on an individual.

Use an outside view to calibrate expectations and identify questions. If comparable learners often need 10–14 weeks for a transition, plan support across that interval. Then inspect the individual’s access, prior knowledge, health, responsibilities and goals. The distribution can justify resources without defining the person’s worth or limiting opportunity mechanically.

Some decisions require professional assessment or consent, not an informal base-rate exercise. The method’s role is proportionate. It is strongest for repeatable forecast quantities with recorded outcomes and weakest when the reference class is poor or the decision concerns non-substitutable human values. Knowing when not to calculate is part of wisdom.

Contents · Previous chapter · Next chapter

After an outcome, people can make the path look obvious. A project that finishes in nine weeks may be celebrated as proof that the eight-week forecast was sound, even if it succeeded through an unforeseeable cancellation of other work. A 14-week result may be blamed on incompetence even when the original forecast correctly assigned substantial chance to that duration.

Evaluate the process with information available at the time. Preserve the reference class, adjustments, forecast range and reasons. Then compare the realised outcome and update the model. One result does not validate or invalidate a probability judgement by itself.

Ask whether the forecast was calibrated across repeated decisions, whether evidence was used consistently and whether new information triggered appropriate updates. Outcome quality and reasoning quality can diverge. Wisdom improves the latter so that performance becomes more reliable across cases.

Contents · Previous chapter · Next chapter

At week four, suppose the team completes data cleaning two weeks earlier than the original inside plan, while ethics approval arrives as expected. Update the remaining-duration forecast using progress evidence and the historical behaviour of similar projects at that stage. Do not simply subtract four from the original 12-week total if the reference class shows that later analysis and revision dominate variance.

Record what has genuinely been retired. If cleaning is complete and verified, that risk has changed. If early work was achieved by postponing documentation, the burden may have moved rather than vanished. A status report should distinguish completed, transferred and unresolved work.

Forecast updates are not opportunities to forget the initial estimate. Retain versions to study calibration. If every update becomes optimistic until the final delay, the process needs repair. A forecast history can reveal whether the team incorporates bad news symmetrically with good news.

Contents · Previous chapter · Next chapter

Suppose a stakeholder needs at least a two-thirds empirical basis for a promised date using the 12-case record. By 12 weeks, 7 of 12 cases finished, which is 58.3%, below two-thirds. By 13 weeks, 8 of 12 finished, exactly 66.7%. Under this simple rule, 13 weeks is the earliest date meeting the threshold.

The calculation is descriptive, not a universal probability guarantee. Twelve cases are few, and the current case may differ. Still, the threshold forces the decision rule into the open. Moving the promise to 12 weeks requires either accepting lower empirical coverage or presenting justified case-specific evidence.

Now change the required empirical coverage to 80%. Nine of 12 by 14 weeks gives 75%; ten of 12 by 15 gives 83.3%. The earliest observed threshold date becomes 15 weeks. The values determine the consequence tolerance; the distribution shows which date fits it.

Contents · Previous chapter · Next chapter

A school group plans a public workshop and predicts 90 attendees because 120 people clicked “interested.” An outside view uses the last ten comparable events with the same registration process. Attendance as a share of “interested” ranged from 45% to 72%, with a median of 58%. Applying the median to 120 gives about 70 attendees.

The current event has free transport, which past records suggest raises the share by about eight percentage points when other conditions are similar. A transparent adjusted forecast is around 66% of 120, or 79, with a range retained. It is not 90 merely because organisers feel enthusiasm online.

Change the case: heavy rain is forecast and half the past events were indoors near transport. The original class may no longer transfer cleanly. Use weather-conditioned evidence if available and preserve uncertainty. The method is not “always multiply by 58%.” It is a sequence: define, compare, adjust with evidence and keep the limitations visible.

Contents · Previous chapter · Next chapter

Optimism can be sincere, but forecasts may also be shaped by incentives. A team seeking approval may benefit from a low cost or short duration estimate. A seller may select favourable comparisons. An adviser may remember successful recommendations more readily than abandoned ones.

Separate estimate ownership from decision approval where stakes warrant it. Require the reference-class record, inclusion rule and adjustments to be reviewable. Compare forecasts with outcomes over time. These practices do not assume bad faith; they reduce the power of incentives to operate invisibly.

The UK’s project-appraisal guidance uses optimism-bias and reference-class ideas in institutional contexts. That scale is different from a classroom project, but the governance lesson transfers: make assumptions, evidence and contingencies visible before commitment. Independent review is useful when it tests the model, not when it merely supplies another confident opinion.

Contents · Previous chapter · Next chapter

A reference class built only from completed work can make duration and success look better than the full record. Suppose three additional teams began comparable projects but stopped after 8, 12 and 16 weeks without a complete draft. They cannot simply receive invented completion times, yet excluding them would answer “how long among completers” rather than “what happens to teams that begin.”

Create separate outcomes: completed with duration, still active at the observation cutoff, paused and abandoned. For a promise about completion by 12 weeks, an unfinished case that had already passed 12 weeks counts as not completed by that threshold. A case observed for only six weeks is unresolved for a 12-week threshold. Preserve the observation window.

This resembles survival-data reasoning, but the learner need not force a formal model. The essential wisdom is to name the population and keep missing outcomes visible. A completed-case median may still be useful for scheduling conditional on completion. It should not silently become a success forecast for all starters. The question determines the denominator.

Contents · Previous chapter · Next chapter

Twelve cases can challenge an eight-week claim, but they do not describe every future setting precisely. One added case can change a tail proportion substantially. Instead of hiding that fragility, show counts and test sensitivity. If one more comparable team finishes in eight weeks, the observed minimum changes and the share finishing by ten becomes four of 13 rather than three of 12.

Small samples are not zero evidence. They can reveal that the preferred forecast lies outside every observed case or that outcomes vary much more than the plan admits. The responsible conclusion stays proportionate: “this record gives little support for eight weeks,” rather than “eight weeks is impossible.”

Look for additional compatible records, but do not buy sample size by mixing incompatible outcomes. A hundred loosely related cases may be less useful than twelve well-defined ones. Record how conclusions change under plausible inclusion choices. The uncertainty around the reference class belongs in the final judgement alongside uncertainty within the cases.

Contents · Previous chapter · Next chapter

Some decisions have several defensible classes. The current student project can be compared with public-data projects, projects using the same rubric and projects led by a first-time team. Each class highlights a different mechanism. Do not choose one secretly; construct a comparison panel.

Suppose public-data projects centre at 10.5 weeks, strict-rubric projects at 13 and first-time-team projects at 14. The current case has all three features. There is no automatic formula for averaging medians because the classes overlap and are not independent. Their spread identifies the main uncertainty: faster data access competes with a demanding quality threshold and inexperience.

Use the inside plan to examine those mechanisms and adopt a range that survives the panel, perhaps 11–15 weeks. Then define checkpoints that discriminate between explanations. If data are clean by week two but rubric interpretation remains unstable, the faster class has not resolved the dominant risk. Multiple outside views are useful when they structure learning, not when their numbers are blended into decorative precision.

Contents · Previous chapter · Next chapter

Before work starts, identify evidence that would change the forecast. For the central example, week three might require approved design, accessible data and a verified analysis plan. If all three are complete, compare the project with past cases that reached the same state. If approval is missing, update using the observed approval delay rather than waiting for optimism to repair itself.

Pre-registering the question reduces selective updating. Teams often treat good news as structural and bad news as temporary. A symmetric rule states how both directions count. For example, retire one week of risk when a dependency is completed and verified; add its current expected delay when the checkpoint is missed; do neither merely because participants feel confident.

The checkpoint also improves care. People relying on the commitment learn when they will hear again and which evidence matters. A forecast becomes a living, accountable record rather than a number defended until it fails. That is practical wisdom: remain committed to the people and purpose while allowing the prediction to change with reality.

Contents · Previous chapter · Next chapter

“The outside view ignores all details” is too crude. It begins outside to establish a base, then admits case-specific evidence transparently. “The average is the answer” ignores tails, thresholds and consequences. “Our case is unique” can be true in some respects while still sharing mechanisms with past cases.

“A base rate proves what will happen” confuses distribution with destiny. “A detailed plan defeats the base rate” confuses description with calibration. “Adjustments are subjective, so anything goes” ignores the discipline of directional evidence, dependency checks and preserved baselines.

The best result is not blind averaging. It is a forecast whose reference class, outcome definition, distribution, adjustments and limits can be inspected. Wise judgement remains revisable because its evidence trail survives.

Contents · Previous chapter · Next chapter

Write the decision and outcome measure. Freeze the inclusion rule before collecting comparison outcomes. List the cases, dates, conditions and observed values. Show the distribution with counts and relevant quantiles or thresholds. Identify candidate differences and the evidence for each adjustment.

Then record values and consequences: which error is more harmful, who bears it and what recovery exists. State the forecast range, commitment and review date separately. Assign an owner for new evidence and an update rule.

This record can be short for a low-stakes decision. Its purpose is not paperwork. It prevents memory from rebuilding the forecast after the outcome and makes disagreement productive. One person can challenge the reference class while another challenges an adjustment; both can see what would change the conclusion.

Contents · Previous chapter · Next chapter

Eight comparable school productions took 6, 7, 7, 8, 8, 9, 11 and 13 weeks from final script to dress rehearsal. A new team’s inside plan says six weeks. The median of the reference class is eight weeks. Only one of eight finished by six, while six of eight finished by nine.

The new production uses a previously built set, which credible records suggest saves one week, but it has two first-time lead performers. Begin at eight, apply the evidenced set advantage once and treat performer experience as uncertainty unless a quantified comparison exists. A reasonable working forecast might centre near seven to nine weeks, with the unadjusted class retained.

Now change one fact: the old set requires safety repairs estimated at one to three weeks. The “advantage” is no longer a simple reduction. Update the dependency and range. Explain why repeating “the set already exists” would count only the favourable half of the evidence.

Contents · Previous chapter · Next chapter

What exactly is being predicted? Which past cases qualify, and who chose them? What happened across the whole distribution? Which current differences have evidence? Are several reasons versions of one cause? What would make the forecast update?

Also ask who bears the cost if the optimistic case fails. Can the commitment be staged or made reversible? Is the reference class being used to deny a person an opportunity when support could change the outcome? Numbers do not remove the need for care.

Good advice names its limits. It can say, “Comparable cases usually took longer; this feature may improve our position; here is the range and the checkpoint.” That is more useful than certainty performed for reassurance.

Contents · Previous chapter · Next chapter

When two advisers disagree, ask each to provide an outcome definition, reference class, distribution and adjustment. One may forecast nine weeks from public-data projects; another may forecast 14 from first-time teams using the strict rubric. Their disagreement becomes interpretable: they weight different mechanisms, rather than merely possessing different confidence.

Create a reconciliation table. List shared facts, disputed classifications, unique evidence and predictions that would separate the models. If data cleaning finishes in week two, the first adviser’s mechanism gains support; if rubric interpretation triggers major revision, the second adviser’s concern gains support. Do not average nine and 14 into 11.5 unless a combination rule is justified.

This method protects relationships. People can revise a forecast without being declared foolish, because the evidence and assumptions were visible. It also prevents authority from ending the inquiry. Expertise matters through relevant knowledge, evidence access and a record of correction, not through certainty alone. Wisdom uses disagreement to design the next observation and keeps the decision connected to the people who must live with it.

Contents · Previous chapter · Next chapter

Is an outside view always better? It is valuable when relevant past outcomes exist, but a poor reference class can mislead. Should I ignore my plan? No. Use the plan to understand mechanisms and manage work after calibration. Can I choose the most similar three cases? Only if the similarity rule is defensible and the small sample limitation remains visible.

Does uniqueness make comparison impossible? Rarely in every respect. Find shared mechanisms, while preserving truly novel conditions. Can the method decide moral questions? No. It informs forecasts; values, rights and duties still govern the decision. What if the distribution and expert judgement disagree? Investigate definitions, evidence access, incentives and case differences. Preserve both until the disagreement is resolved or bounded.

When should I stop refining? When further detail is unlikely to change the decision enough to justify its cost, and the remaining uncertainty is acceptable to those affected.

Contents · Previous chapter · Next chapter

Return to The Gold Standard of Wisdom for evidence, values, uncertainty, authority and practical judgement. Use offering useful help while respecting a refusal when advice meets autonomy. The Gold Standard of Education keeps changed-context transfer visible, and the How X Works hub opens the wider mechanism estate.

For the research foundation, read Kahneman and Lovallo’s 1993 paper Timid Choices and Bold Forecasts. For an official applied example, Homes England’s 2024 guidance on optimism bias and contingency explains reference-class forecasts based on outcomes from comparable projects. These sources support the method; they do not make every life decision a construction forecast. The gold standard is calibrated advice that keeps both shared evidence and relevant human particulars in view.

Contents · Previous chapter · Return to Wisdom