The description–experience gap is a predictable difference in how people sometimes choose under risk when the same outcomes arrive as a written probability table rather than as sampled events. Reading “5% chance of a costly failure” is not the same cognitive task as trying a system twelve times and seeing no failure. One gives a stated distribution; the other requires us to infer a distribution from a small, possibly unrepresentative history.
Research on risky choice has repeatedly found differences between these two ways of learning. Classic experiments found patterns consistent with low-probability events carrying more influence in decisions from description and less influence in decisions from sampled experience. That is a qualified behavioural pattern, not a law that every person overweights one source or underweights the other. The immediate practical lesson is to separate what is possible, how often it occurs, what has actually been observed, and which decision matters.
This guide shows the mechanism, original exact probability calculations, sample design, recent observations, personal and organisational decision traps, and repair methods. The numbers in the worked scenarios are illustrative and reproducible, not measured outcomes from eduKate learners or participating organisations. The research sources are linked in a dedicated section and in the discussion of their specific claims.
Contents and reading routes
First principles: two paths, original evidence, small samples, a 95% observation threshold, rare wins, rare losses and sampling quality. These chapters teach what a single observed history can and cannot establish.
Human mechanisms and diagnosis: recency, probability weighting, three experiment designs, weak descriptions, the value of experience, denominators, classroom demonstration and team decisions.
Practical route: everyday boundaries, diagnostics, a repair loop, a worked audit, independent transfer, limitations, FAQs, primary evidence and related learning.
The two paths into a risky decision
A decision from description begins with a representation of outcomes and their chances. You may read a product’s known warranty rates, a weather probability, an experimental option with two rewards, or a clearly stated lottery. If that description is reliable and complete, the decision maker can inspect every listed outcome without waiting to experience it personally. The written percentages do not dictate a particular risk preference, but they provide a distribution on which a calculation can be based.
A decision from experience begins with encounters. A student may have seen a dozen problem formats, a production manager may have observed ten equipment runs, or a team may have watched a prototype succeed repeatedly. If probabilities are not explicitly supplied, the decision maker must infer them from observed events. The denominator then becomes the number and quality of encounters, not an authoritative stated probability, and an unseen event may be genuinely possible.
The difference is not simply reading versus doing. Experience changes the available evidence, how outcomes are noticed, how many trials are sampled, which events receive attention and whether feedback arrives at all. A beautiful chart of badly sampled records can still represent experience. Conversely, a warning based on years of careful measurements is a description even if someone else earned those measurements through experience. Classify how probabilities became known before diagnosing a judgement.
Imagine a school activity where one learner is handed a card saying that a rare reward appears on 5% of independent draws, while another tests an unlabelled box twelve times. Both face the same designed distribution. Yet the first has a complete stated probability and the second has an uncertain estimate, possibly based on no observed reward. Asking whether their choices match without noting the evidence difference confuses the problem before the decision has even begun.
What the original experiments established
In their 2004 Psychological Science study, Ralph Hertwig, Greg Barron, Elke U. Weber and Ido Erev compared choices made with described risky prospects and with experience-based sampling. Their reported pattern challenged the assumption that decision weights inferred from described gambles automatically explain the same person’s behaviour when probabilities must be learned by trying options. The paper examined both missed events in small samples and the influence of later observations.
The study’s conceptual contribution is stronger than a catchy phrase such as people ignore danger. It showed that the source and process of acquiring information can alter choice. That invites a research question: did an observed rare outcome fail to occur in the sample, was it noticed but given little weight, or did the decision maker use a different strategy altogether? These explanations imply different educational and organisational repairs.
Later studies refined the account. Hau, Pleskac, Kiefer and Hertwig examined sample size and experienced probabilities in 2008. They reported that larger samples reduced the gap but did not eliminate it entirely. This matters because a repair that only says try the risky option more often may improve coverage while leaving differences in weighting, attention or decision goals unexamined.
A 2018 meta-analytic review by Wulff, Mergenthaler-Canseco and Hertwig considered more than 70,000 choices from more than 6,000 participants in the sampling-paradigm literature. It identified small-sample error as an important contributor and found that problem structure moderated the phenomenon. That does not establish a universal size for the effect in schools, workplaces or any particular country; it shows why simplistic one-cause explanations are inadequate.
First mechanism: the sample may never contain the rare event
Suppose a rare event has a true probability of 5% on each independent draw. The chance that one draw does not contain it is 95%, or 0.95. If the draws are independent and identically distributed, the probability of missing it throughout twelve draws is 0.95 raised to the twelfth power, approximately 0.5404. Thus a twelve-draw experience has about a 54.04% chance of showing no example at all, although the event is real.
Notice the denominator: twelve independent draws, not twelve different people, twelve calendar days or twelve types of risk. The calculation needs stable independent trials with probability 0.05 each time. If the environment changes after each choice, observations are correlated, or the observer samples only easy cases, the simple power formula may not describe the situation. The mathematics is clear; its assumptions must be checked separately.
For twenty independent draws the probability of zero rare outcomes is 0.95 to the twentieth power, approximately 35.85%. For forty draws it is about 12.85%. More observations usually make total omission less likely under the model, but even forty experiences do not guarantee that a 5% event appears. A quiet history cannot be converted to proof of impossibility.
Now imagine two otherwise identical people. One reads a verified 5% incident probability; the other observes twelve quiet trials. The first can reason about a nonzero expected rate immediately. The second might assign the event little attention because it did not occur. Their different choices could reflect different information, not necessarily different intelligence, courage or moral character. A fair diagnostic must expose the sample.
How many trials before there is a 95% chance of seeing it once?
Turn the preceding calculation around. To have at least a 95% chance of seeing at least one event when its independent probability is 5%, require that the probability of seeing none be at most 5%. In symbols, 0.95 raised to n must be no greater than 0.05. Taking logarithms gives n at least ln(0.05) divided by ln(0.95), about 58.4 draws; the smallest whole number is 59.
Fifty-nine is not a magic sample size for understanding all risks. It only clears one specific threshold under the stated model: at least one sighting of an event with fixed independent probability 5%, with 95% assurance. It gives no promise of precisely estimating the event rate, the severity of its consequences, its clustering or the behaviour of an entirely different environment. Meeting a detection condition is not the same as building a reliable predictive model.
If the event probability falls to 2%, the analogous condition is 0.98 to the n no greater than 0.05. The smallest integer n is 149. That is a dramatic change from 59 trials because the event is much rarer. A manager saying we tested twenty times and nothing failed has provided only a small amount of evidence about a potential 2% failure mode; the chance of no sightings in twenty such independent trials is still about 66.76%.
The practical repair begins by asking what the sample was meant to establish. Was it a classroom illustration, a rough discovery exercise, a reliability demonstration, a formal regulatory test, or evidence for a safety-critical release? Each has a different acceptable error budget. The calculations here teach how omission happens. They are not a certification plan and should never be treated as a substitute for qualified domain-specific reliability analysis.
Worked example: a rare win can be underrated by experience
Consider a fictional learning game with two options. Option A pays four points with certainty. Option B pays ninety points with probability 0.05 and zero points otherwise. The objective expected reward per draw for B is 0.05 times 90 plus 0.95 times zero, equal to 4.5 points. A gives four points per draw. Under risk-neutral expected-value maximisation with the stipulated distribution, B has an advantage of 0.5 point per draw.
Suppose a learner samples B twelve times before making a final decision. If the rare win never appears, the observed average for B is zero divided by twelve, or zero points, which makes A’s four points look preferable. If B appears once, the empirical average becomes ninety divided by twelve, or 7.5 points; now B looks dramatically better. The actual expected value remains 4.5 in either sample, because the underlying probability was specified as unchanged.
The same individual could therefore flip a preference solely because of the twelve outcomes shown. That is a sampling story, not proof that people always optimise sample means. People can use heuristics, remember distinctive outcomes, dislike variability or pursue different goals. The example isolates the informational issue: a finite experience does not reproduce every property of the generating distribution.
The result is especially instructive in education. A child who has never drawn the large reward may be entirely reasonable to doubt a claim based only on personal trials. The teacher’s useful intervention is to show the two sources of information side by side, calculate the objective expectation, disclose assumptions and invite the child to predict what a second batch might show. Scolding the child for being irrational teaches very little.
Worked example: the same omission can conceal a costly loss
Now reverse the stakes. A safe option loses three points on every draw. A risky option loses eighty points with probability 0.05 and loses nothing with probability 0.95. Its expected loss is 0.05 times 80 plus 0.95 times zero, equal to four points per draw. If the objective is minimising expected point loss and the chances are known, the safe three-point loss is preferable.
Someone who tries the risky option twelve times and sees no loss might estimate its average loss as zero divided by twelve, or zero. The dangerous option now appears costless. If one eighty-point loss occurs in twelve trials, the observed sample mean loss becomes eighty divided by twelve, approximately 6.67 points per trial. The sudden change in empirical estimates comes from sampling, even though the true model has not changed.
This is the critical asymmetry that superficial advice misses. Missing a rare win can make someone avoid a favourable gamble; missing a rare severe loss can make someone accept an unfavourable risk. Both arise from the same sampling mechanism. A decision aid must therefore ask what the rare outcome is, whether it helps or harms, who bears it and whether an expected-value criterion is ethically or practically suitable.
Do not use toy points as a safety policy. Real losses may be irreversible, concentrated on vulnerable people or unacceptable even at very low probabilities. A hospital, construction site or security team needs appropriate expert standards and safeguards, not permission to accept an objectively dangerous practice because an expected-value worksheet happened to return a convenient number. The example teaches why event absence must not be mistaken for zero possibility.
More trials do not correct a biased way of gathering them
Suppose a technical team runs one hundred successful tests, but all were performed on a stable indoor network. The service will later be used across intermittent mobile connections, expired sessions and noisy networks. There is no reason to think those one hundred tests form an independent representative sample of the deployment environment. The number one hundred creates an impression of precision without covering the failures that matter.
A useful sampling frame lists which conditions can appear and why. For a learner evaluating an unfamiliar exam strategy, that could mean routine questions, tricky wording, time pressure and unfamiliar presentation. For a service, it could mean diverse device classes, paths, timeouts, permission states and recovery conditions. Sampling all four kinds gives more meaningful evidence than repeating only the easiest condition many times.
Selection can be voluntary. A person exploring an unfamiliar option may stop after a satisfying result, or avoid testing after a bad one. Both choices alter the observed sequence and what is remembered. A manager who keeps testing until the dashboard turns green has not followed the same sampling rule as a manager who decided the test count before the first result. Report the stopping decision along with the results.
The correction is not to demand random sampling where it is impossible or unethical. It is to disclose the sampling frame, record missing conditions, distinguish natural observations from designed stress tests and explain what cannot be inferred. A thousand biased examples can remain poor evidence about an unsampled population. The description–experience gap cannot be repaired merely by enlarging an unexamined denominator.
Second mechanism: the last few experiences can loom large
People may give recent feedback more influence than older feedback, especially while actively sampling and deciding when to stop. The 2004 study examined recency as one candidate explanation. Later synthesis shows its role depends on experimental design, including whether the decision maker can freely terminate exploration. It is more accurate to say recency can contribute under some conditions than to claim every experienced decision follows a fixed last-outcome rule.
Imagine a fictional option sampled twenty times with one rare loss near the beginning and nineteen uneventful draws afterward. Another person observes the same multiset of outcomes but sees the loss on the twentieth trial. Both have one loss in twenty, so their simple observed frequency is identical: five percent. If their later choices differ, the position, attention or interpretation of outcomes may be relevant even though the arithmetic sample mean is unchanged.
A robust log separates sequence from summary. Store the order of encounters when ethically and legally appropriate, calculate the overall frequency and inspect whether a dramatic late observation changed a decision. Never infer a recency effect solely from one person’s final choice; perhaps the environment changed, the final observation was more diagnostic or the chooser’s goal shifted. A controlled comparison is needed to isolate order.
Decision repair can explicitly delay irreversible action after a striking event, compare both early and late segments, and ask for a predeclared decision rule. Yet delay is not always possible or appropriate, especially during an active safety incident. In those cases an organisation should have a pre-approved response policy rather than depend on whichever episode happens to be freshest in someone’s memory.
Third mechanism: knowing a frequency is not the same as using it
Even when sampled frequencies approximate the true probabilities, the description–experience gap can remain. Hau and colleagues reported that enlarging samples reduced but did not erase the phenomenon. The later meta-analysis likewise identified residual differences and possible changes in how probabilities influence choice. These findings rule out an explanation in which every gap is purely a statistical accident from missing rare events.
One person may understand that a rare outcome occurs in five of one hundred cases yet treat it as especially salient because a description explicitly names it. Another may remember the same count from a long sequence of mundane trials and give the ordinary outcome more attention. Neither observation proves which subjective decision weight was used. Researchers infer as-if weighting patterns from choices under specified models and experimental assumptions.
Separate three layers on paper: the described objective frequency, the experienced empirical frequency and the revealed preference among options. A person could correctly estimate the chance of a loss but still prefer a safer option because loss severity or personal circumstances make it undesirable. Conversely, a person could underestimate the probability but adopt the safe action because consequences are unacceptable. Preference differences are not synonymous with probability misunderstanding.
The diagnostic is therefore two questions, not one. First ask what probability the person believes and how that estimate was formed. Then ask which option they prefer and why, including consequences, reversibility, time horizon and tolerance for uncertainty. If you infer belief from choice alone, you may label a cautious but well-informed decision as ignorance, or miss a genuine knowledge problem because the person’s choice accidentally matches the recommended action.
Three experimental designs that should not be conflated
The sampling paradigm gives participants opportunities to observe outcomes before making a final consequential choice. Often the samples themselves are non-consequential, so a person can explore options and then select. A description condition provides probability information directly. The contrast is especially useful for studying information acquisition because the number and composition of observations can be recorded separately from the final choice.
A partial-feedback paradigm differs: participants make repeated consequential choices and usually see the result only for the option selected. Exploration and reward-seeking occur together, which changes motivation. A person who chooses the same option repeatedly may learn little about alternatives. The unchosen counterfactual is missing, so the data are selected by the person’s own behaviour. Findings from this design do not automatically mean what they would in a free-sampling design.
A full-feedback paradigm reveals outcomes for both the selected and unselected alternatives after a decision. That richer feedback can teach participants about forgone results, but it also changes attention and regret. Seeing what would have happened is not the same as having chosen that alternative. The 2018 meta-analysis and later research discussions distinguish these designs precisely because experience is not one uniform information source.
An eduKate classroom demonstration should label its design honestly. If learners merely draw tokens to inspect two boxes and then choose, that is a sample-and-choose lesson. If every draw wins or loses real classroom points and only the chosen box reveals a result, it resembles partial feedback. If both results are displayed, it resembles full feedback. Mixing them without explanation makes any comparison of learner decisions difficult to interpret.
Written descriptions can also mislead the reader
Descriptions have an advantage only when they are accurate and intelligible. A printed 5% probability may come from a narrow population, a model with unknown assumptions or a study conducted years ago in different conditions. Numbers can appear precise while the underlying evidence is uncertain. Some descriptions exclude inconvenient outcomes entirely or combine different outcome categories under one label.
Before treating written probabilities as ground truth, ask who measured the event, what denominator was used, which conditions were included, how the observation window was defined and whether the estimate has uncertainty. A chance of one event per completed transaction is different from one per participant, per day or per year. A statement such as five percent of systems fail is incomplete until the unit and failure definition are clear.
Descriptions may also alter attention through vivid labels or framing. A rare loss described with dramatic language could appear more salient than the same loss expressed as one event in twenty. The ideal comparison uses logically equivalent options and examines how presentation changes judgement. In ordinary life that equivalence is difficult to guarantee, making evidence quality and plain-language explanation critical.
The goal is not to replace experience with printed authority. It is to let the two streams correct each other. Description provides a broader map if its sources and denominators are sound; experience tests whether the map transfers to the present context. When they disagree, investigate whether the population changed, the data were biased, the experiment was small or the description was wrong. That is a learning loop rather than blind loyalty to either source.
Why learning from experience can be intelligent
Experience is not a defect. Direct feedback can reveal cues that a written summary omitted: a machine’s operating rhythm, a student’s confusion during multi-step reasoning, or an interface that produces errors only under a particular workflow. People often learn effective local strategies through repeated interaction, including where the environment is changing and a stale statistical description would mislead.
The description–experience gap therefore is not a scoreboard where the reader of a probability table always wins. A published distribution can be irrelevant after a major policy, product or environmental shift. A small but well-targeted experiment may reveal that change earlier than a large historical report. The key question is whether the experience was diagnostic of the future decision’s conditions, not merely whether it felt vivid.
Experienced decision makers also face different incentives. In an interactive task, they may seek information strategically and adapt their sampling. That can be wise when sampling has cost or risk. A rule demanding fifty-nine exploratory trials regardless of their consequences would be absurd in many settings. Optimising what to observe can matter more than collecting a fixed number of indiscriminate draws.
The mature skill is to integrate experience with base rates, mechanisms and appropriate external evidence. Treat a surprising event as a clue, update the model, and test whether the clue recurs or reflects a one-off circumstance. Teach learners to distinguish I saw this, This often happens here, and This will probably happen next time. Those are three different strengths of claim.
Denominators: the small detail that changes the story
Imagine a department reports that there were three serious failures in a quarter. Without the number of opportunities for failure, the count cannot be compared with another department’s one failure. Three failures among sixty relevant operations is five percent; one among five operations is twenty percent. On these fictional figures the department with more total failures has the lower observed rate. The example teaches why narrative experience should be paired with exposure counts.
The word opportunity requires its own audit. If an operation can fail three times during one session, counting sessions rather than failure opportunities may be defensible for some goals and misleading for others. For rare events, unit choice is especially powerful because the numerator stays small. Define the event, unit of exposure, time period and censoring rules before dividing anything.
A frequency of zero among twelve observations is zero percent observed, not a certified zero-percent underlying probability. A description stating 5% after carefully sampled data is a model estimate, not necessarily an immutable natural constant. A decision maker needs to know which kind of number is present. A neat chart that prints 0.0% next to an unobserved event should explain the denominator and what remains uncertain.
Teach this by asking for two sentences: Out of twelve independent attempts, no rare event was observed; and Under the stipulated 5% model, the probability of this happens to be about 54%. These statements are compatible. Learning to hold them together is a form of quantitative intelligence, because it stops observation and inference from collapsing into the same claim.
A classroom experiment that does not claim invented findings
Prepare two descriptions of the same fictional box. The safe box yields four tokens each trial. The other yields ninety tokens on one of twenty equally likely outcomes and zero otherwise. One group reads the full payoff distribution, while another group investigates a simulator that draws independently using the known one-in-twenty rule. The intended learning target is to compare what learners knew, not to force an expected choice.
Before showing any simulated draws, ask students to calculate the rare box’s expected reward: ninety divided by twenty, equal to 4.5 tokens, compared with the safe box’s four tokens. Ask them why 4.5 is not a reward they will actually receive on one trial. It is an average over the stipulated distribution, not a prediction that each outcome will land near 4.5. This distinction protects the lesson from a common expected-value misconception.
Next have students record how many times the rare event appears in twelve draws, preserving the sample size rather than only the final average. If zero rare outcomes appear, the sample average is zero; if one appears, it is 7.5 tokens. Let students explain why both samples can occur even when the distribution is unchanged. A changed sequence with the same count can then open a discussion of recency and memory.
The teacher should never invent learners’ observed choices or claim the activity proved a theory without collecting data. A useful assessment is an independent transfer: give a new probability and a new sample size, require a no-event calculation and ask which missing assumptions would break it. That tests mechanism understanding. A poster saying rare events are underweighted is only a slogan until students can handle an unfamiliar case.
How groups can build an institutional description–experience gap
Organisations turn experience into stories and dashboards. A team may believe a process is safe because no failures appeared in its last ten launches, while an industry-wide source documents a low-frequency catastrophic failure. The reverse can happen: one memorable local incident may dominate discussion even when broad evidence shows that a safer intervention has worked elsewhere. Neither extreme can be evaluated fairly without denominators, exposure and context.
A good incident review distinguishes first-hand local evidence from general statistics. What did our team actually observe? Under which conditions? What was the external distribution based on? Have conditions changed enough to make it less applicable? What costly outcomes were never tested? This is not a contest between veterans and analysts. It is an evidence integration problem.
Group deliberation introduces a second risk: shared anecdotes may circulate more often than privately held contrary data. The site’s Hidden Profiles guide explains that separate mechanism. One story told by ten people is not ten independent observations. Record the original event, the number of retellings and who owns the unique evidence before counting consensus.
For procurement, systems reliability and school operations, a practical meeting format uses four columns: stated risks, directly observed cases, unobserved but plausible outcomes and next evidence needed. Assign owners for verifying each column. A decision memo can then explain why a particular action is justified despite uncertainty, without claiming that a small successful pilot established universal safety.
Personal risk decisions without false reassurance
People often estimate everyday risks using accessible experience. A route crossed safely many times feels familiar; a device that has never visibly failed feels dependable. Familiarity supplies information about local conditions, yet rare losses may remain unseen during an individual’s limited exposure. The appropriate response is not to become fearful of every possible harm, but to assess evidence and consequences together.
In medical contexts, a person should use trustworthy clinician guidance and relevant studies rather than conclude from a handful of uneventful experiences that a rare adverse effect is impossible. A package insert can itself be difficult to interpret: the stated frequency must match the treatment, population, dose and observation period. This article does not give medication advice. It shows why direct anecdote and population evidence can answer different questions.
In financial contexts, a long run of favourable returns does not establish that major losses cannot occur. Conversely, a vivid bad year may make a sensible long-term plan feel impossible. The decision depends on the actual product, horizon, liquidity needs and capacity for loss, not a single toy probability. No specific investment recommendation follows from the description–experience research.
For digital safety, attackers may exploit precisely the fact that people rarely encounter account compromise firsthand. Someone who has never lost an account could discount protective steps. Yet an exaggerated description of attack probabilities is also poor education. Better messaging uses grounded mechanisms, feasible protective actions and honest uncertainty. The distinction between observed absence and nonzero risk helps without resorting to fear.
Diagnose whether the error is evidence, memory or preference
Start with a four-part interview rather than a label. First ask what information was available: explicit probabilities, raw observations or remembered stories. Second ask for the sample: how many trials, in which conditions, and how the stopping point was chosen. Third ask which outcomes were vivid, recent or missing. Fourth ask what the decision maker wanted to optimise: expected reward, avoiding any catastrophe, learning or protecting another person.
An evidence problem appears when the sample is too small or biased to support the claim. A memory problem appears when the person gives unusual weight to one recent or emotionally striking event despite acknowledging the wider record. A representation problem appears when a written percentage is misunderstood or its denominator is invisible. A preference difference appears when two people understand the probabilities yet value consequences differently.
These diagnoses call for different repairs. For sample insufficiency, broaden the frame or use credible outside data. For memory imbalance, inspect the complete ordered record rather than only the last story. For confusing descriptions, rewrite probabilities with absolute frequencies and transparent denominators. For real preference differences, discuss goals and accountability rather than forcing agreement. Wrong treatment of a correct diagnosis can be worse than no intervention.
A useful check is to show the same learner an equivalent description after their experience and ask whether their probability estimate changes. If their belief changes but their decision does not, investigate values, stakes and options. If their belief remains inconsistent with clear evidence, investigate comprehension or trust. In every case remember that one answer does not reveal an underlying psychological mechanism with certainty.
A practical evidence-and-decision repair loop
Set a concrete target: decide which of two options to use, or decide whether evidence justifies a wider rollout. Name the outcome and the smallest genuinely meaningful unit. Gather the best available description, including source date, population, uncertainty and assumptions. Separately collect the relevant experience log, ensuring the log contains unsuccessful and unselected cases whenever possible rather than just successes.
Reconcile the two sources. Compare the described event probability with the empirical event count divided by the observed denominator; calculate the probability of omission under plausible assumptions. If the evidence comes from a tiny or biased sample, do not quietly pool it with a much larger source as though both were exchangeable. Explain any difference in exposure conditions and decide what additional evidence could change the recommendation.
Next choose a decision rule openly. Expected value may be useful for a toy repeated game with small fungible points. A real decision might instead use a maximum acceptable loss, mandated safety standard, affordability constraint or human rights obligation. Record that rule before observing more outcomes so the team cannot move the threshold opportunistically after seeing the result it prefers.
Finally probe transfer and revisit. Ask what would change if the rare-event probability halved, if the loss became irreversible, if observations were correlated or if data came from another population. A good conclusion survives explicit changed-context questions or says where it stops applying. Make the feedback loop visible: evidence, prediction, output, comparison, correction and a documented next test.
A reproducible decision-audit worksheet
Use the following fictional record. A risky training simulation produced no rare losses in eighteen independent opportunities; a credible design specification states the chance of a rare loss is 5% in each opportunity. The choice is between that risky simulation and an alternative with a stable three-point loss. When a rare failure occurs in the risky simulation it costs eighty points, while all other risky trials cost zero.
Column one is description: a 5% probability and an eighty-point loss produce an expected loss of four points per trial. Column two is experience: zero observed failures in eighteen trials gives a sample loss average of zero points per trial. Column three is sampling uncertainty: 0.95 to the eighteenth power is approximately 39.72%, so seeing no failure is not surprising under the specified generating model.
Column four is decision criterion. If these are repeatable fictional points and the sole goal is minimising objective expected loss, choose the stable three-point loss rather than the risky expected four-point loss. If instead the goal were to keep the realised worst-case loss below a fixed threshold, the calculation and comparison would be different. Neither criterion may be appropriate for real safety decisions, where formal requirements override a classroom gamble.
Column five is the independent audit question: what would make the eighteen observations inapplicable? Answers include a changed generator, correlated draws, missing failures in the log, a different population or an incorrect design description. The final note should not simply read The risky option is safe because we saw no loss. It should state the evidence, assumptions, criterion and limitation in separate clauses.
Independent transfer: an unfamiliar two-percent event
Try this changed-context problem before reading its answer. A rare benefit appears with probability 2% on each independent trial and pays 150 points; otherwise the option pays zero. A safe option pays 2.5 points per trial. A learner samples the rare option twenty times without seeing the benefit. Find the objective expected reward, the chance of that quiet twenty-draw experience and the smallest number of independent draws needed for at least a 95% chance of seeing the rare benefit once.
The expectation is 0.02 times 150 plus 0.98 times zero, equal to three points per draw. Under expected-value maximisation the rare option’s three points exceed the safe 2.5 by 0.5 point. Yet the observed twenty-trial average is zero points because the benefit did not appear. Both statements can hold simultaneously. The learner’s observed history is not a denial of the stipulated probability model.
The chance of no rare benefit in twenty independent trials is 0.98 raised to twenty, approximately 66.76%. The criterion for a 95% chance of at least one occurrence is 0.98 raised to n no greater than 0.05. Taking logarithms and rounding up to a whole trial gives 149. The sample size is more than seven times twenty; this shows why quiet experience is common when events are rare.
Now alter the assumptions: suppose trials are correlated because the same hidden environmental condition persists. The preceding independent-draw powers are no longer justified without a dependence model. A correct learner should not merely replace one number in the formula and declare success. The transfer skill is to recognise which structure allowed the original calculation and identify which new structure breaks it.
Limitations, context and the danger of overselling one bias
Research does not imply a simple inborn rule that everybody ignores rare dangers in experience. The 2018 synthesis found meaningful moderators, including whether the problem involved a risky and safe option or two risky options, and differences in sampling opportunities. The gap may be smaller, differently expressed or absent in a particular task. People also vary in goals, background knowledge, attention, sampling budgets and trust.
Nor is every difference between a risk description and a lived history a cognitive error. If a written estimate is outdated or based on another environment, local experience could be the better predictor. When feedback is delayed or outcomes are hidden, the observer may have no genuine access to the event count. When rare events are clustered rather than independent, classical binomial calculations based on independent draws are inappropriate.
Expected values illustrate one decision rule rather than every ethical or rational rule. Outcomes may be irreversible, distributively unfair or impossible to offset with gains elsewhere. For an organisation, the costs of rare safety failures cannot simply be averaged away in a classroom exercise. Regulations, protection duties, professional standards and stakeholder consent may impose constraints the toy payoff table lacks.
A confident article should say what the evidence supports and what remains open. This guide explains why descriptions and samples can yield different choices, how to calculate one omission mechanism and how to create a better evidence audit. It does not diagnose a specific individual, certify a safety process, prove that an observed decision came from recency or promise a universal method to eliminate human judgement bias.
Frequently asked questions
What exactly is the description–experience gap?
It is a difference observed in some risky-choice tasks between decisions based on explicit outcome probabilities and decisions based on sampling or feedback. The source of information changes what is known and sometimes the weights apparent in choices. The effect is moderated by task design and should not be treated as a trait assigned to everyone.
Does experience always make people underweight rare events?
No. The classic as-if pattern is a research result under specified tasks, not an inevitable rule. A vivid recent rare event, an unusual sample or a different payoff structure can produce different choices. The meta-analytic evidence identifies sample size, problem structure and decision design as relevant moderators.
Why does zero out of twelve not prove zero percent risk?
A sample proportion of zero describes the twelve observations. Under a stable independent 5% model, the probability of twelve consecutive non-events is about 54.04%. The event may be real while absent from the sample. Inference about an unknown true probability requires a suitable statistical model and more careful design.
What sample size guarantees I will see a rare event?
No finite number guarantees seeing an event when each independent trial gives it a probability less than one. You can instead target a confidence level under assumptions. At 5% probability, 59 independent trials provide at least a 95% chance of seeing it at least once; that is not proof of precise frequency estimation.
Is the expected-value choice always the best choice?
No. Expected value is one useful rule for specified repeatable payoffs. People may reasonably prioritise downside protection, fairness, liquidity, legal duties or irreversibility. A decision is not diagnosed from expected value alone. Specify the actual objective and constraints before treating a choice as incorrect.
How do recent outcomes change these judgments?
Recency may influence what information receives attention, especially under some optional-stopping conditions. But different choices after a late event do not prove recency caused them. Compare otherwise equivalent histories with different outcome order and account for changes in the environment, goals and feedback.
What is the difference between observed frequency and probability weighting?
Observed frequency is a count divided by a denominator. Probability weighting is a model-based account of the influence assigned to that chance during choice. A person can estimate a probability correctly yet care strongly about a rare severe loss. Belief and preference should be measured separately.
Should teachers hide probabilities so students learn by doing?
Not as a general rule. The useful educational task is to contrast the information obtained through experience with a reliable description, then explain where each helps and fails. Reveal the generation rule during debriefing and use a changed-condition test. Do not present a classroom simulation as an empirical replication without data.
Can teams fix the problem by collecting more data?
Larger representative samples reduce omission risk under suitable assumptions. They cannot repair irrelevant conditions, biased sampling frames, missing unsuccessful cases or changing environments on their own. Teams should also record the decision rule, compare outside evidence, inspect chronology and audit denominators.
Does this research prove anything about artificial superintelligence?
No. This is evidence about human choice and information acquisition. Statistical sampling ideas may inform evaluation of automated systems, but there is no inference from these experiments that artificial superintelligence exists or that an AI agent has human experience. Treat application-specific claims as separate research questions.
Can I apply the five-percent calculation to a real medical or safety decision?
Only as a basic mathematical illustration if the trials truly match the independent and identically distributed model. Real medical or safety risks may involve unequal exposures, dependence, changing conditions and serious ethical constraints. Consult appropriate qualified guidance and standards rather than using this article as clearance to accept a hazardous action.
Original studies and evidence boundaries
Hertwig, Barron, Weber and Erev (2004), Decisions from Experience and the Effect of Rare Events in Risky Choice, Psychological Science, volume 15, pages 534–539. This is the foundational comparison of choice from descriptions and samples and discusses small-sample omission and recency. The original publication is dated August 2004.
Hau, Pleskac, Kiefer and Hertwig (2008), The Description–Experience Gap in Risky Choice: The Role of Sample Size and Experienced Probabilities, Journal of Behavioral Decision Making, volume 21, pages 493–518. This primary experimental work examines how changing the amount of sampled experience affects the gap and tests alternative explanatory strategies.
Wulff, Mergenthaler-Canseco and Hertwig (2018), A Meta-Analytic Review of Two Modes of Learning and the Description–Experience Gap, Psychological Bulletin, volume 144, pages 140–176, DOI 10.1037/bul0000115. It reports the 70,000-plus choice synthesis and highlights sample size, problem structure and optional stopping as moderators, so claims must not be stretched into a universal rule.
Hertwig and Erev (2009), The Description–Experience Gap in Risky Choice, Trends in Cognitive Sciences, volume 13, pages 517–523, reviews the difference among experimental paradigms and interpretation of rare events. For a more recent research synthesis, read Hertwig and Wulff (2022), A Description–Experience Framework of the Psychology of Risk, Perspectives on Psychological Science. The latter is a broader framework, not the source of this article’s fictional numerical outcomes.
Verified eduKate reading routes
Begin with the How Intelligence Works master page. To study a different mechanism in repeated choices, continue to Probability Matching, which separates outcome frequencies from an optimal choice rule rather than explaining description–experience differences.
For team deliberation, read Hidden Profiles, which explains overlooked information distributed across colleagues. For a different sensitivity to how options are carved into groups, read Partition Dependence. Those are neighbours, not replacements for this article’s rare-event sample mechanism.
Close by checking whether you can explain three things without notes: why no event in twelve independent 5% trials is plausible; why reading a reliable probability and sampling an option provide different evidence; and why a changed stopping rule or population invalidates a careless transfer of an earlier conclusion. Those explanations are better signs of understanding than memorising the phrase experience gap.