A town plan can look extraordinarily certain. Parcels have edges. Roads have widths. Population forecasts arrive as neat columns. A coloured map can make the future appear to have already agreed with itself.
But planning rarely begins with certainty. It begins with an incomplete description of a moving place.
Some people have not been counted. Some journeys are invisible because no sensor records them. Some datasets were collected under definitions that changed halfway through the series. Some neighbourhood conditions move faster than the official statistics. Some commercial datasets disappear when a vendor changes policy. Some information is precise enough to be dangerous because its precision hides the assumptions underneath it.
This is the data gap. It is not a temporary inconvenience on the way to perfect information. It is one of the permanent operating conditions of town planning.
The planner’s job is therefore not simply to collect more data. It is to know what is known, what is missing, what is uncertain, what is biased, what is too old, what is too coarse, what is too private to use, what has changed definition, and which decisions can safely proceed despite those limitations.
A map is an argument, not the territory
Every planning map is a model. The map selects variables, boundaries, categories and time periods. It compresses reality so that a decision can be made.
That compression is useful. Without it, planning would be impossible. But compression also removes information. A land-use map may show “residential” while hiding whether the homes are affordable, overcrowded, vacant, ageing, newly built, rented, owner-occupied or occupied by people whose daily needs are very different. A transport map may show a bus line while hiding reliability, waiting time, crowding, slopes, crossings, heat exposure and whether the service still runs when a shift worker actually needs it.
The first discipline is therefore epistemic: treat every dataset as a claim about the town rather than the town itself.
This matters more as planning becomes more data-rich. A dashboard with hundreds of layers can create an illusion of completeness. More layers do not automatically create more truth. They can just create more opportunities for mismatched definitions, duplicated error and misplaced confidence.
The seven common forms of the data gap
Missing data is only one kind of weakness. A robust planning process distinguishes at least seven.
1. Coverage gaps
A dataset may omit certain places, households, businesses or behaviours. Informal settlements, unregistered work, temporary residents, homeless populations and people who avoid formal systems are especially easy to undercount. The people most exposed to planning failure can therefore be the least visible in administrative data.
2. Time gaps
Official data often arrives slowly. A census may be excellent for structural patterns and weak for sudden change. Building approvals may show intended development rather than completed development. Traffic counts may describe an ordinary weekday that no longer exists after a major employer closes or a new rail line opens.
3. Granularity gaps
Citywide averages can conceal neighbourhood crises. Neighbourhood averages can conceal a single dangerous crossing. A block-level model may still miss the experience of a wheelchair user approaching one kerb ramp. The right scale depends on the decision.
4. Definition gaps
Two agencies can use the same word differently. “Affordable housing,” “green space,” “vacancy,” “household,” “job,” “trip” and “public transport access” can each be defined in multiple defensible ways. If definitions change between years, a trend line can move even when reality does not.
5. Behaviour gaps
People do not always behave the way a model assumes. A theoretically walkable route may be avoided because it feels unsafe. A park may exist but be culturally unwelcoming. A parking price may change route choice in one district and barely matter in another. Behaviour must often be observed, not inferred from geometry alone.
6. Governance gaps
Data can disappear because ownership changes. A private mobility service may stop sharing trip data. A sensor contract may end. A platform may close an API. A ministry may revise a collection method. The American Planning Association’s 2026 Trend Report for Planners explicitly identifies disappearing data as a planning concern. That matters because an analytical system that depends on one fragile source is not resilient, however sophisticated it looks.
7. Trust gaps
Communities may distrust official data because earlier plans ignored their experience. Agencies may distrust community reports because methods are inconsistent. Planners may distrust vendor models because the assumptions are proprietary. Trust is itself part of the data infrastructure.
The first question is not “How much data do we have?”
The better question is: What decision are we trying to make?
A town does not need perfect knowledge before every action. It needs evidence proportionate to the consequence of the action.
Repainting temporary lane markings is reversible. Reserving a future rail corridor is difficult to reverse once buildings occupy the land. Locating a playground can be adjusted later. Approving thousands of homes in a flood-prone basin can lock exposure into the urban structure for generations.
The more irreversible the decision, the more demanding the evidence standard should become.
This is a powerful way to manage uncertainty. Instead of insisting that every question receive the same quantity of analysis, planning can match the evidence burden to the cost of being wrong.
Build an evidence ladder, not a single-source dependency
No source sees the whole town. Good planning therefore combines sources that fail in different ways.
- Census and household surveys are strong for population structure but may be infrequent.
- Administrative records can be current but reflect only people who interact with formal systems.
- Building and land records describe legal parcels and development but not necessarily lived use.
- Transit, traffic and mobility data reveal movement but can miss walking quality, informal transport or people without devices.
- Remote sensing can show land cover, expansion, heat and physical change while revealing little about household circumstances.
- Sensors can measure conditions frequently but only where equipment exists and remains calibrated.
- Interviews, observation and participatory mapping can expose mechanisms that quantitative data misses.
- Research literature can provide tested relationships, but transferring findings between places requires care.
The aim is triangulation. If several independent sources point in the same direction, confidence rises. If they disagree, the disagreement becomes information.
Suppose a neighbourhood appears well served by parks on a GIS layer, yet resident surveys report poor access. That is not a reason to discard one source. It is a reason to investigate. Perhaps the park entrances are poorly located. Perhaps a major road cuts the neighbourhood in two. Perhaps the parks flood after rain. Perhaps the available space is technically public but practically unusable.
The contradiction tells the planner where the model is thin.
Denominators decide what a problem looks like
One of the quietest ways to distort planning is to choose the wrong denominator.
Ten road injuries may sound high or low. Ten injuries per year among 1,000 daily crossings is different from ten injuries among 100,000. A district with 50 hectares of green space may sound generous until population density is considered. A town with 20 clinics may be well supplied overall and poorly supplied where elderly residents actually live.
Counts answer “how many?” Rates answer “relative to what?” Accessibility measures answer “reachable by whom, from where, in what time, under what conditions?”
Planning becomes more intelligent when it moves beyond totals to exposure, access and distribution.
The average town is often a town that nobody lives in
Averages smooth variation. That is sometimes useful and sometimes destructive.
If average travel time falls while the longest commutes rise sharply for low-income workers, the town has not improved equally. If average tree canopy increases because wealthy districts become greener while heat-vulnerable areas lose shade, the average hides the planning failure.
The UN-Habitat SDG 11 Global Report 2026 highlights continuing urban-data gaps, particularly the need for disaggregated information at city and neighbourhood scales to understand inequality. This is not a technical footnote. Distribution is often the policy.
Good planning therefore asks for the spread, not merely the mean: who is above it, who is below it, and whether the gap matters.
Make uncertainty visible
Planning documents often communicate a single forecast because a single number is easy to draw. Yet many important variables are better represented as ranges.
Population might grow slowly, moderately or quickly. Sea level, insurance costs, construction prices, migration patterns and technology adoption may follow different paths. The point of scenarios is not to guess which future will occur. It is to test whether a plan remains workable across several plausible futures.
This changes the planning conversation. Instead of arguing over whether a forecast is “right,” planners can ask what decisions are robust if the forecast is wrong.
A corridor reservation that remains useful under three growth scenarios is more robust than one justified only by a single optimistic projection. A drainage upgrade that performs reasonably under a range of storm intensities may be preferable to a highly tuned design with no margin.
Uncertainty should be mapped just as deliberately as land use.
Create a confidence layer
Imagine a planning map with two layers. The first shows the result. The second shows confidence in the result.
A population estimate may be high-confidence in one district and low-confidence in another. A pedestrian demand model may be strong near counted intersections and weak on informal paths. A heat map derived from remote sensing may be spatially detailed but limited in representing indoor exposure.
A confidence layer forces the planning team to confront uncertainty explicitly. It also tells researchers where new measurement will have the highest value.
This is better than attaching a generic disclaimer to the end of a report. A disclaimer says uncertainty exists. A confidence map says where it exists and whether it matters.
Proxies are useful servants and dangerous masters
When direct measurement is difficult, planning uses proxies. Night-time light may approximate economic activity. Mobile-device presence may approximate footfall. Distance to a station may approximate transit access. Tree canopy may approximate shade.
A proxy is not the thing itself.
Distance to a station does not reveal service frequency, cost, accessibility, crowding or whether the route to the station is safe. Mobile data under-represents people without compatible devices and may be shaped by platform coverage. Property price can signal desirability while also reflecting speculation, exclusion and credit conditions.
The disciplined question is: what mechanism connects the proxy to the phenomenon we care about, and under what conditions does that relationship break?
Data lineage is part of planning quality
A number without provenance is a rumour wearing a spreadsheet.
For important planning indicators, a team should be able to answer:
- Who collected the data?
- For what original purpose?
- When was it collected?
- How was the sample constructed?
- What geography does it represent?
- What definitions were used?
- What transformations were applied?
- What records were excluded?
- What model generated any derived values?
- Can the process be reproduced later?
This chain is data lineage. It allows another planner to understand how the number reached the map.
Lineage also protects institutional memory. Staff change. Consultants leave. Software is replaced. If the logic of a planning indicator exists only in one analyst’s laptop, the town has not really acquired knowledge.
Privacy is a design constraint, not an obstacle to be bypassed
Fine-grained data can improve planning and harm people if used carelessly.
Location traces can reveal routines. Administrative records can expose vulnerable households. Small-area reporting can unintentionally identify individuals. A planning agency therefore needs to ask not only whether data is useful but whether its collection and use are proportionate.
Useful principles include data minimisation, aggregation, controlled access, clear retention rules, purpose limitation and documentation of who can see what. The strongest planning system is not the one that knows everything about everyone. It is the one that knows enough to make good public decisions without creating unnecessary surveillance.
Lived experience is evidence, but it must be structured
Residents know things that official datasets cannot know. They know which underpass feels unsafe after 9 p.m., which bus stop floods, where delivery riders cluster, which shortcut children actually use and which public space is nominally open but socially closed.
That knowledge becomes more useful when collection is systematic.
Participatory mapping can record locations consistently. Walk audits can use repeated criteria. Interviews can sample different groups rather than only the loudest voices in a meeting. Photovoice, intercept surveys and observation can document patterns. Community reporting can be validated against administrative and spatial data.
The goal is not to choose between quantitative data and lived experience. It is to use each to test the other.
This connects directly to TPW-0021 — The Listening Town. Participation improves planning when local knowledge changes the model rather than being attached to it as decoration.
Artificial intelligence does not remove the data gap
AI can classify imagery, detect patterns, summarise records, generate scenarios and accelerate analysis. It can also scale errors.
If training data excludes a population, automation can reproduce the exclusion faster. If a model infers land use from imagery, mixed or informal uses may be misclassified. If an algorithm predicts demand from past demand, it may underestimate places where poor service suppressed use. If a generative system produces confident prose, the confidence of the sentence may exceed the confidence of the evidence.
The correct question is not “Can AI predict this?” It is “What evidence did the system use, how uncertain is the result, and what public decision will follow from it?”
The APA Foresight programme treats AI governance and changing data conditions as planning issues precisely because technology changes the institutional environment, not merely the speed of calculation.
Beware the dashboard that becomes the objective
What gets measured can begin to shape what gets managed.
This is useful when indicators align with public goals. It becomes dangerous when teams optimise what is easy to measure instead of what matters.
A city can increase kilometres of cycle lane without creating a coherent cycling network. It can increase housing approvals without producing completed homes. It can add green-space acreage while failing to provide usable shade near vulnerable populations. It can reduce average travel time while worsening access for shift workers.
Every indicator should therefore have a counter-question: what important outcome could improve or deteriorate while this metric moves in the desired direction?
Plan for discontinuity
Data systems should be designed on the assumption that some sources will fail.
If a transport model depends on one commercial dataset, create a fallback. If a sensor network supplies a critical indicator, preserve manual or administrative alternatives. If a methodology changes, overlap old and new series long enough to understand the break. Archive metadata, codebooks and versions.
This is the informational equivalent of redundancy in physical infrastructure.
A resilient town needs resilient evidence.
The decision matrix: uncertainty × consequence
A simple operating model is to place decisions on two axes.
- Low uncertainty, low consequence: proceed and monitor.
- Low uncertainty, high consequence: proceed only with strong verification and governance.
- High uncertainty, low consequence: experiment, pilot and learn.
- High uncertainty, high consequence: delay irreversible commitment, gather more evidence, preserve options and use scenarios.
This turns “we need more data” from an automatic excuse into a disciplined decision rule. Sometimes more data is essential. Sometimes a pilot is better. Sometimes the cost of delay is larger than the value of additional certainty.
The value of a reversible plan
When uncertainty is high, flexibility has value.
A street can be designed so kerb space can change use. A public building can reserve structural capacity for future adaptation. A parcel can be temporarily activated while long-term demand becomes clearer. A utility corridor can preserve space for future services even before the technology is known.
The principle is simple: uncertainty does not always require inaction. It often requires option value.
This links the data problem to The Reserve Map. The future cannot be known perfectly, but choices can be protected.
How to audit a planning dataset before trusting it
Before a dataset becomes a planning argument, run a short audit.
- Purpose: What decision will this evidence influence?
- Origin: Who collected it, and why?
- Date: Does the time period match the decision?
- Coverage: Who or what is missing?
- Scale: Is the geography fine enough?
- Definition: Are terms consistent across sources and years?
- Bias: What behaviour or group is systematically under-represented?
- Transformation: What cleaning, weighting or modelling occurred?
- Uncertainty: Can error or confidence be estimated?
- Privacy: Is the proposed use proportionate and protected?
- Dependency: Could the source disappear?
- Validation: What independent evidence could challenge the result?
This audit will not make the data perfect. It makes the limitations legible.
From data-rich to knowledge-rich
The town with the most sensors is not necessarily the town that understands itself best.
Knowledge requires connection between evidence and mechanism. Why is a place changing? Which constraints are causal? Which groups experience the system differently? Which variable can policy actually influence? Which apparent trend is merely a measurement artifact?
This distinction matters because planning is not a reporting exercise. It is an intervention discipline.
A planner does not need to know everything about a town. The planner needs to know enough about the relevant system to change it without creating a worse problem somewhere else.
The Data Gap in the wider Town Planning series
Several earlier articles describe systems that become stronger when uncertainty is handled explicitly. The Learning Town explains feedback and adaptation. The Digital Shadow examines simulation and digital twins. The Equity Audit asks who gains and who carries burdens. The Plan Stack shows how evidence becomes formal decisions at different scales.
The Data Gap sits underneath all of them. If the evidence base is weak, hidden or unstable, the sophistication of the planning machinery above it cannot rescue the result.
A town is allowed to say “we do not know yet”
Institutional confidence is often mistaken for institutional competence. They are not the same.
A competent planning system can say that evidence is incomplete, specify what is uncertain, identify what would change the decision and choose an action that preserves room to learn.
That is not weakness. It is disciplined humility.
The alternative is false precision: a forecast with decimal places, a map with sharp boundaries and a plan whose apparent certainty is greater than the knowledge supporting it.
Town planning works when it can distinguish a strong signal from a convenient number.
And sometimes the most important thing on the map is the part that is still blank.
The value-of-information test
More data is not automatically worth collecting. Measurement consumes money, staff time, public attention and sometimes privacy. The useful question is whether additional information is likely to change the decision.
Imagine a town choosing between two alignments for a new bus corridor. Existing data already shows that one alignment reaches substantially more residents, connects to the rail network and avoids a flood-prone road. A six-month survey might refine the passenger forecast, but if no plausible result would reverse the preferred alignment, waiting for perfect information has little value. The town should proceed and monitor.
Now imagine the two alignments are close in cost and accessibility, but one may displace small businesses while the other may create dangerous school-crossing conflicts. In that case, targeted fieldwork, interviews and movement counts may have high value because the evidence could change the choice. The correct research programme is therefore decision-shaped: spend measurement effort where uncertainty and consequence intersect.
Nowcasting and forecasting solve different problems
Planning frequently mixes two questions that should be separated. Nowcasting asks what is happening in the town now, especially when official statistics arrive slowly. Forecasting asks what might happen later. The first is an observation problem; the second is a model of change.
Near-real-time indicators can improve nowcasting. Transit taps, electricity demand, school enrolment, building completion records, vacancy observations and mobility counts may reveal change faster than a census. But fast data can be noisy and unrepresentative. A sudden drop in mobile-device activity may reflect a platform change rather than a population movement.
Forecasts require another discipline. They should state assumptions about births, migration, household size, employment, construction, climate and behaviour. A model that fits yesterday beautifully can still fail tomorrow if the mechanism changes. Town planning therefore benefits from a chain of evidence: current observation, explanation of the mechanism, alternative futures and a review point where the forecast is compared with what actually happened.
Red-team the model before the city is built around it
Important planning models deserve adversarial review. A red team does not begin by asking how to prove the preferred plan. It asks what would have to be true for the model to be misleading.
If a pedestrian model predicts low demand, did poor crossings suppress existing walking? If a school forecast predicts falling enrolment, does it assume household composition will remain unchanged after thousands of new homes are built? If a flood model shows acceptable risk, what happens when drains are partially blocked or several failures occur together? If a retail-demand model relies on card spending, which cash-based or informal activities are missing?
This kind of challenge is especially important when a model produces an apparently precise number. Precision can come from mathematics while accuracy still depends on assumptions. A useful review therefore tests sensitivity: change the assumptions that matter and observe whether the decision remains stable.
Version the town’s evidence
A plan can remain in force for years while the evidence beneath it changes. New census results arrive. A rail line opens. A major employer moves. A flood map is revised. A planning department should therefore know not only which dataset it uses but which version of that dataset supported each decision.
Versioning creates an audit trail: source, date, definition, transformation, model version and decision date. When a result later looks wrong, the institution can discover whether the problem came from the source data, the model, the assumptions or a genuine change in the world.
This matters for public trust as well. A resident challenging a decision should be able to see the evidence that existed when the decision was made, rather than a moving set of numbers reconstructed years later. Reproducibility is not only a scientific virtue. It is a governance virtue.
When missing data should slow a decision
Some gaps are too important to work around. A project should slow down when a missing variable could reveal severe irreversible harm, when affected groups are systematically absent from the evidence, when the decision would close future options, or when the model cannot be independently checked.
Consider a new settlement proposed on land with uncertain groundwater behaviour. If the uncertainty concerns only the exact landscaping budget, construction can proceed while details are refined. If the uncertainty concerns whether widespread subsidence could damage buildings and flood defences, the information gap is decision-critical. The town should not confuse schedule pressure with evidence.
The same principle applies socially. If a redevelopment plan lacks reliable information on tenants, informal workers or care networks that could be displaced, the gap is not an administrative inconvenience. It concerns people who may bear irreversible cost. The evidence programme must be repaired before the plan is treated as complete.
When missing data should not become an excuse for paralysis
The opposite failure also occurs. Institutions can keep requesting studies because uncertainty provides political shelter from making a difficult choice. Climate adaptation, road safety and housing shortages can worsen while analysis continues.
A disciplined plan distinguishes unknowns that could change the decision from unknowns that merely change the decimal places. It uses pilots when actions are reversible, stages investment where learning is expected, monitors leading indicators and specifies triggers for changing course.
This turns uncertainty into an operating system. The question is no longer “Do we know enough to be certain?” Town planning almost never offers certainty. The question becomes “Do we know enough to make this decision responsibly, and have we designed a way to learn before the next irreversible step?”
The public should be able to see the unknowns
Planning consultation becomes more credible when uncertainty is communicated before a decision rather than discovered after it. A useful public document can distinguish observed facts, estimates, assumptions, model outputs and unresolved questions. These categories should not be visually identical.
That does not weaken a plan. It tells residents where evidence is strong and where judgement is doing more work. It also makes participation more useful. Instead of asking the public to comment on a finished-looking map, planners can ask targeted questions where local knowledge may actually change the result.
A town that marks uncertainty honestly can still act decisively. In fact, it often acts better because it knows which assumptions must be watched after implementation.
The mature planning system is not the one with no blanks
The mature system knows which blanks matter, which can be tolerated, which require another measurement and which should be protected as uncertainty rather than filled with false confidence.
That distinction is a form of competence. The map remains useful precisely because the institution understands where the map stops.