An urban rainfall dashboard reports 8 mm in a two-hour period, but the sensor missed one fifteen-minute interval. The observed readings may be correct; the full two-hour total is not established. The gold standard of urban environmental API window completeness disclosure is to show how much of a defined reporting period was validly observed, distinguish missing evidence from measured zero, and communicate the limits of a partial aggregate before users treat it as complete.
Quick answer: Define the exact time window and expected observation method; count or measure the valid source coverage; separate unavailable, invalid and estimated intervals; calculate completeness using an appropriate denominator; label partial or provisional results clearly; reconcile late valid data through controlled revisions; and verify that APIs, CSV exports and dashboards carry the same scientific meaning. No universal completeness threshold is assumed.
Quick navigation
- A plausible total can hide an incomplete observation period
- Define the exact reporting window before checking coverage
- Expected records are a method-dependent denominator
- Worked example: seven valid quarters of a two-hour rainfall period
- A coverage percentage cannot fill the missing rainfall
- Late arrival can change completeness and the reported total
- A row that exists may still fail the completeness requirement
- Why record-count completeness fails with unequal durations
- A count of samples does not automatically describe elapsed coverage
- Define provisional, incomplete and final as documented states
- Transparency about thresholds requires care
- Completeness and timeliness are separate data-quality dimensions
- The API contract should expose why a window is incomplete
- Worked conceptual JSON response
- How missing intervals should appear to ordinary readers
- A release gate should test difficult windows
- A completeness scorecard that does not hide scientific gaps
- Break-tests for a supposedly complete period
- Frequently asked questions
- Sources and connected Gold Standard reading
A plausible total can hide an incomplete observation period
An urban rainfall API publishes ‘8 mm in the last two hours’. The number is the sum of recorded fifteen-minute accumulations, but the sensor was offline for one quarter-hour. Eight millimetres may be the amount recorded during the available intervals. It is not necessarily the total rainfall for the complete two-hour period. A headline that omits the gap can imply more certainty than the observations support.
Window completeness answers how much of an intended time period has adequate underlying evidence. This is related to, but distinct from, data freshness and quality. A value can be recent and correctly measured yet represent only part of an hour. A window can have all expected timestamps but contain readings invalidated by quality review. A service can update successfully while still lack a complete measurement record.
The gold standard is to make those conditions visible alongside the output. The producer should define what it expected to receive, what it actually received, what passed the applicable quality rules and what type of derived result can be reported. The consumer should not have to infer completeness from the smoothness of a graph or the presence of a numeric field.
Define the exact reporting window before checking coverage
A completeness assessment needs a defined time window: its start, end, time zone and inclusion convention. A ‘two-hour total ending at 10:00 Singapore time’ should correspond to a known physical interval. If one system uses a rolling two-hour window and another uses the current calendar-hour pair, their completeness denominators may differ even when both display ‘two hours’.
The source observations must also have correct temporal meaning. A rain gauge record timestamped at the end of a fifteen-minute interval represents an accumulation over that interval, not necessarily rainfall occurring at one instant. To judge coverage, reconstruct the physical intervals and determine which parts of the requested window were observed under the dataset’s valid method.
Time-zone handling matters for daily reporting. The same instant can have different UTC and Singapore local labels. A valid completeness check should preserve the intended physical interval, not compare date strings as though they were abstract labels. This builds on the separate Gold Standard of Aggregation Window Boundary Integrity, whose main concern is assigning observations to the right windows.
Expected records are a method-dependent denominator
If a sensor’s defined operating plan produces one valid fifteen-minute accumulation every quarter-hour, a two-hour period contains eight expected non-overlapping intervals. But this calculation applies only under that sampling plan. An event-driven device, an irregular research logger or an approved hourly aggregate does not share the same eight-slot denominator. Completeness must follow the dataset’s measurement method, not a universal count chosen for convenience.
Where the rate or interval changes during the reporting period, the expected number of records can change. A planned maintenance interval may be handled differently from unplanned downtime under the service’s actual rules. A physically absent observation should not be counted as measured merely because the system generated a placeholder row or copied a previous value.
A strong API contract states how expected coverage is defined. The calculation may use expected records, expected measurement duration, a validated sampling schedule or another domain-specific rule. A client should know which denominator was used, especially if it compares two sensors with different reporting cadences.
Worked example: seven valid quarters of a two-hour rainfall period
Consider a fictional city rainfall sensor with a defined sequence of eight fifteen-minute accumulation intervals from 08:00 to 10:00 Singapore time. The device recorded seven valid accumulations and was offline during 09:15–09:30. The observed values across the other intervals are 0, 2, 4, 1, 0, 1 and 0 millimetres. The numbers are invented teaching data, not actual NEA readings or a current weather report.
The seven valid intervals contribute a known observed accumulation of 0 + 2 + 4 + 1 + 0 + 1 + 0 = 8 mm. Because one fifteen-minute interval was not observed, the total for the complete two-hour period is not established by these measurements alone. Recording ‘0 mm’ for the missing interval would be unsupported. The device could have measured zero rain during that time, but it did not record the evidence necessary to say so.
The slot-based completeness is 7 ÷ 8 × 100% = 87.5%. Since all expected intervals are equal fifteen-minute durations, time coverage is also 7 × 15 = 105 observed minutes out of 120, or 87.5%. The equality between record and time coverage is a property of this illustrative uniform schedule, not a universal rule for irregularly sampled networks.
| Singapore-time interval | Rainfall recorded | Observation status |
|---|---|---|
| 08:00–08:15 | 0 mm | Valid zero |
| 08:15–08:30 | 2 mm | Valid |
| 08:30–08:45 | 4 mm | Valid |
| 08:45–09:00 | 1 mm | Valid |
| 09:00–09:15 | 0 mm | Valid zero |
| 09:15–09:30 | — | Missing; sensor offline |
| 09:30–09:45 | 1 mm | Valid |
| 09:45–10:00 | 0 mm | Valid zero |
| Observed partial sum | 8 mm | 7/8 valid intervals; 87.5% coverage |
A coverage percentage cannot fill the missing rainfall
Stating that coverage is 87.5% does not mean the missing 12.5% of time received 12.5% of the measured rainfall. Rainfall can be highly uneven. A short gap during a shower may contain more precipitation than a much longer dry interval. Scaling the observed 8 mm by 120/105 would yield roughly 9.14 mm, but that calculation assumes a uniform or representative rainfall rate that has not been established. It is not an observed two-hour total.
The trustworthy output can say: ‘8 mm observed across 105 of 120 minutes; one interval unavailable; full-period total not determined under this measurement method.’ A model might later estimate a missing interval using a documented approved method, but the estimate must be labelled as derived and carry its own uncertainty and provenance. An estimated total should not be silently substituted for the originally observed accumulation.
This distinction protects both public communication and downstream analytics. A dashboard can show a partial total and a clear coverage status rather than either invent zero rainfall or hide the known observations entirely. The choice of whether a partial product is useful depends on the intended user and applicable reporting method.
Late arrival can change completeness and the reported total
Suppose a validated recording for the missing 09:15–09:30 interval arrives later and shows 3 mm. The eight-slot completeness becomes 8/8, or 100%, under the fictional quality rules. The full observed two-hour total becomes 8 + 3 = 11 mm. The environment did not produce three millimetres of new rainfall after 10:00; the publisher received evidence later for rainfall that had already occurred.
The service should preserve the original provisional output and publish a traceable corrected version when appropriate. A consumer needs to know that the earlier 8 mm represented incomplete coverage and that the later 11 mm reflects an additional validated source observation. The change is a data availability and aggregation correction, not a new rain event in the current hour.
A late record might instead arrive but fail quality checks, in which case the physical interval could remain inadequately covered for the intended report. Receipt and validity are different facts. Completeness should be based on the approved eligibility definition rather than the number of messages received.
A row that exists may still fail the completeness requirement
An API could show eight rows for eight intervals, yet one row contains a stale repeated reading, an invalid quality flag or a value generated through a fallback estimate. Depending on the defined product, those may not count as valid observed coverage. A full table of timestamps is therefore not conclusive evidence of complete measurements.
Quality flags should travel with data and remain understandable to consumers. An estimated observation may be permissible for a specifically documented modelled product while not satisfying a requirement for directly observed reporting. The provider needs an explicit eligibility rule for each output rather than one hidden generic ‘not null’ check.
Separating physical observation, data validity and estimated fill prevents double counting and false assurance. A producer can report observed coverage, modelled coverage and unresolved gaps as distinct quantities when relevant. Do not combine them into a single reassuring 100% without explaining the scientific meaning.
Why record-count completeness fails with unequal durations
Consider two records, one representing ten minutes and another representing twenty minutes. If the reporting window is thirty minutes and both valid intervals are present, duration coverage can be complete. If only the ten-minute interval is present, one of two records exists, or 50% by record count, but only ten of thirty minutes are covered, or about 33.3% by duration. Those percentages answer different questions.
Now imagine a window with one one-minute record and one fifty-nine-minute aggregate. Treating the two equally in a completeness average would make little physical sense if the intended coverage concerns time. A reliable system should use the actual observation durations and the method-defined weighting where relevant. Irregular instantaneous samples can require different treatment again, because one reading does not automatically represent an interval of constant conditions.
The data contract should explain which measure of completeness is being published and what the records represent. Consumers need not understand a complex statistical model to interpret ‘105 of 120 minutes observed’, but the calculation should be scientifically justified.
A count of samples does not automatically describe elapsed coverage
An instrument might report an instantaneous temperature each minute, or a five-minute average every five minutes. For an hourly temperature product, the provider must define how those observations are expected to support the intended average. A count of sixty timestamps can look complete while clock drift, duplicated records or missing validity information undermines interpretation.
Some methods require scheduled observations at particular moments; others depend on time-weighted or interval-averaged data. The right completeness check follows the physical quantity and approved calculation method. A value described as a one-hour mean should not be generated by averaging whichever readings happened to arrive during the last hour unless that method is appropriate for the dataset.
A dashboard that displays only ’60/60′ may hide which minutes were missing, whether the same instant was counted twice or whether a late reading fell outside the intended interval. Add sufficient interval context and validity status for the user to understand the result.
Define provisional, incomplete and final as documented states
Different agencies and data services use different status labels. One useful conceptual distinction is provisional for a result that may change under a defined process, incomplete for a reporting window lacking required valid evidence, and final for a result that has met the provider’s actual finalisation rule. These are not universal NEA status codes and must not be presented as such.
A provisional value may be fully covered but await quality review. An incomplete window may already contain useful valid observations. A final product may later be formally corrected if new evidence establishes an earlier error. The statuses are related but not interchangeable. Their rules, revision history and consumer expectations should be documented.
Never treat ‘final’ as meaning that the physical world cannot be studied further or that a future data correction is impossible. It describes the product’s status under the current defined publishing procedure. A robust API can communicate later supersession without hiding the fact that an earlier final value was previously published.
Transparency about thresholds requires care
Some professional datasets prescribe minimum observation counts or coverage requirements before an aggregate can be published or called valid. Others apply variable-specific or purpose-specific methods. This article does not invent a universal 75%, 80% or 90% threshold for city rainfall, temperature or pollutant products. Use the actual authorised measurement and quality standard for each service.
A publisher should explain its chosen threshold, justification and consequence. For example, does failing the threshold prevent a daily indicator from appearing, label it provisional, or allow a partial observation summary with a warning? The answer depends on how the indicator will be used. A research exploratory chart and an official public-health advisory may need different safeguards.
Publishing a high percentage without context can still mislead. A window that is 95% covered but missing the crucial peak of a short storm may not adequately describe the entire event. Completeness and representativeness are different questions. Both matter when the result is used for consequential decisions.
Completeness and timeliness are separate data-quality dimensions
The World Meteorological Organization’s WIGOS Data Quality Monitoring System distinguishes availability, quality and timeliness in observing networks. A station can deliver a fully valid set of readings several hours late; the window might eventually be complete but not timely for an operational use. Conversely, a timely API can contain an incomplete or invalid window.
Track actual observation time, receipt time, release time and latest validated status where relevant. When a batch arrives late, update the appropriate historical window rather than attribute the past measurements to the time they arrived. A data product should explain whether a freshness problem, a coverage problem or a quality problem prevents a firm conclusion.
Using WMO’s general distinction does not mean every urban environmental API belongs to WIGOS or must implement the same thresholds. It is a scientifically useful framework for avoiding a one-dimensional ‘good data’ badge.
The API contract should expose why a window is incomplete
A machine consumer needs to know more than one boolean. Useful fields might identify the reporting interval, expected and valid observations, observed duration, coverage calculation, data-status label, quality eligibility rule and output revision. A human-readable note can describe the affected intervals or reason for a gap. The exact field names are an illustrative design, not a standard mandated by WMO or NEA.
Do not force every API to embed a large array of missing intervals inside every response if the service has another documented metadata route. But the information required to interpret an incomplete aggregate should remain discoverable, and clients should not invent completion based on a successful HTTP status. A request can succeed technically and return a partial result scientifically.
Where a number is an estimate or a backfilled correction, include provenance or an appropriate reference. A public client should be able to distinguish measured, derived and unknown quantities, even when its design uses a compact status indicator.
Worked conceptual JSON response
One illustrative response could expose 8 mm as a partial observed accumulation for the 08:00–10:00 window, 7 valid records of 8 expected, 105 observed minutes of 120, one missing interval and a provisional or incomplete status under this fictional method. A later revision might report 11 mm and complete coverage after validated backfill. The contract should prevent a consumer from mistaking these two states for two separate rain events.
An output revision identifier allows dashboards and exports to establish which result they are showing. A consumer can update the derived total without changing the raw readings for periods already observed. Where a detailed provenance record is maintained separately, the response can link to that authoritative record.
Preserve the actual meaning of absence. An absent field, null value, measured zero, invalid reading and provisional estimate must not be collapsed into the same numeric representation by a client library or spreadsheet import.
{
"dataset": "fictional-urban-rainfall",
"window_start": "2026-10-20T08:00:00+08:00",
"window_end": "2026-10-20T10:00:00+08:00",
"observation_unit": "mm",
"observed_accumulation": 8,
"expected_intervals": 8,
"valid_intervals": 7,
"observed_minutes": 105,
"expected_minutes": 120,
"coverage_percentage": 87.5,
"full_period_total_known": false,
"status": "incomplete",
"revision": "example-v1"
}How missing intervals should appear to ordinary readers
Readers need an immediate answer, not a wall of backend terminology. ‘Rain recorded from 08:00 to 10:00: 8 mm from 7 of 8 fifteen-minute intervals. One interval was unavailable; this is not a confirmed full-period total.’ That wording communicates the useful observation and its limit together.
A compact chart can show missing periods explicitly, with text labels in addition to colour. Do not draw an unbroken line across a gap without disclosing interpolation. Screen readers should be able to hear an appropriate status and understand which interval is unavailable. The Singapore Government Design System’s clarity principles are a useful communication reference even for local environmental displays.
Do not overstate uncertainty either. Seven observed intervals still provide seven valid measurements. A user can learn from those readings while recognising that the complete window remains unknown. Good disclosure makes the difference between partial evidence and no evidence clear.
A release gate should test difficult windows
Use synthetic fixtures at midnight boundaries, overlapping intervals, missing first or last periods, duplicates, late batches and quality-flag changes. Verify the expected denominator, valid count, covered duration and output status under the documented method. A test that checks only whether the numeric total looks plausible cannot catch a lost interval.
Include a case with seven valid equal-duration records, one missing record and a later validated backfill. The system should first produce an incomplete record with 87.5% coverage and later a versioned correction with full coverage and the appropriate revised sum. The earlier result should not be silently overwritten without provenance where historical accountability matters.
Tests should inspect API responses and actual public or institutional clients. A backend status of incomplete can be lost when a CSV exporter drops quality columns, or a mobile screen may display only the sum. The final user-visible interpretation is part of the output contract.
A completeness scorecard that does not hide scientific gaps
Useful indicators include proportion of windows meeting the approved observation requirements, actual observation duration, invalid and missing intervals, late-arrival correction frequency, number of outputs published without proper status and downstream consumers still using stale revisions. Reports should identify the relevant geographic and measurement context, not pool unrelated sensors into one meaningless success percentage.
An instrument may show excellent availability while an urban network samples only a narrow set of micro-environments. Temporal completeness cannot establish spatial representativeness. Likewise, 100% observed coverage does not prove every sensor is calibrated or all units and transformations are correct. Good data governance keeps those dimensions separate.
Use the scorecard to discover recurring gaps and support maintenance or processing improvement, not merely improve a headline percentage. A rise in known missingness may reflect better transparency after an audit. The goal is trustworthy products, not cosmetic completeness.
Break-tests for a supposedly complete period
A gauge is offline for fifteen minutes but the importer inserts zero. Can the system flag the unsupported measurement? An API has eight rows but one is invalid. Does the validity-based completeness drop? A late observation arrives after publication. Can the correct old window be revised without suggesting that new rainfall occurred? Two records overlap and duplicate one period. Does the expected coverage calculation identify the overlap?
Another test uses uneven ten-minute and twenty-minute intervals. Can the client distinguish 50% by record count from 33.3% of duration when only the ten-minute interval exists? A third test compares calendar-day and rolling-two-hour products. Can the system state which physical windows each covers rather than assume matching timestamp labels?
The highest-level challenge is a very short storm hidden inside a missing period. Does an 87.5% coverage statistic justify estimating the full rainfall total as 8 ÷ 0.875? Not without a scientifically justified model. A correct explanation identifies the unobserved amount as unknown.
Frequently asked questions
Does 87.5% completeness mean 87.5% of the rain was measured? No. It describes observed time or expected slots under the defined method. Rain may be uneven. Is missing rainfall the same as zero rainfall? No; zero requires an actual valid zero observation.
Can a complete window still be low quality? Yes. Observations may fail calibration or other quality requirements. Can a late record change yesterday’s total? Yes, when the publisher accepts additional valid evidence and releases an appropriate correction. It does not mean the physical event happened later.
Is there one official completeness threshold for every environmental variable? No universal threshold is asserted here. Apply actual provider and domain-specific rules. What should an API expose? Enough window, coverage, quality and version information to interpret the result accurately.
Does WMO WIGOS cover every city sensor? No. Its availability, quality and timeliness distinctions are informative but formal system scope varies. Can a partial result still be useful? Yes, if its observed amount and limitations are clear.
Sources and connected Gold Standard reading
The WMO WIGOS Data Quality Monitoring System, updated 5 June 2026, describes availability, quality and timeliness as monitoring dimensions. The NEA air-quality FAQs provide Singapore examples of indicators with distinct reporting periods and meanings. This article’s rainfall observations, statuses and coverage percentages are fictional illustrations, not official Singapore reporting criteria.
Read the connected eduKate articles: Late-Arrival Window Recalculation covers accepted backfill; Aggregation Window Boundary Integrity covers interval assignment; Temporal Aggregation Unit Consistency covers physical quantities; and Quality Flag Governance covers observation status.
Final principle: A plausible environmental total is not necessarily a complete total. Trustworthy APIs say exactly how much of the intended window was validly observed, what remains unknown and whether the published number may change after further evidence.