A city environmental dashboard loses its connection over a long weekend. During the outage, the data publisher corrects a historical station location, replaces several data objects and updates the associated quality flags. When the dashboard reconnects, a new notification arrives—but a reliable client cannot assume that this is the only change it missed. The gold standard of urban environmental API offline consumer correction reconciliation is to establish the current authoritative data and metadata state after an interruption, compare it with the consumer’s last verified state, repair affected local records and ensure that older messages cannot restore superseded information.
Quick answer: Persist a verifiable checkpoint of what the client actually processed; reconnect to the authoritative catalogue or service rather than assume notifications are complete; discover relevant corrections and superseded versions; fetch and validate required data objects; reconcile by stable object identity and version rules; apply updates and deletions safely and idempotently; rebuild affected derived outputs; and record which revisions have been verified. A successful network reconnection is not proof that stored data are current.
Find the part you need
- Why a missed correction matters more than a missed refresh
- Messages, data objects and dataset state are distinct
- What a reliable checkpoint contains
- Worked three-revision outage example
- A safe reconciliation sequence
- Duplicates and out-of-order corrections
- Historical metadata and time-aware joins
- Offline files and derived products
- Recovery after a partially completed update
- Monitoring, verification and closure
- Break-tests
- Frequently asked questions
- WMO context and connected Gold Standard reading
1. Offline consumers may miss the part that changes meaning
Many environmental applications receive repeated updates: fresh temperature, rainfall or air-quality observations, changed station metadata, quality flags and corrected historical data. An offline client might simply show old values for several hours, but the harder problem arises when the publisher changes the interpretation of data already stored locally. For example, an incorrect station coordinate may be corrected retroactively, or a previously published observation may be marked invalid after review.
When the client reconnects, downloading the latest hourly reading alone does not resolve those historical changes. A graph may still place an earlier observation at the wrong location, or an analytical summary may continue using a value that the publisher superseded. The consumer must therefore distinguish catching up with new observations from reconciling changes to old observations or metadata.
Push notifications are useful, but their delivery history may be incomplete from the consumer’s point of view after an extended outage. Messages can expire under the broker’s retention policy, or the consumer may not have been subscribed correctly during the interruption. A resilient design supports a separate authoritative state query or archive reconciliation rather than make long-term correctness depend on a perfect notification stream.
This is the consumer-side companion to the Gold Standard of Correction Propagation Verification. The publisher tries to ensure the correction reaches important users; the offline consumer designs its own return-to-service process so that it can recover when an event was missed. Both must be explicit about what they can actually prove.
2. An event, a resource and a dataset revision are not the same identity
A notification event tells the client that a resource may need attention. The referenced data object is the actual resource. The dataset’s metadata describes the broader collection, its meaning and applicable conditions. They can each have different identifiers. Confusing them creates subtle problems, such as importing the same data twice from two distribution servers or treating a fresh message as a new observation when it merely advertises an existing object.
The World Meteorological Organization’s approved WIS 2.0 Guide explains that notifications can be republished by multiple Global Brokers and Global Caches. It distinguishes notification message identity from the identity of the referenced data object and describes deduplication and replacement handling. Those requirements govern participating WIS 2.0 systems, not automatically every municipal API, but they illustrate why consumer design should keep event identity and data identity separate.
A single data object can be advertised more than once. Conversely, a replacement object may supersede an earlier resource even when the observation’s station and timestamp remain the same. An API endpoint that returns the latest value needs version or provenance information so the consumer can identify whether its local copy remains authoritative. A proper reconciliation compares resource semantics, not only the date of the most recently received notification.
| Identifier | What it identifies | Common error |
|---|---|---|
| Notification ID | A particular published message | Treating repeated advertisements as separate data objects. |
| Data object ID | The referenced observation file or resource | Importing duplicate copies from alternative caches. |
| Dataset metadata ID | The dataset’s discovery and definition record | Assuming a change to metadata is a new sensor observation. |
| Revision or publication version | An authoritative state for a resource or dataset | Applying a superseded revision after a newer one. |
| Station or sensor ID | The observing asset or station relationship | Assuming current station metadata always applied to history. |
A local checkpoint should reflect which resource versions were successfully incorporated, not merely the last message the client saw. The system must remain able to tell the difference between ‘received’, ‘downloaded’, ‘validated’, ‘stored’ and ‘used in the published output’.
3. A checkpoint is evidence of processed state
Imagine a client persisting only ‘last notification seen: 11:05’. That value says little about whether the referenced object was downloaded, whether its integrity was checked, or whether the database transaction committed. If the client crashes after receiving the notification but before saving the data, it might skip the resource on restart. A more reliable checkpoint reflects what the system successfully processed according to its rules.
The appropriate checkpoint may include dataset identity, authoritative metadata revision, processed resource versions or object identifiers, a bounded event cursor when the messaging system supports one, last successful reconciliation time and status of the derived outputs. The exact fields depend on architecture and data volume. Do not store a giant list of every event forever merely to prove recent processing; use suitable versioned snapshots or compact state indexes where appropriate.
A checkpoint should distinguish the last observed server response from last verified complete local state. Only advance the latter after the relevant local writes, validation and necessary consistency checks have succeeded. A crash between those steps should lead to safe retry rather than a false claim that the client is current.
Time is not a reliable revision rule by itself if the publisher’s specification uses another version scheme. A newer HTTP request timestamp can return an older cached object. A later event may refer to a different resource and not replace an earlier one. Follow documented resource identity and replacement semantics.
4. Worked example: a dashboard misses three important updates
Consider a fictional urban-monitoring service publishing hourly temperature observations. A downstream dashboard is current at dataset revision 20 at 08:00 Monday, then loses connectivity until Thursday. During the interruption the producer issues three material changes to station ST-042. The illustrative dates and revision numbers below do not describe any official Singapore dataset.
| Event during outage | Published change | Why it matters |
|---|---|---|
| Revision 21, Monday afternoon | Corrects an old ST-042 station coordinate for a specified historical period. | Previously stored map points may have incorrect geographic attribution. |
| Revision 22, Tuesday | Replaces one observation value after a verified processing error. | Derived daily mean may need rebuilding. |
| Revision 23, Wednesday | Marks a different observation invalid and updates quality metadata. | Charts and alerts need to respect the new quality status. |
On Thursday the dashboard receives a new hourly reading under revision 23. A naive client downloads the reading and sets its displayed status to ‘live’. Its older stored records remain associated with revision 20. The dashboard may therefore combine a current observation with an incorrect historical map, a superseded value and an invalid observation still shown as measured. Freshness of one new record is not correctness of the stored dataset.
A robust client starts by identifying its verified checkpoint: revision 20. It queries the authoritative resource catalogue or update history for changes since that state, subject to the provider’s supported interface. If the provider exposes a complete versioned snapshot instead of replayable individual events, the client can compare the authoritative snapshot with its local objects. The important requirement is reconciliation with a trustworthy source, not reliance on one particular messaging technology.
The client discovers that ST-042 historical locations need rebuilding, one data object was replaced and another was invalidated. It downloads or references the current approved resources, verifies the relevant identifiers, time intervals and quality states, and applies changes without silently changing unrelated stations. It then recomputes the daily mean and affected map layer, with a record that both outputs now use revision 23.
One corrected value can change a daily average
For illustration, suppose a daily four-reading series originally contained 30.0°C, 31.0°C, 33.0°C and 32.0°C. The mean was (30 + 31 + 33 + 32) ÷ 4 = 31.5°C. The verified correction changes the third reading from 33.0°C to 32.0°C, making the revised mean (30 + 31 + 32 + 32) ÷ 4 = 31.25°C. These invented values show why a client must rebuild a derived summary after an old resource changes.
The next day’s invalidated reading raises a different issue. Whether it should be excluded from a mean, represented as missing, or retained with a quality warning depends on the dataset’s documented meaning and the intended analysis. The consumer must not simply turn an invalid observation into zero. The correction process should preserve the authoritative quality status and any approved calculation rules.
When recovery completes, the dashboard should show the correct historical site locations for the relevant periods, the corrected daily summary and an appropriate explanation of the invalid observation. It can also display the last successful reconciliation time. Calling it fully reconciled before those affected outputs are verified would overstate the evidence.
5. An offline-recovery sequence should have testable stages
| Stage | What the client does | Evidence of success |
|---|---|---|
| Reconnect | Establish secure access to relevant authoritative source. | Service authenticated and dataset identified. |
| Discover | Compare verified checkpoint with current state and history. | Scope of missing updates known or full resync selected. |
| Fetch | Obtain required revised objects or versioned snapshot. | Resource identity and integrity checks pass. |
| Validate | Confirm schema, timestamps, station scope and quality semantics. | No malformed or conflicting correction silently accepted. |
| Apply | Perform idempotent local updates under controlled commit. | Superseded objects are not duplicated or resurrected. |
| Rebuild | Recompute derived charts, maps and summaries that depend on changes. | Affected outputs reflect the authoritative version. |
| Commit checkpoint | Advance the complete-state marker after successful checks. | Restart can safely resume without loss. |
| Communicate | Show status, exceptions and remaining uncertainty. | Users are not misled about stale or partial data. |
Sometimes an incremental history is incomplete or unavailable after a long outage. The safe option may be a full authoritative resynchronisation of the affected dataset or time interval. A provider that guarantees only a short update-retention window should document it clearly. The client should know when its last checkpoint is too old for incremental catch-up and stop trying to invent missing event history.
Avoid making a full resynchronisation destructive by default. Preserve necessary prior versions or audit references under retention rules so the corrected state can be explained. If historical observations were valid in their earlier context, they should not be erased simply because metadata changed later. The task is to restore correct interpretation, not to wipe the past.
6. Duplicates and out-of-order delivery need separate handling
A reconnecting client can receive the same notification more than once from a broker or alternative cache. If it treats every copy as a new event, it might import duplicate observations and inflate counts. The approved WIS 2.0 Guide discusses deduplicating notifications by message identity and data resources by data identity. In a different API, use that provider’s documented rules, but preserve the distinction.
Messages can also arrive out of order. A client might receive an older replacement notification after it has already applied a newer one. A naive ‘last message wins’ rule can revert the resource to an outdated state. The replacement policy must use an authoritative version or publication-order relation applicable to that resource. An event’s time of arrival at the consumer is not automatically the publication or validity time.
An object can be corrected after it was first stored; the consumer should apply the corrected version only to the affected object, not overwrite unrelated records with older snapshot values. Consistency checks need to confirm whether the publisher uses complete snapshots, partial corrections or explicit deletions. A partial update should not be interpreted as evidence that every omitted field should become null.
A minimal conceptual reconciliation rule
for each authoritative resource change:
identify dataset and object
read its documented version or replacement relation
if the local object is already that version:
do nothing
else if the change supersedes the local version:
validate and apply it atomically
else if the change is older or unrelated:
do not overwrite the newer local object
rebuild dependent outputs
advance verified checkpoint only after completionThis is pseudocode, not a universal WIS2 algorithm or executable specification. Real implementations must follow actual provider contracts and handle deletes, incompatible schemas, retention, parallel processing and transactional failure modes. The central invariant is that a retry cannot make the state less correct.
7. Historic station metadata needs time-aware reconciliation
A station’s current coordinates or instrument height may differ from the conditions under which old measurements were made. If a client reconnects and fetches only today’s station description, then joins that description to all stored historical observations, it may corrupt geographic interpretation. Reconciliation must respect the metadata valid at each observation time where the dataset’s semantics require it.
The problem can appear after a relocation, sensor replacement, corrected clock offset, change in reporting interval or retrospective quality review. Station identity alone is not enough. A versioned site-history record helps a client determine whether each observation belongs to the earlier or later operating period. An authoritative correction can change historical attribution without changing the raw measured value.
When the effective time of a station change is uncertain, do not manufacture a precise boundary to make the join convenient. Use the data steward’s approved uncertainty or affected-interval representation. An offline consumer should not independently invent a calibration or geographic adjustment merely because it missed the notification describing the change.
A historical reprocessing task may touch years of observations, while the live dashboard’s current reading remains unaffected. The client can prioritise safe current service and schedule a controlled backfill under approved conditions, but should label incomplete historical products until reconciliation is verified.
8. Static exports and offline caches may have their own owners
Some clients are offline-first applications that maintain local databases. Others run periodic batch imports, render static tiles or export CSV files for researchers. An API’s current response cannot automatically update a PDF, a downloaded spreadsheet or an embedded graphic. A reconciliation programme should distinguish assets that can be updated programmatically from those requiring a versioned replacement notice.
A mobile app may continue displaying stored values while reconnecting. That can be useful if clearly labelled as historical or last known, but it is misleading to display a real-time safety implication when the values are stale or corrected. Where relevant, show last observation time and last successful dataset reconciliation separately. A healthy network signal does not mean the information on screen has caught up.
Caches can be keyed by dataset revision, station and time interval to reduce ambiguity. But implementation design must avoid explosive storage growth or unintended disclosure. Rebuilding only affected derived products may be appropriate when lineage is reliable. Where lineage is incomplete, a broader controlled rebuild might be safer than pretending a narrow update covered all dependencies.
Public communications should explain supersession. A stable correction notice can tell users which older export or report was affected and where the revised product can be obtained. Historical versions may remain available for reproducibility with clear status labels. The goal is not to erase the existence of an error but to prevent the old version being mistaken for current authority.
9. Recover from partial updates without creating a second error
Imagine the client downloads revision 23 and updates its observation table, but crashes before updating the map. Its data table is current while the visible map is stale. If the checkpoint is marked complete prematurely, the next restart may skip the map rebuild. A robust design records the staged recovery status and advances the fully reconciled checkpoint only once required dependent products are verified or appropriately labelled as pending.
Atomic transactions can help protect local consistency, but a single database transaction may not include an external tile renderer or static report generator. The system can use a controlled state machine, versioned outputs or a replayable work queue. The exact choice depends on scale. The principle is that partial work should be detectable and resumable.
A second failure mode occurs when retries duplicate records. Suppose a data import succeeds but the acknowledgement is lost, and the job runs again. If resource identity and replacement rules are unclear, counts can double. Idempotent apply operations prevent repeated attempts from changing the correct final state. Duplicate handling should be tested explicitly, not assumed from the word ‘retry’.
A third failure arises from premature deletion. A correction notice may indicate that a resource is withdrawn, but the client should verify that the deletion is authoritative for the particular object and version. It should not delete a whole station’s valid historical data because one measurement was invalidated.
10. Closure requires output evidence, not only an empty message queue
Operational indicators can include the age of the last verified checkpoint, the difference between authoritative and local dataset revisions, unresolved corrections, failed object downloads, quality-flag mismatches, duplicate-suppression events and the status of important derived products. These provide stronger assurance than a simple ‘consumer connected’ light.
| Indicator | What it establishes | What it does not establish |
|---|---|---|
| Message subscriber connected | The client has an active route to some updates. | All corrections during an outage were received. |
| Current revision discovered | The client knows the authoritative target state. | Affected local objects have been repaired. |
| Objects reconciled | Relevant source records match the target revision. | Every derived report has been rebuilt. |
| Derived outputs verified | Named maps and summaries reflect corrected inputs. | Every unknown external copy was updated. |
| Checkpoint committed | Recovery completed through the defined verification boundary. | The underlying producer’s scientific correction is necessarily right. |
A service can report honest partial status: ‘Current hourly readings available; historical station corrections are still rebuilding.’ That gives users a meaningful understanding of what can be relied upon. A silent partial recovery can be more harmful than a visible temporary limitation, especially when a chart is used for planning or public information.
Where the product informs consequential decisions, the responsible domain authority should define appropriate release and incident criteria. This general guide does not specify official Singapore environmental advisories or authorise replacing established public-health guidance with third-party estimates.
11. Break-tests for an offline client’s return to service
- A correction event was published while the subscriber was disconnected and has expired from the broker. Can the client still establish authoritative state?
- A data object is advertised by two separate caches. Does the importer avoid recording it twice?
- An older correction arrives after a newer one. Can the system prevent reversion?
- A station’s current location differs from its earlier location. Does historic mapping use valid-time metadata?
- One observation is invalidated rather than set to zero. Does the client preserve the distinction?
- The import commits but the map renderer crashes. Can recovery resume and keep the checkpoint honest?
- A complete snapshot contains no record for an object previously stored. Does the client follow documented deletion semantics rather than guessing?
- The mobile UI says ‘online’ while its underlying data revision is obsolete. Can users see the genuine state?
- The source correction changes only a quality flag. Does the dashboard update interpretation even though the displayed number is unchanged?
- The outage exceeds the incremental history retention period. Does the client take an approved full-resync path instead of pretending the missed events can be reconstructed?
The gold standard can tolerate interruptions. It cannot tolerate concealing uncertainty about whether the corrected information actually reached the user. A robust system makes the gaps detectable, the recovery repeatable and the status truthful.
12. Frequently asked questions
Does reconnecting to the API mean my data are current?
No. Connectivity is separate from whether missed corrections were processed and affected local outputs rebuilt. Verify the authoritative revision and actual consumer state.
Are push notifications guaranteed to include every event I missed?
Not universally. Delivery and retention guarantees depend on the system. A resilient client should support authoritative reconciliation when notification history is incomplete.
What is the difference between event ID and data ID?
An event ID identifies a particular notification. A data ID identifies the referenced resource. The same resource may be advertised by several messages, especially in distributed systems.
Should every outage trigger a complete re-download?
Not automatically. A reliable incremental catch-up may suffice within supported history. If that history is unavailable or inconsistent, a controlled full resynchronisation may be necessary.
What if a corrected value has an earlier observation timestamp?
An old observation can have a newer authoritative revision. Compare the documented replacement or version relation, not observation time alone.
Can the client overwrite old observations with the station’s current location?
That is often incorrect. Historical geographic attributes should reflect the effective station history where necessary for the intended interpretation.
Does this article specify the WMO WIS2 algorithm for all APIs?
No. WIS2 guidance is authoritative for its participating system; the pseudocode and municipal example are an educational adaptation requiring the local provider’s actual contract.
When can the consumer say recovery is complete?
When the defined authoritative resource scope, local state and material dependent outputs meet approved verification conditions, with any known exceptions documented.
13. Sources and connected Gold Standard reading
The WMO Guide to the Information System 2.0, approved June 2026 explains notification subscriptions, duplicate messages, data-resource identity and replacement behaviour within WIS2. Its discussion of multiple Global Brokers and Global Caches is valuable background for resilient consumers. It does not make every urban environmental API a WIS2 service or mandate the pseudocode in this guide.
Follow the connected eduKate pathway: Correction Propagation Verification concerns the publisher’s confidence that consumers adopted corrections; Corrected Dataset Release Validation examines source release; Historical Metadata Reprocessing covers valid-time corrections; and Station Identity Continuity explains stable observing-system identity.
Takeaway: A disconnected client can miss more than recent measurements. It can miss corrections that change what its stored history means. The gold standard makes reconnection a verified reconciliation of authoritative state—not merely an acknowledgement that the network is back.