THE GOLD STANDARD OF HYDROLOGY · SPECIALIST STUDY
Migrate a forecast archive without confusing smaller files, intact bytes and preserved scientific meaning
A hydrological archive can survive a storage migration yet lose its meaning. Variables may change type, calendars may shift, missing-value codes may be rewritten, chunking may make common access impractical, or lossy quantisation may change threshold decisions while files still open successfully. Good migration keeps an immutable source, maps every schema and transformation, distinguishes lossless compression from precision loss, renews checksums without breaking provenance, and proves that representative forecasts remain reconstructible.
This guide belongs to the Hydrology subject root, the Gold Standard of Education, and the How X Works hub. It begins with a concrete decision, then tests whether the method still holds when the context changes.
Choose a reading route
Define the preservation claim · chapters 1, 2, 3
Map formats and transforms · chapters 4, 5
Budget precision and tolerance · chapters 6, 7, 8
Validate access and reconstruction · chapters 9, 10, 11
Renew custody and govern retirement · chapters 12, 13
The whole problem at a glance
| Question | Evidence | Unsafe shortcut |
|---|---|---|
| Are the bytes intact? | Checksums before and after each custody step | Assuming readable means unchanged |
| Are the values preserved? | Typed variable comparison and declared tolerance | Comparing file sizes |
| Is the meaning preserved? | Units, coordinates, calendar, ensemble and provenance checks | Matching variable names only |
| Can forecasts be reconstructed? | Representative rerun and access-path test | Opening one sample file |
Byte identity, numerical equivalence, semantic equivalence and practical reconstructibility need separate tests.
CHAPTER 1 OF 13 · Define the preservation claim
1. Define what the migration must preserve
State whether the archive supports exact byte custody, numerical analysis, forecast verification, legal or public accountability, model reruns, warning reconstruction or routine access. Identify variables, locations, lead times, ensembles, metadata and decisions that matter. Different purposes justify different formats and tolerances; write them before choosing compression so the result is not judged by storage savings alone.
Build the evidence record around the decision rather than the convenience of the software. Name the protected or measured unit, the version in force, the time boundary, the numerator and denominator, exclusions, missing cases and the person authorised to act. Keep raw observations separate from interpretation, and retain failed and rejected cases as well as successes. That separation lets a later reviewer distinguish a changed world from a changed judgement.
Convert the finding into a bounded action. Predefine the trigger, permitted response, owner, expiry, rollback and verification window. Keep one worked case that passes and one that fails. A system which stores only successful cases cannot demonstrate that its control detects the failure it was designed to contain.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Add a public-accountability use after planning a research-only migration; strengthen chronology, notices and immutable issued products. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 1 check · what should survive an audit
- State the decision and measurement unit in one sentence.
- Preserve version, timing, denominator, exclusions and missing-case evidence.
- Connect the finding to a bounded action, rollback and later verification.
CHAPTER 2 OF 13 · Define the preservation claim
2. Inventory formats, schemas, codecs and readers
Record every source format and version, variable type, dimension, coordinate reference, unit, calendar, missing-value convention, scale and offset, chunking, compression filter, checksum, software reader and external dependency. Include odd files and failed products rather than sampling only clean cases. A format name such as NetCDF does not uniquely describe its data model or encoding.
Build the evidence record around the decision rather than the convenience of the software. Name the protected or measured unit, the version in force, the time boundary, the numerator and denominator, exclusions, missing cases and the person authorised to act. Keep raw observations separate from interpretation, and retain failed and rejected cases as well as successes. That separation lets a later reviewer distinguish a changed world from a changed judgement.
Convert the finding into a bounded action. Predefine the trigger, permitted response, owner, expiry, rollback and verification window. Keep one worked case that passes and one that fails. A system which stores only successful cases cannot demonstrate that its control detects the failure it was designed to contain.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Discover classic NetCDF, NetCDF-4 and proprietary sidecars under one extension; route them through distinct verified transforms. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 2 check · what should survive an audit
- State the decision and measurement unit in one sentence.
- Preserve version, timing, denominator, exclusions and missing-case evidence.
- Connect the finding to a bounded action, rollback and later verification.
CHAPTER 3 OF 13 · Define the preservation claim
3. Keep the source immutable and build transformation lineage
Create a read-only source manifest and independently verified copy before conversion. Give each target object a lineage record linking source checksum, converter code and version, parameters, environment, start and completion time, operator or service, warnings and target checksum. Never replace the only source in place. Failed attempts remain evidence for diagnosing later discrepancies.
Build the evidence record around the decision rather than the convenience of the software. Name the protected or measured unit, the version in force, the time boundary, the numerator and denominator, exclusions, missing cases and the person authorised to act. Keep raw observations separate from interpretation, and retain failed and rejected cases as well as successes. That separation lets a later reviewer distinguish a changed world from a changed judgement.
Convert the finding into a bounded action. Predefine the trigger, permitted response, owner, expiry, rollback and verification window. Keep one worked case that passes and one that fails. A system which stores only successful cases cannot demonstrate that its control detects the failure it was designed to contain.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Restart a partially failed batch; prove which targets came from which converter version rather than relying on modification time. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 3 check · what should survive an audit
- State the decision and measurement unit in one sentence.
- Preserve version, timing, denominator, exclusions and missing-case evidence.
- Connect the finding to a bounded action, rollback and later verification.
CHAPTER 4 OF 13 · Map formats and transforms
4. Map schema and semantics explicitly
Create a source-to-target map for variable names, types, dimensions, coordinate axes, bounds, units, calendars, missing values, flags, ensemble members, probabilistic meaning and provenance. Validate decoded meaning, not just raw arrays. A calendar conversion or reordered dimension can produce plausible numbers assigned to the wrong valid time, basin or member.
Build the evidence record around the decision rather than the convenience of the software. Name the protected or measured unit, the version in force, the time boundary, the numerator and denominator, exclusions, missing cases and the person authorised to act. Keep raw observations separate from interpretation, and retain failed and rejected cases as well as successes. That separation lets a later reviewer distinguish a changed world from a changed judgement.
Convert the finding into a bounded action. Predefine the trigger, permitted response, owner, expiry, rollback and verification window. Keep one worked case that passes and one that fails. A system which stores only successful cases cannot demonstrate that its control detects the failure it was designed to contain.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Transpose member and lead dimensions in the target; a value multiset matches while forecast semantics fail. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 4 check · what should survive an audit
- State the decision and measurement unit in one sentence.
- Preserve version, timing, denominator, exclusions and missing-case evidence.
- Connect the finding to a bounded action, rollback and later verification.
CHAPTER 5 OF 13 · Map formats and transforms
5. Separate lossless compression from lossy quantisation
Deflate and compatible shuffle filters can reduce NetCDF storage without changing reconstructed values. Quantisation deliberately reduces precision before compression and is irreversible. Record the algorithm, parameters and affected variables in metadata and lineage. Never describe a smaller lossy target as merely “compressed,” because downstream users need to know that scientific values changed.
Build the evidence record around the decision rather than the convenience of the software. Name the protected or measured unit, the version in force, the time boundary, the numerator and denominator, exclusions, missing cases and the person authorised to act. Keep raw observations separate from interpretation, and retain failed and rejected cases as well as successes. That separation lets a later reviewer distinguish a changed world from a changed judgement.
Convert the finding into a bounded action. Predefine the trigger, permitted response, owner, expiry, rollback and verification window. Keep one worked case that passes and one that fails. A system which stores only successful cases cannot demonstrate that its control detects the failure it was designed to contain.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Apply the same codec name with a different significant-digit parameter; treat it as a new data product version. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 5 check · what should survive an audit
- State the decision and measurement unit in one sentence.
- Preserve version, timing, denominator, exclusions and missing-case evidence.
- Connect the finding to a bounded action, rollback and later verification.
Worked NetCDF migration
| Element | Source | Target validation |
|---|---|---|
| Discharge | float64, m3 s-1 | Same units; tolerance by variable purpose |
| Time | hours since epoch, Gregorian | Identical decoded valid times |
| Ensemble | member dimension 0–49 | Membership and order preserved |
| Compression | uncompressed | Deflate is lossless; optional quantisation tested separately |
| Fixity | SHA-256 source manifest | New target checksum linked to source and transform |
The example is illustrative; format choices depend on archive use, tools and policy.
CHAPTER 6 OF 13 · Budget precision and tolerance
6. Build a variable-specific precision budget
Decide acceptable error from measurement resolution, model precision, downstream calculations and decision thresholds. Coordinates, identifiers, flags and cumulative quantities may need exact preservation while large continuous fields tolerate bounded error. Test absolute, relative and tail errors, and protect near-zero values where relative error is unstable. One global decimal setting is rarely defensible.
Build the evidence record around the decision rather than the convenience of the software. Name the protected or measured unit, the version in force, the time boundary, the numerator and denominator, exclusions, missing cases and the person authorised to act. Keep raw observations separate from interpretation, and retain failed and rejected cases as well as successes. That separation lets a later reviewer distinguish a changed world from a changed judgement.
Convert the finding into a bounded action. Predefine the trigger, permitted response, owner, expiry, rollback and verification window. Keep one worked case that passes and one that fails. A system which stores only successful cases cannot demonstrate that its control detects the failure it was designed to contain.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Use a discharge tolerance for precipitation accumulation; rebuild the budget from the variable’s scale and downstream use. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 6 check · what should survive an audit
- State the decision and measurement unit in one sentence.
- Preserve version, timing, denominator, exclusions and missing-case evidence.
- Connect the finding to a bounded action, rollback and later verification.
CHAPTER 7 OF 13 · Budget precision and tolerance
7. Test threshold and rank sensitivity
Small numeric changes can flip exceedance, warning category, rank, tie or ensemble probability near a boundary. Construct cases just below and above operational thresholds, extreme events and low-flow conditions. Compare member counts, quantiles, probabilities and decisions. A low mean error does not protect a rare but consequential threshold crossing.
Build the evidence record around the decision rather than the convenience of the software. Name the protected or measured unit, the version in force, the time boundary, the numerator and denominator, exclusions, missing cases and the person authorised to act. Keep raw observations separate from interpretation, and retain failed and rejected cases as well as successes. That separation lets a later reviewer distinguish a changed world from a changed judgement.
Convert the finding into a bounded action. Predefine the trigger, permitted response, owner, expiry, rollback and verification window. Keep one worked case that passes and one that fails. A system which stores only successful cases cannot demonstrate that its control detects the failure it was designed to contain.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: A 0.1% error changes one member at a 10-of-50 warning rule; evaluate decision sensitivity, not only field RMSE. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 7 check · what should survive an audit
- State the decision and measurement unit in one sentence.
- Preserve version, timing, denominator, exclusions and missing-case evidence.
- Connect the finding to a bounded action, rollback and later verification.
CHAPTER 8 OF 13 · Budget precision and tolerance
8. Declare reconstruction tolerances before comparison
Use bitwise equality for truly lossless byte-preserving claims, exact decoded equality for lossless re-encoding where bytes differ, and predeclared numerical plus semantic tolerances for intentional precision change. State aggregation, spatial and temporal scope, missingness and maximum allowed failures. Do not tune a tolerance after seeing the largest discrepancy.
Build the evidence record around the decision rather than the convenience of the software. Name the protected or measured unit, the version in force, the time boundary, the numerator and denominator, exclusions, missing cases and the person authorised to act. Keep raw observations separate from interpretation, and retain failed and rejected cases as well as successes. That separation lets a later reviewer distinguish a changed world from a changed judgement.
Convert the finding into a bounded action. Predefine the trigger, permitted response, owner, expiry, rollback and verification window. Keep one worked case that passes and one that fails. A system which stores only successful cases cannot demonstrate that its control detects the failure it was designed to contain.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Move between numerical libraries that change floating-point reduction order; distinguish migration error from rerun nondeterminism. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 8 check · what should survive an audit
- State the decision and measurement unit in one sentence.
- Preserve version, timing, denominator, exclusions and missing-case evidence.
- Connect the finding to a bounded action, rollback and later verification.
CHAPTER 9 OF 13 · Validate access and reconstruction
9. Preserve metadata, provenance and discoverability
Carry global and variable attributes, conventions, history, institution, model version, issue and valid time, observation vintage, licence, citation and quality flags. Where a target format lacks a field, create a governed companion manifest with durable linkage. Validate catalogue indexing and search because perfectly preserved files that users cannot discover are operationally lost.
Build the evidence record around the decision rather than the convenience of the software. Name the protected or measured unit, the version in force, the time boundary, the numerator and denominator, exclusions, missing cases and the person authorised to act. Keep raw observations separate from interpretation, and retain failed and rejected cases as well as successes. That separation lets a later reviewer distinguish a changed world from a changed judgement.
Convert the finding into a bounded action. Predefine the trigger, permitted response, owner, expiry, rollback and verification window. Keep one worked case that passes and one that fails. A system which stores only successful cases cannot demonstrate that its control detects the failure it was designed to contain.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: The new catalogue drops a non-standard model-cycle attribute; map it to a controlled field or linked manifest. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 9 check · what should survive an audit
- State the decision and measurement unit in one sentence.
- Preserve version, timing, denominator, exclusions and missing-case evidence.
- Connect the finding to a bounded action, rollback and later verification.
Preservation validation matrix
| Test | Sample design | Failure response |
|---|---|---|
| Schema and metadata | Every variable and coordinate | Block migration batch |
| Value comparison | Full critical fields plus stratified bulk sample | Classify transform or corruption |
| Threshold decisions | Flood, low-flow and warning cases | Tighten precision or exclude lossy transform |
| Reader compatibility | Current and preservation readers | Keep conversion path or alternate copy |
| Reconstruction | Ordinary and extreme forecast packages | Retain source until repaired |
Averages alone can hide local precision loss near a decision threshold.
CHAPTER 10 OF 13 · Validate access and reconstruction
10. Renew checksums without breaking custody
Verify source checksums before reading, compute target checksums after durable write, and record the transform connecting them. A changed checksum is expected after re-encoding; it is not evidence of corruption when lineage and value tests pass. Choose current approved algorithms, preserve old manifests for historical custody and schedule fixity audits plus independent replicas.
Build the evidence record around the decision rather than the convenience of the software. Name the protected or measured unit, the version in force, the time boundary, the numerator and denominator, exclusions, missing cases and the person authorised to act. Keep raw observations separate from interpretation, and retain failed and rejected cases as well as successes. That separation lets a later reviewer distinguish a changed world from a changed judgement.
Convert the finding into a bounded action. Predefine the trigger, permitted response, owner, expiry, rollback and verification window. Keep one worked case that passes and one that fails. A system which stores only successful cases cannot demonstrate that its control detects the failure it was designed to contain.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Replace a deprecated checksum algorithm; retain the old digest and document the verified crosswalk to the new one. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 10 check · what should survive an audit
- State the decision and measurement unit in one sentence.
- Preserve version, timing, denominator, exclusions and missing-case evidence.
- Connect the finding to a bounded action, rollback and later verification.
CHAPTER 11 OF 13 · Validate access and reconstruction
11. Validate reader compatibility and access patterns
Test the preservation reader, current analysis stack, catalogue preview, subsetting service and representative downstream scripts. Chunking can improve one access pattern and make another costly; unsupported filters can make standards-compliant containers unreadable in ordinary tools. Measure retrieval, decode time and memory for real basin, lead-time and ensemble slices.
Build the evidence record around the decision rather than the convenience of the software. Name the protected or measured unit, the version in force, the time boundary, the numerator and denominator, exclusions, missing cases and the person authorised to act. Keep raw observations separate from interpretation, and retain failed and rejected cases as well as successes. That separation lets a later reviewer distinguish a changed world from a changed judgement.
Convert the finding into a bounded action. Predefine the trigger, permitted response, owner, expiry, rollback and verification window. Keep one worked case that passes and one that fails. A system which stores only successful cases cannot demonstrate that its control detects the failure it was designed to contain.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Optimise chunks for whole-grid time steps; test basin time series before accepting the layout. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 11 check · what should survive an audit
- State the decision and measurement unit in one sentence.
- Preserve version, timing, denominator, exclusions and missing-case evidence.
- Connect the finding to a bounded action, rollback and later verification.
CHAPTER 12 OF 13 · Renew custody and govern retirement
12. Run preservation and reconstruction drills
Select ordinary, extreme, sparse, high-resolution and historically important cases. Restore from the independent copy, verify checksums, open with designated readers, reconstruct a forecast or verification product, and compare value, semantics and decision outputs. Record time, missing dependencies and manual steps. Repeat after storage or software migration, not only at initial conversion.
Build the evidence record around the decision rather than the convenience of the software. Name the protected or measured unit, the version in force, the time boundary, the numerator and denominator, exclusions, missing cases and the person authorised to act. Keep raw observations separate from interpretation, and retain failed and rejected cases as well as successes. That separation lets a later reviewer distinguish a changed world from a changed judgement.
Convert the finding into a bounded action. Predefine the trigger, permitted response, owner, expiry, rollback and verification window. Keep one worked case that passes and one that fails. A system which stores only successful cases cannot demonstrate that its control detects the failure it was designed to contain.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: The original converter container is no longer supported; exercise the preserved source-to-target specification with a new verified implementation. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 12 check · what should survive an audit
- State the decision and measurement unit in one sentence.
- Preserve version, timing, denominator, exclusions and missing-case evidence.
- Connect the finding to a bounded action, rollback and later verification.
CHAPTER 13 OF 13 · Renew custody and govern retirement
13. Retire old formats only after evidence and handoff
Inventory users, APIs, citations, legal holds and reconstruction dependencies. Publish cutover dates and compatibility guidance, keep a rollback window, monitor requests for legacy objects and transfer preservation ownership. Retain the source or a verified independent preservation copy until target validation, access and reconstruction gates pass. Document any irreversibly lost capability rather than implying full equivalence.
Build the evidence record around the decision rather than the convenience of the software. Name the protected or measured unit, the version in force, the time boundary, the numerator and denominator, exclusions, missing cases and the person authorised to act. Keep raw observations separate from interpretation, and retain failed and rejected cases as well as successes. That separation lets a later reviewer distinguish a changed world from a changed judgement.
Convert the finding into a bounded action. Predefine the trigger, permitted response, owner, expiry, rollback and verification window. Keep one worked case that passes and one that fails. A system which stores only successful cases cannot demonstrate that its control detects the failure it was designed to contain.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: A legacy reader is the only way to interpret one proprietary flag table; preserve the reader environment or exclude those files from retirement. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 13 check · what should survive an audit
- State the decision and measurement unit in one sentence.
- Preserve version, timing, denominator, exclusions and missing-case evidence.
- Connect the finding to a bounded action, rollback and later verification.
Verification lab: work the decision twice
Migrate one fictional NetCDF-3 forecast package to NetCDF-4 with lossless deflate. Compare decoded arrays, units, calendar, dimensions, valid times, members and threshold decisions while linking source and target checksums.
Apply quantisation to discharge, precipitation and latitude using one shared setting. Show why variable-specific precision budgets and near-threshold tests reject the shortcut.
Rechunk the same archive for cloud access. Measure whole-grid, basin-series and single-member reads, then run a reconstruction drill from the independent replica before retiring the source.
Limits and misconceptions
A readable file is not proof of preserved meaning
Units, calendars, dimensions, ensemble semantics and provenance can still be wrong.
Compression and quantisation are not synonyms
Lossless filters preserve values; quantisation deliberately changes precision.
A new checksum cannot replace lineage
It proves target fixity only when linked to the verified source and transformation.
Low average error does not protect decisions
Near-threshold and tail cases need explicit sensitivity tests.
Questions readers ask
Can NetCDF be migrated without changing values?
- Yes with a verified lossless re-encoding, though target bytes and checksums will usually differ.
What is scientific compression?
- Storage reduction chosen with explicit knowledge of variable meaning, precision and downstream decisions.
When is quantisation acceptable?
- Only when variable-specific error budgets and threshold tests show the precision loss is fit for declared uses.
Should target checksums match source checksums?
- Only for identical bytes. Re-encoded targets need new checksums linked through transformation lineage.
How are tolerances chosen?
- Before migration, from source precision, scientific use, downstream calculations and operational thresholds.
When may the old format be retired?
- After value, semantic, reader, access, reconstruction and ownership gates pass with rollback resolved.
Practise, verify and continue
Create a migration specification for one forecast package: source manifest, schema map, lossless baseline, optional quantisation budget, threshold tests, reader matrix, source and target checksums, reconstruction drill and retirement gate.
Move the design from NetCDF files to object-based chunked arrays. Preserve the evidence questions while rebuilding metadata consolidation, chunk integrity, reader compatibility and custody controls for many objects.
Practical evidence check
Before implementation, write a one-page evidence map with the decision, unit of analysis, version or time boundary, inputs, outcomes, uncertainty, authority and stop rule. Attach one worked case that passes and one that fails. A reviewer should be able to reproduce the reasoning without relying on the memory of the person who built the system.
Run one changed-context rehearsal before approval. Alter a mechanism rather than a cosmetic detail, then identify which calculation, safeguard or governance gate must change. Record any conclusion that does not transfer. This short rehearsal is often the fastest way to expose a hidden assumption before it becomes an operational dependency.
Related eduKate reading
Legacy forecast archive reconstruction and evidence retention
Reforecast archive provenance and issuance-time reconstruction
Post-cutover surveillance and decommission assurance
Forecast-system transitions and safe decommissioning
Authoritative sources and scope
Unidata netCDF FAQ and archival compatibility guidance
Unidata netCDF variable, chunking, compression and quantisation APIs
Unidata netCDF quantisation documentation
NASA Planetary Data System preservation and data-integrity policies
The sources support netCDF compatibility, compression, quantisation metadata and preservation controls. The schema, tolerances and migration example are illustrative. Archives must set requirements from their scientific variables, operational decisions, records obligations, licences and available readers.