A catalogue has ten million records.
Someone discovers that one field is badly designed.
The new schema can fix the mistake.
It can also make ten million old records ambiguous overnight.
Schema evolution is the discipline of improving a metadata model while preserving enough semantic and operational continuity that old records, old software and old evidence remain interpretable.
This is the fourth pillar beneath How Metadata Works. The master owns schemas and application profiles generally. This article owns change through time: adding fields, changing constraints, deprecating terms, migrating records, versioning crosswalks and deciding when compatibility is more valuable than elegance.
Quick Read
Metadata schemas evolve because domains change, standards improve and old modelling decisions reveal limits. The safest changes are usually additive: introduce a new optional property, define it clearly and let old records remain valid. Breaking changes—renaming fields, changing meanings, narrowing value domains, changing cardinality or reusing an old identifier for a new concept—require explicit versions and migration rules. Mature vocabularies protect persistent term identifiers, deprecate rather than silently redefine, and document change history. DCAT 3 is a useful current example: it retains the DCAT namespace and was designed so existing DCAT 2 implementations that do not use new features can remain conformant. Schema evolution succeeds when a system can answer both “what does this record mean now?” and “what did this record mean under the rules that created it?”
schema v1 + records → new requirement → classify change → additive / compatible / breaking → publish v2 → migrate or dual-read → validate → update crosswalks/APIs → preserve v1 interpretation → retire safely
Why Schemas Change
The world changes.
The organisation changes.
The software changes.
The understanding changes.
A metadata model that never evolves usually does not mean the domain stopped evolving.
It may mean the schema has become too brittle to express the new reality.
A New Field Is the Easiest Change
Schema v1 describes:
- title;
- creator;
- date;
- identifier.
Schema v2 adds:
accessibilityFeature
If the new property is optional, old records can remain valid while new records become richer.
This is additive evolution.
Additive Does Not Mean Consequence-Free
Old software may ignore the new field.
That is often acceptable.
But if the new field carries safety-critical rights or privacy information, silent ignorance may not be acceptable.
Compatibility must be judged against application behaviour, not only parser success.
Unknown-Field Tolerance Is a Powerful Compatibility Pattern
A v1 reader receives a v2 record containing one unfamiliar optional property.
If it can ignore that property safely while preserving the rest, the ecosystem can evolve gradually.
This pattern is especially useful in extensible JSON/RDF systems.
It is unsafe when unknown fields change the meaning of known fields or carry mandatory control information.
Renaming a Field Is More Dangerous Than It Looks
Schema v1:
author
Schema v2:
creator
If the meaning is genuinely unchanged, an alias or deprecation bridge may preserve compatibility.
If the meaning changed too, calling it a “rename” hides a semantic migration.
Never Reuse an Old Identifier for a New Meaning
Property URI X meant “publication date.”
Years later, designers decide X should mean “last updated date.”
Every historical record using X has now become ambiguous.
persistent identifier + changed semantics = retroactive corruption.
Publish a new term instead.
Deprecation Is a Compatibility Tool
A deprecated field says:
Old records using this remain interpretable, but new records should use the replacement.
This is different from deleting the term definition.
Deletion removes the route needed to understand history.
Good Deprecation Names the Replacement
Deprecated:
date
Prefer:
dateCreated;dateIssued;dateModified.
But the migration cannot choose among those automatically if old date values did not record which event they described.
Deprecation documentation should state where automatic conversion is safe and where human/context review is required.
Changing Cardinality Can Break Existing Meaning
v1:
creator = one value
v2:
creator = many values
Expanding one-to-many is often relatively compatible.
The reverse—many to one—is lossy because old records may contain multiple legitimate creators that no longer fit.
Changing Requiredness Is a Deployment Decision
A formerly optional field becomes required.
Every old record that lacks it now fails v2 validation.
Possible strategies:
- grandfather old records under v1;
- backfill from trustworthy sources;
- introduce a transition period;
- allow explicit unknown values;
- make the field required only for new resource types.
“Required” is not only schema design. It creates migration labour.
Changing a Datatype Can Be Worse
v1 stores:
date = "Spring 1998"
v2 requires an ISO date.
The old value cannot be converted exactly because “Spring 1998” is intentionally imprecise.
A migration that invents 1998-03-01 creates false precision.
A better v2 model may need date ranges or uncertain-date representation rather than merely a stricter datatype.
Changing a Controlled Vocabulary Creates Orphan Values
Old term:
Computer Studies
New vocabulary splits it into:
- Computer Science;
- Information Technology;
- Digital Literacy.
The old classification cannot always be split retrospectively without re-examining the resource.
Schema evolution therefore intersects with vocabulary evolution and reclassification policy.
DCMI Shows the Value of Stable Term Identifiers
The current DCMI Metadata Terms maintain persistent URIs for widely used metadata properties and publish definitions, usage notes and version history.
This lets systems refer to stable semantic identifiers rather than depending only on local display labels.
A human label can be improved without silently changing the term’s identity.
DCAT 3 Is a Useful Modern Compatibility Example
W3C DCAT 3 is a Recommendation dated 22 August 2024. Its specification retains the DCAT namespace, adds capabilities such as richer versioning and dataset-series support, and explicitly aims to preserve backward compatibility with prior DCAT terms.
W3C notes that existing DCAT 2 deployments that do not overlap with the new features do not need to change simply to remain conformant with DCAT 3.
This is controlled evolution rather than forced migration for its own sake.
Backward Compatibility Has Several Meanings
- parse compatibility: new software can read old records;
- validation compatibility: old records still satisfy new constraints;
- semantic compatibility: old terms still mean what they meant;
- behaviour compatibility: applications take the same safe actions;
- round-trip compatibility: records can move between versions without loss.
A system can pass one compatibility test and fail another.
Forward Compatibility Matters Too
Can an old reader encounter a new record and continue safely?
Unknown optional fields may be ignored.
Unknown required semantics may require rejection or explicit downgrade.
Forward compatibility is not “old software accepts everything.”
It is “old software knows what it can safely ignore.”
Schema Version Should Travel With the Record When Semantics Depend on It
A record contains:
date = 2024-05-01
Without schema version, the future reader may not know whether that date meant creation, publication or last update.
Version metadata protects historical interpretation when property semantics or constraints evolved.
But Version Numbers Alone Are Not Enough
“Schema v3.”
What changed from v2?
Without change notes, migration guidance and machine-readable definitions, version numbers become labels rather than compatibility tools.
Change Logs Should Classify Impact
- new optional property;
- new controlled term;
- clarified documentation only;
- constraint tightened;
- property deprecated;
- property replaced;
- semantic definition changed;
- breaking structural redesign.
Consumers can then decide whether they need migration or merely awareness.
Semantic Versioning Is a Useful Analogy, Not a Universal Metadata Law
Software commonly treats major/minor/patch numbers as compatibility signals.
Metadata vocabularies can borrow the idea:
- major = breaking semantic/structural change;
- minor = backward-compatible additions;
- patch = clarifications or non-semantic repairs.
But a standards community may use another versioning policy. The important requirement is explicit, documented meaning.
Migration Is a Transformation With Provenance
Record v1 becomes record v2.
The migration may:
- rename properties;
- split fields;
- convert datatypes;
- map vocabularies;
- add inferred identifiers;
- leave some values unresolved.
The first pillar, Metadata Provenance, owns the lineage of those transformed values.
Do Not Overwrite the Last Good Source During Migration
Convert one million v1 records to v2 in place.
Then discover the migration rule was wrong.
If the original v1 representation is gone, repair becomes reconstruction.
Safer patterns include:
- immutable source snapshot;
- versioned target store;
- migration receipts;
- rollback capability;
- sample validation before bulk conversion.
Dual Read Can Ease Transitions
During migration, new software can read both v1 and v2.
Writers produce only v2.
Over time, v1 inventory shrinks.
This avoids a single “flag day” on which every producer and consumer must upgrade simultaneously.
Dual Write Is More Dangerous
Every update is written to v1 and v2 stores.
If one write succeeds and the other fails, the versions diverge.
Dual write requires strong consistency, repair and ownership rules.
Where possible, one canonical owner plus derived compatibility projections is easier to reason about.
Compatibility Views Can Keep Old Consumers Alive
Canonical store moves to v2.
An adapter exposes a v1-compatible projection for legacy clients.
The adapter owns the loss.
The third pillar, Metadata Crosswalks, explains why that compatibility view must disclose which v2 distinctions cannot be represented in v1.
Validation Rules Evolve Separately From Vocabulary Terms
The property creator remains stable.
Application profile v2 now requires at least one creator for published books.
The vocabulary term did not change.
The local constraint changed.
Schema/version governance should distinguish vocabulary evolution from profile/validation evolution.
SHACL Makes Constraint Versions Concrete
W3C SHACL represents validation conditions as shapes graphs and produces validation reports over RDF data graphs.
Publish shape set v1 and shape set v2 separately and a system can say:
this record conforms to profile v1 but fails profile v2 because the new required field is missing.
That is far more informative than “invalid metadata.”
A Validation Failure Can Mean “Old,” Not “Wrong”
Historic record was valid under the schema that governed it.
New profile demands more fields.
The record now fails current validation.
The correct interpretation may be:
valid v1 record, not yet migrated to v2.
Version-aware validation preserves this distinction.
Currentness and Schema Evolution Interlock
A record has current resource facts but uses an obsolete schema.
Another record uses the newest schema but contains stale values.
Those are different failure modes.
The second pillar, Metadata Currentness, owns whether field assertions still describe the relevant world state.
Schema Evolution owns whether the representation model remains supported and interpretable.
Crosswalks Must Evolve With Both Ends
Crosswalk A1→B1.
A becomes A2.
B becomes B3.
The mapping is no longer one object.
It is a matrix of version relationships.
Mapping provenance should name both source and target versions so stale crosswalks can be detected.
APIs Need Version Negotiation or Stable Compatibility Contracts
An API changes response metadata structure.
Existing clients break.
Options include:
- versioned endpoints;
- media-type/profile negotiation;
- additive response evolution;
- deprecation windows;
- compatibility adapters.
How APIs Work owns the broader interaction contract. Schema Evolution owns the changing metadata payload model inside that contract.
Search Indexes Need Reindexing After Semantic Change
Schema v2 splits one subject field into three facets.
The canonical metadata migrates.
The old search index still contains one flat field.
Until reindexing completes, the public discovery layer still behaves as if v1 were current.
Migration is incomplete until derived representations catch up.
Do Not Migrate What You Do Not Understand
Old field values look strange.
Documentation is missing.
A developer guesses what they meant and converts them.
This transforms uncertainty into false certainty.
Quarantine unresolved values and preserve the old semantics until evidence supports a migration rule.
Historical Interpretation Is a Preservation Requirement
A 2004 catalogue export should remain intelligible in 2040.
That may require preserving:
- schema documentation;
- term definitions;
- controlled-vocabulary versions;
- validation rules;
- crosswalks;
- serialisation details;
- migration history.
Preserving bytes without preserving the descriptive contract can leave future systems with perfectly intact but semantically orphaned records.
Governance Matters More Than Clever Migrations
Who can add a property?
Who can deprecate one?
Who decides a change is breaking?
Who maintains old documentation?
Who approves migration logic?
Without governance, schema evolution becomes a sequence of local fixes that accumulate incompatible meanings.
A Better Schema-Evolution Model
new requirement → inspect existing semantic owner → classify compatibility → publish versioned term/profile change → document migration → preserve source → migrate with provenance → dual-read/adapter period → validate both states → update crosswalks/indexes/APIs → deprecate safely → preserve historical interpretation
A 30-Lens Schema Evolution Audit
- Requirement: what new job cannot v1 express?
- Owner: does an existing property already own the concept?
- Version: what schema/profile release is changing?
- Change type: additive, clarifying, constraining or breaking?
- Identifier: will old semantic IDs remain stable?
- Definition: is any existing meaning changing?
- New property: can addition solve the need?
- Deprecation: should an old property remain readable?
- Replacement: is the successor explicit?
- Alias: is this truly a rename?
- Cardinality: did one/many constraints change?
- Requiredness: will old records fail?
- Datatype: can old values convert without invention?
- Vocabulary: did controlled concepts change?
- Unknown fields: can old readers ignore additions safely?
- Backward parse: can new readers read old records?
- Forward parse: can old readers tolerate new records?
- Semantic compatibility: do known terms retain meaning?
- Behaviour compatibility: do consumers act safely?
- Migration: which records require transformation?
- Source preservation: is the v1 original retained?
- Provenance: can migrated values trace back?
- Validation: which profile/version does each record conform to?
- Crosswalk: have mappings been updated?
- API: do payload contracts need versioning?
- Index: must derived search structures be rebuilt?
- Rollback: can a bad migration be reversed?
- Deprecation window: how long are old consumers supported?
- Historical interpretation: can future readers reconstruct v1 meaning?
- World return: does the evolved model improve real retrieval/use without breaking legitimate old records?
Laboratory 1: Optional to Required
Take a schema where language is optional. Make it required in v2. Design three migration strategies for the old records and identify where each could create false data.
Laboratory 2: Rename or Redefine?
Compare “author” and “creator.” Decide whether the change is a harmless label update, an alias or a genuinely broader concept requiring a new property and explicit mapping.
Laboratory 3: Validate v1 and v2
Create two small validation profiles. Show a record that is valid under v1 but not v2, then explain why “not v2 conformant” should not be rewritten as “bad historical metadata.”
For Primary Readers
Your class form used to ask only “Name.” The new form also asks “Preferred name.” Old forms are still understandable. The new field adds information without making every old form nonsense.
For Secondary Readers
Classify schema changes as additive or breaking and explain what happens to old records, old software and crosswalks after each change.
For Advanced Readers
Model schema evolution as versioned semantic contract change under backward/forward compatibility constraints. Safe migration preserves term identity, source provenance, historical interpretation and explicit loss boundaries while derived consumers transition at independently governed rates.
Common Misconceptions
- “Adding an optional field can never break anything.” Old consumers may safely ignore it only if the field does not carry mandatory control semantics.
- “Renaming a field is cosmetic.” A rename often hides a changed or broadened meaning.
- “Deprecated means delete.” Deprecation preserves historical interpretability while guiding new usage elsewhere.
- “If a record fails the newest schema, it was always invalid.” It may be a legitimate v1 record awaiting migration.
- “Migration is complete when the database is converted.” APIs, indexes, crosswalks, caches and downstream systems must also transition.
Research Corridor
- W3C DCAT 3 — a current example of vocabulary evolution designed to retain backward compatibility while adding versioning and other capabilities.
- DCMI Metadata Terms — persistent metadata term identifiers, definitions and version history.
- W3C SHACL — versionable shapes and validation reports for RDF constraint profiles.
- eduKateSG — How Standards Work.
Frequently Asked Questions
What is metadata schema evolution?
It is the controlled change of a metadata vocabulary, schema or application profile over time while managing compatibility, migrations and historical interpretation.
What is the safest kind of schema change?
Often an optional additive property with stable existing semantics, provided older consumers can safely ignore it and no hidden mandatory behaviour depends on it.
Why should deprecated metadata terms remain documented?
Because old records may continue to contain them. Preserving their historical definitions lets future software and researchers interpret those records correctly and migrate them deliberately.
Final Thought: A Living Schema Must Remember Its Former Languages
A good metadata model learns.
A mature one learns without making its own history unreadable.
Schema evolution succeeds when the new model can say more about the world without forcing old records to say something they never meant.
METADATA · FOUR PILLAR LEGS
Return to How Metadata Works, or continue through Metadata Provenance, Metadata Currentness and Metadata Crosswalks. Return to the Information & Representation Hub.