Data provenance explains where a fact, dataset or result came from, which steps transformed it and which source versions support its interpretation. A traceable result is easier to verify, correct and reproduce—but a complete lineage record is not itself proof that the original observations are valid or that their reuse is permitted.
This guide follows data from source identity and W3C PROV relationships through measurement records, database derivations, rights, corrections, privacy and reproducibility. The practical cases show how an apparently correct chart can mislead when its input version, unit, denominator or transformation is lost.
Find your evidence question
What provenance records · Chapters 1–8 · Start here
Measurements, queries and derivation · Chapters 9–16 · Start here
Corrections, rights and trust · Chapters 17–23 · Start here
Reproducibility, privacy and knowledge · Chapters 24–30 · Start here
All chapters · Original cases · Deeper audits · Connected subject guides
Complete chapter index
What provenance records
- 1. What data provenance actually answers
- 2. Provenance is not the same as citation or ownership
- 3. W3C PROV provides a common conceptual model
- 4. Entities need clear identity and versions
- 5. Activities connect raw records to results
- 6. Agents record responsibility without pretending certainty
- 7. Derivation edges should not hide changed meaning
- 8. Source registries prevent orphaned claims
Measurements, queries and derivation
- 9. Timestamps carry several meanings
- 10. Schema provenance explains field definitions
- 11. Measurement provenance begins before analysis
- 12. Collection and sampling are part of lineage
- 13. Filters create a new population of records
- 14. Joins create provenance for each combined row
- 15. Why-provenance and where-provenance ask distinct questions
- 16. Named graphs and semantic provenance
Corrections, rights and trust
- 17. Transformation code requires a versioned environment
- 18. Human corrections need their own provenance
- 19. Provenance cannot be reduced to a checksum
- 20. Provenance and cryptographic signatures
- 21. Data licences travel differently from data lineage
- 22. Corrections must propagate beyond one dashboard
- 23. Retractions and source withdrawal
Reproducibility, privacy and knowledge
- 24. Reproducibility is an outcome of enough recorded detail
- 25. Privacy puts a boundary around provenance disclosure
- 26. Claims need evidence spans, not only document links
- 27. Human review should be visible without exposing private work
- 28. Crosswalking sources requires relation-level provenance
- 29. A practical investigation from one published number
- 30. Frequently asked questions and the next learning step
Four different evidence questions
| Question | Relevant evidence | Common gap |
|---|---|---|
| Which data were used? | Source and exact version | Ambiguous mutable filename |
| How was it calculated? | Recorded transformation and parameters | Undocumented filters or joins |
| Can I trust the source? | Measurement and independent review | Hash or citation mistaken for validation |
| May it be republished? | Licence, consent and authority | Availability mistaken for permission |
| What if it changes? | Correction and dependency history | Unrevised downstream claims |
1. What data provenance actually answers
Data provenance explains where a record or result came from, how it was produced and which people, organisations or processes contributed to it. A chart showing improved examination results has limited evidential value if nobody can identify the underlying assessment version, selected population, scoring changes and subsequent corrections. Provenance makes that production history inspectable.
The key question is not merely whether a spreadsheet exists. It is whether an authorised reader can follow a meaningful claim back through source records and transformations and assess limitations. Provenance supports traceability; it does not automatically prove that a source was accurate or that processing was ethically authorised.
2. Provenance is not the same as citation or ownership
A citation points readers to a source; attribution may credit an author or creator; licensing describes permitted reuse; provenance describes a chain of entities and activities used to create a result. These relationships overlap but answer different questions. A dataset can cite a study, yet omit the transformation that actually produced its values.
Record source identity, versions, derivation steps, responsible agents and restrictions separately. Do not assume an openly linked PDF grants a licence to republish all its content. A provenance record can document a restricted source without exposing its protected contents. Correct rights handling is part of trustworthy evidence navigation.
3. W3C PROV provides a common conceptual model
The World Wide Web Consortium’s PROV Data Model distinguishes entities, activities and agents. Entities are things such as datasets, documents or derived tables; activities are processes performed over time; and agents are people, organisations or software systems associated with responsibility for activities or entities.
Use the model to describe relationships such as an activity using an input and generating an output. The standard helps systems exchange descriptions of lineage, but every record still needs domain-specific meanings and reliable identifiers. An automatically generated PROV diagram does not validate the source facts represented inside it.
4. Entities need clear identity and versions
A data entity might be a CSV file, a particular revision of a report, a database table snapshot or one calibrated measurement series. If ‘the dataset’ changes whenever new rows arrive, a derivation record must identify which version the analysis actually read. Otherwise reviewers can follow a link but cannot reproduce the reported number.
Use stable identifiers and version or snapshot references appropriate to the source. Timestamps alone are not sufficient if two different versions can share a display name or if clocks are inaccurate. Record checksum, source URI, record key or release identifier where relevant without claiming any single identifier proves truth.
5. Activities connect raw records to results
An activity could ingest a file, validate a schema, convert units, join datasets, filter out observations or calculate a statistic. The process should document its inputs, output, software or method version, important parameters and relevant errors. A report with impressive statistical analysis may still be uninterpretable if its preprocessing was never recorded.
An activity labelled clean_data is too vague to explain whether rows were deleted, corrected or imputed. Describe the transformation concretely: which fields were converted, which invalid records were quarantined and which assumptions were used. A reproducible script or pipeline specification is stronger evidence than a generic label.
6. Agents record responsibility without pretending certainty
Agents may include an author, laboratory, school, service operator or software system involved in generating an entity or running an activity. The fact that one agent executed an action does not automatically imply they approved every consequence, authored every input or carry sole legal responsibility.
Separate execution, review, approval and data stewardship roles. Where actions are automated, record which service version was responsible and which human authority governed the process. Do not infer personal blame from a log entry without investigating permissions, delegation, system configuration and institutional circumstances.
7. Derivation edges should not hide changed meaning
If table B was derived from table A, state whether B preserves the original observations, aggregates them or changes how variables are defined. One derivation may be a simple column rename; another may remove missing cases or create a prediction. Both are derivations, but their implications for interpreting B differ.
Use a directed lineage graph with typed transformations and explicit semantic changes. A chain of edges is not an explanation by itself. Readers should be able to determine which operations are reversible, which lose information and which require assumptions that may not survive a changed context.
8. Source registries prevent orphaned claims
A source registry records authoritative titles, organisations, URLs or identifiers, access dates, release versions, formats and relevant rights status. When a source is updated or moved, the registry can retain the identity and status of what an article previously used. It also provides one controlled place for corrections or deprecation notices.
A registry should not treat every item as equally authoritative. Distinguish primary measurement records, scholarly literature, official policies and community commentary. A stable registry improves retrieval and governance when each entry includes scope, uncertainty and review ownership rather than just a collection of links.
9. Timestamps carry several meanings
A measurement time is when a physical observation occurred, a publication time is when a report was released, and an ingestion time is when a database received it. Correction time records when an earlier claim was amended. These values may differ substantially and can be essential to reconstructing what someone knew at a decision point.
Use time zones, precision and clock assumptions appropriate to the system. A late-arriving record does not necessarily describe a new event. Analysts should avoid treating database arrival order as event order unless the collection process guarantees that relationship. Temporal provenance prevents false trends caused by delayed data.
10. Schema provenance explains field definitions
A column called Score might represent raw marks, a percentage, a scaled examination value or a predicted probability. Storing only the number loses the measurement’s meaning. A schema version should define units, missing-value rules, population and permissible comparisons for each field.
Record changes whenever meanings shift. An assessment revised from thirty questions to twenty cannot be compared by raw marks without a suitable linking argument. Database integrity can ensure a value fits a type while provenance explains the construct and method used to produce it.
11. Measurement provenance begins before analysis
A physical measurement depends on instrument, calibration, configuration, environmental conditions and units. If a device is replaced midway through a series, the apparent trend may reflect a new measurement chain. Provenance should identify enough of that chain to assess comparability and error.
The Scientific Instrumentation root explains traceable readings and drift. Data provenance complements it by preserving which measurements, equipment versions and calibration references entered a published result. A detailed software script cannot correct missing knowledge about a miscalibrated sensor.
12. Collection and sampling are part of lineage
Survey or observational data often represent only people who were selected, contacted and responded. Provenance needs more than a list of completed rows: it should identify the target population, frame, inclusion criteria and collection procedure, subject to privacy safeguards.
A thousand survey replies from self-selected volunteers do not become a probability sample because the database contains a thousand records. Link the analytical dataset to its sampling design, response codes and missingness handling. Read Survey Methodology for the related inference requirements.
13. Filters create a new population of records
A preprocessing rule may keep only complete responses, users over a chosen age or data from sensors passing a quality threshold. These filters change which observations contribute to the result, sometimes changing the estimand or introducing selection bias. A label like final_dataset does not tell reviewers why records disappeared.
Preserve a selection count at each stage, meaningful exclusion reasons and the exact rule version. Sensitive row-level information need not become public. A transparent aggregate flow can still show the difference between originally eligible, successfully observed and finally analysed units.
14. Joins create provenance for each combined row
A relational join can bring together facts from different tables, but incorrect keys or many-to-many relationships can multiply observations. A derived row should retain enough link information to identify the source records from which it came, where appropriate and authorised.
If three course enrolments join with four attempts only on learner ID, twelve combined rows can appear. A sum calculated afterward may count the same attempt multiple times. Lineage explains the expansion, while correct database modelling prevents it. See Database Theory.
15. Why-provenance and where-provenance ask distinct questions
In theoretical database provenance, where-provenance concerns which input locations contributed to a particular output value under a specified query and annotation model. Why-provenance can describe alternative combinations of source tuples that witness an output. The terms have formal definitions and do not mean that a query has proven causal explanation.
Use the appropriate query-semantics framework instead of treating ‘why’ in a provenance label as psychological or causal motivation. A query may derive a result from several possible joins, but that logical derivation does not establish that the observed real-world event was caused by those records.
16. Named graphs and semantic provenance
RDF named graphs can group triples in identified graph contexts, while provenance records may describe the origin, publication or review status of a graph. An RDF graph name does not by itself establish its author’s identity or a complete historical chain; that information must be expressed separately.
When using SPARQL or knowledge graphs, document graph identifiers, dataset versions and entailment assumptions. Derived triples under inference should be distinguished from directly supplied assertions. An automated reasoner’s result may be logically supported under a rule set while the originating evidence remains uncertain.
17. Transformation code requires a versioned environment
A computation depends on source code, libraries, configuration, random seeds and sometimes hardware-specific arithmetic. Two runs with the same script filename may produce different outputs when dependencies or inputs differ. Lineage must therefore record more than a friendly software product name.
Capture versioned code references, parameters, data snapshots and relevant environment settings. Container hashes or build manifests may assist reproducibility, but also depend on external services or secrets that should not be published. Share a reproducibility recipe that balances transparency with lawful and secure access.
18. Human corrections need their own provenance
A reviewer may correct a date, change a data label or reject an implausible measurement. Those decisions can improve quality but should not become invisible edits that overwrite earlier states without context. Record what changed, why, under whose authorised process and which published outputs are affected.
Separate a genuine corrected fact from a modelling preference or editorial clarification. A correction log helps downstream articles, dashboards and datasets decide whether they need recalculation. It also supports restoration when a proposed correction is later found mistaken.
19. Provenance cannot be reduced to a checksum
A cryptographic hash can help detect whether bytes differ from a known reference under suitable assumptions. It cannot independently prove the creator’s identity, scientific validity or right to process the file. A matching digest shows consistency with particular bytes, not that those bytes were originally correct.
Combine fixity checks with trusted source identity, signatures where appropriate, audit records and independent review. If someone signs an incorrect dataset, the signature can authenticate a statement or artifact while the scientific content remains wrong. Keep integrity, authenticity and validity distinct.
20. Provenance and cryptographic signatures
Digital signatures can help verify that a statement or artifact was signed using a particular key, subject to key management and the signing protocol. They do not automatically establish that the named human personally reviewed every record, or that the signing key was secure at the relevant time.
Document key ownership, signing scope, revocation and trust assumptions. A signature on a manifest may cover the hashes of several artifacts rather than their meaning. Strong provenance connects cryptographic evidence to a governed publishing process without pretending mathematics proves institutional honesty.
21. Data licences travel differently from data lineage
A dataset may incorporate public-domain measurements, copyrighted commentary and restricted participant records. The provenance graph can identify each source, but licences must be evaluated separately. The fact that an output used an open source does not make all derived text freely reproducible.
Record source-level rights, required attribution, commercial-use limits and any restrictions on AI training or redistribution. Where permission is absent, a link and original summary may be appropriate while copying protected content is not. Rights conditions can change with the particular version and provider.
22. Corrections must propagate beyond one dashboard
If an original dataset is corrected after several articles and models have been published, those downstream uses may remain wrong. A dependency graph can identify outputs that used the earlier version. Recovery involves assessing the materiality of the change and issuing recalculations, corrections or withdrawal notices where warranted.
A dependency link is not the same as automatic invalidation of every output. Some aggregates may be unaffected; others may change a consequential decision. Use targeted tests and clear review authority. A published correction should identify affected versions, not silently swap the file and leave citations pointing to an ambiguous past.
23. Retractions and source withdrawal
A scientific paper may be retracted or corrected; a repository can withdraw a dataset; an institution may revoke permission for an output. The provenance system should recognise such status changes and help reviewers identify dependent claims. Simply deleting the registry entry destroys useful evidence about how an earlier conclusion arose.
Maintain appropriate tombstone metadata, public notices or access-restricted records according to policy and law. Re-evaluate the underlying claim against independent evidence rather than assume a retraction automatically reverses every related fact. Provenance makes such reassessment feasible.
24. Reproducibility is an outcome of enough recorded detail
To reproduce a computational result, a qualified reviewer needs appropriate access to inputs, method, parameters and environment. Provenance contributes the map, but the materials must actually be preserved and executable. Confidentiality may require secure enclaves, synthetic examples or restricted access instead of public raw records.
Distinguish computational reproducibility from independent replication with newly collected data. Re-running the same code on the same dataset can verify processing without validating the original measurement or sampling design. Both are useful but answer different questions.
25. Privacy puts a boundary around provenance disclosure
Detailed lineage may reveal who entered a clinical record, which students attended a service, or which internal dataset was queried. Publishing every link can create privacy and security risks even when output data are aggregated. A responsible system separates public methodological descriptions from authorised internal detail.
Minimise personal identifiers, restrict access by purpose and apply appropriate retention rules. A public article can show source families, method and aggregate counts without exposing protected row IDs. Provenance should support accountability, not become a surveillance database.
26. Claims need evidence spans, not only document links
A citation to a long report is more useful when a reviewer can identify the page, section, paragraph or data table supporting the specific claim. An evidence span links one atomic assertion to a precise part of a source version. This helps distinguish a directly documented fact from a later interpretation or inference.
Do not create imaginary quotations, page numbers or citations. If a source is inaccessible or unclear, label the claim accordingly or withhold it. Reliable evidence navigation favours accurate, narrow citations over a decorative bibliography that cannot support the article’s actual statements.
27. Human review should be visible without exposing private work
Automated pipelines can flag anomalous data, unsupported assertions and broken source links. Reviewers must decide whether a result is sufficiently supported for its intended use and document material corrections. Review can be represented as a distinct authorised activity in the lineage graph.
Avoid equating approval metadata with objective truth. A reviewer may have limited scope, and subsequent evidence can overturn an earlier decision. Document what was checked, when and against which version. Strong governance leaves a defensible path for correction and appeal.
28. Crosswalking sources requires relation-level provenance
A knowledge library may connect equivalent or broader concepts across dictionaries, curricula and scholarly databases. Every asserted relation needs evidence of the two concept identities, the relation type and the source or reasoning supporting the link. A close conceptual match is not automatically an exact match.
Retain source versions and explain transformations when mapping identifiers or terms. A curriculum alignment can become obsolete after an examination syllabus changes. Versioned crosswalk provenance helps readers see whether the connection is current, provisional or superseded, without publishing private implementation details.
29. A practical investigation from one published number
Take one number in a public report and ask what exact entity it represents, which source version supplied it, what transformations produced it and which assumptions govern comparison. Work backwards until the result can be reproduced by an appropriately authorised reviewer. Then introduce a hypothetical correction to an input record and identify which outputs need reassessment.
The goal is to trace reasoning, not merely add metadata fields. A simple claim with clear lineage is often more useful than a complex dashboard whose derivation is hidden. Move from this exercise to automated dependency analysis, governed source registries and correction workflows as scale requires.
30. Frequently asked questions and the next learning step
Does a citation prove data quality? No. Does a matching hash prove that a measurement was scientifically correct? No. Does provenance reveal every cause of a real-world event? No. Should every protected source record be published for transparency? No. Does a data correction automatically invalidate every derivative? Only after impact assessment.
Begin with a three-step data process: source table, cleaned table and chart. Give each version an identity, list transformations and simulate one corrected source value. Explain the changed chart and the authority for the correction. That practice scales toward accountable scientific and educational information without confusing traceability with certainty.
Twenty-five original provenance failure and repair cases
1. One chart, two incompatible source versions
A published chart cites ‘results.csv’, but the file was overwritten after a scoring correction. The title and URL are unchanged while the values differ. A reviewer cannot determine which version produced the chart. Preserve a release ID or verified snapshot and record the calculation against that version.
2. A corrected unit changes an environmental conclusion
A sensor export records concentration in micrograms, but a downstream script treats the same column as milligrams. A large discrepancy appears despite perfectly executed code. Source and schema provenance identify the unit mismatch, allowing the derived result to be corrected and affected reports reviewed.
3. A survey total hides missing households
A community analysis cites 150 completed forms but ignores that several were submitted by the same household. If the target quantity concerns households rather than forms, the denominator is wrong. Collection and identity rules belong in the evidence chain. Correct arithmetic cannot fix the wrong sampling unit.
4. A join doubles an assessment mark
A pupil has two enrolment records and three assessment attempts. A join by pupil ID alone can produce six combinations. Summing marks after this join may count each attempt twice. Row-level derivation shows why the output expanded, and a corrected query can preserve the intended assessment unit.
5. A derived field has an undocumented formula
A report contains a column called adjusted_score, but the adjustment formula is missing. Nobody can tell whether it rescales to a percentage, corrects for test difficulty or applies a subjective bonus. Preserve the formula, parameter version and justification before using adjusted results for decisions.
6. Two timestamps refer to different events
One table stores the observation time and another the upload time. A dashboard merges them into a single chronology as if both mean when the event happened. Late uploads create a false surge. Name each time dimension and align records according to the actual question.
7. A cryptographic digest matches an incorrect file
A download’s digest matches its official manifest, but the file contains a mislabelled unit. The check supports byte-level fixity relative to the manifest; it cannot establish scientific validity. Independent data validation must still inspect units, schema and source context.
8. A source is withdrawn after article publication
An institution withdraws a dataset because identifiers were exposed. An article cites one derivative table made from that release. The provenance system should identify the derivative and trigger appropriate rights, privacy and scientific review, not simply delete the citation and forget what happened.
9. One dataset uses two sampling frames
A study merges students sampled from school rosters with voluntary website respondents. A report treats them as one simple random sample. Provenance should retain source-frame and selection information and prevent unsupported inference from the merged total.
10. A chart changes after a dependency update
Two runs use the same source file and script name, but a numerical library version changes a default rounding or optimisation behaviour. A manifest of code and dependency versions allows the discrepancy to be investigated. Source-file provenance alone was insufficient.
11. A review signature covers the wrong object
An approver signs a summary manifest, but its source-file list omits the data used in a later figure. The signature is valid for the signed bytes, not the absent figure’s evidence. Check which artifacts the review actually covered before announcing whole-publication approval.
12. A quoted claim lacks a precise supporting span
An article links a hundred-page report and asserts that a programme improved all students. The source supports only an average effect in a specified group. A claim-to-section evidence link exposes the overstatement and prompts a more accurate summary.
13. A versioned ontology mapping becomes obsolete
A vocabulary term once mapped to a curriculum outcome whose official syllabus wording later changed. The original concept link may still be historically informative but is not automatically current alignment. Record mapping version and effective curriculum period, then review its status.
14. A provenance graph reveals sensitive contact patterns
A project proposes exposing every staff member and participant ID in a public lineage viewer. Even without names, rare combinations can reveal private involvement. Publish an aggregate process description while protecting row-level records behind legitimate access controls.
15. A correction changes one result but not another
A single source row is revised. A table counting all eligible cases changes, while an unrelated summary for another time period may remain unchanged. Lineage guides targeted impact assessment rather than assuming every downstream chart is invalid.
16. A reproducible script is applied to biased data
The same notebook can regenerate a published mean exactly, yet the data omit people without internet access. Computational reproducibility is demonstrated, but representativeness is not. Record sampling and response limitations alongside executable code.
17. A repository moves its hosting URL
An authoritative data release receives a new landing-page address. A source registry retaining the DOI or canonical identifier and release details can update navigation without losing historical provenance. A bare URL-only citation is more fragile.
18. A derivative has an incompatible licence
Two inputs appear openly accessible, but one restricts redistribution of derived content. A public download built from their combination may be impermissible. Source-level rights and permitted operations need to be checked separately from technical reproducibility.
19. An AI-generated summary cites a different revision
A model summarises one policy version, but an editorial system automatically replaces its citation URL with the newest revision. Readers now see a source that may not support the original statement. Link evidence to the actual reviewed version, then re-evaluate the claim if its source changes.
20. A reader reconstructs a number without private data
A school publishes an aggregate score with a clear method and de-identified counts, while keeping individual results private. An authorised reviewer can reproduce the calculation in a controlled setting. Transparency of method does not require exposing personal records.
21. A pipeline quarantines invalid rows without notice
A data-cleaning step silently removes twenty records that fail a format check. The published percentage uses the smaller denominator without explanation. The lineage should record excluded counts and reasons, and the analyst should test whether exclusions bias the result.
22. An audit trail contains a correction with no authority
A database log shows an edit but cannot identify whether it was an authorised correction, automated synchronisation or unauthorised change. Provenance needs appropriate activity and agent context while preserving privacy and the distinction between event recording and responsibility.
23. An external source retracts a claim
A scholarly source is retracted for a methodological error. The provenance graph helps identify which claims relied on it and which were independently corroborated. The correct action is an evidence review and possibly correction, not blind deletion of all related knowledge.
24. A model prediction has no training-data lineage
A machine-learning service releases an accuracy number without documenting the evaluation cohort, label definitions, splits or code version. The metric cannot be meaningfully audited. Model provenance should identify these sources while guarding private records.
25. A classroom worksheet becomes a public fact claim
A pupil invents three measurements for a data exercise. Later the dataset is copied into a public report without its simulated status. Provenance needs to label synthetic teaching examples so fictional numbers cannot be mistaken for actual observations.
Sixteen deeper data-lineage and changed-context audits
1. A revision changes a denominator while leaving the title unchanged
A published performance report is recomputed after a school’s eligibility policy changes, but its headline and filename remain the same. Readers comparing the new percentage with last month’s copy assume the population stayed constant. Provenance should capture the new eligibility rule, cohort boundary and output version. The two numbers may both be mathematically correct while describing different populations. A valid comparison requires an explicit bridging analysis rather than simply subtracting the percentages.
2. A source licence is attached to an identifier but not the actual release
A dataset publisher changes its licensing conditions between releases. An aggregation job records only the catalogue ID, so the team mistakenly applies the latest licence to an older copy whose rights differ. Record the version, licence notice and permitted operations that governed the actual input. A source’s identity remains useful across revisions, but rights are contextual. This is especially important when combining open information with third-party educational materials or commercial data.
3. A generated explanation confuses inference with direct evidence
A knowledge graph infers a broader subject relation using specified rules. A reader sees the resulting connection but assumes that a source directly asserted it. Provenance should distinguish supplied triples, inference rules and derived outputs. The logical derivation can be entirely correct under its rule set while a source-level match remains uncertain. State the distinction so curriculum mappings and semantic crosswalks do not advertise inferred alignments as official equivalences.
4. A replicated source shows two apparently authentic copies
Two storage mirrors contain different bytes under the same release name. Each mirror serves the files from a secure connection, but neither connection alone proves which copy is authoritative. Check independently published release digests, signatures where relevant, source control and correction notices. Quarantine divergence until it can be resolved. A trustworthy system records competing versions rather than choosing the newest file solely because it arrived later.
5. A calibration replacement creates a false improvement trend
A series of laboratory observations moves from one instrument to another. The new device has a different sensitivity and calibration chain, while the analysis labels the entire sequence with one generic method. A visible improvement may reflect measurement change. Connect each observation window to its instrument identity, calibration and measurement uncertainty. When comparability is unsupported, publish separate segments or a qualified analysis rather than imply one continuous physical trend.
6. A data processor corrects only the final chart
An analyst notices that a plotted value is wrong and edits the chart image manually without updating the dataset or code. The immediate picture looks better, but later automatic exports reintroduce the error, and no reviewer can reconstruct the correction. Repair the authoritative source or transformation, version the corrected artifacts and document dependent outputs. A publication fix should close the underlying lineage defect, not create a disconnected decorative result.
7. A provenance identifier silently merges two laboratories
Two laboratories assign identical local run IDs. A warehouse combines them without a namespace, causing unrelated experiments to share one apparent origin. Every downstream result using the merged key becomes difficult to trace accurately. Use globally distinguishing identifiers or explicit composite keys, and preserve lab-specific context. A hash or index can store the mistaken key very efficiently; identity governance is what makes the relationship meaningful.
8. A deletion request conflicts with a required audit record
A user asks for personal data removal while an institution has legitimate retention obligations for selected audit information. Provenance design must distinguish which personal fields should be deleted, which records may lawfully remain, who can access retained evidence and when the obligation ends. Blindly preserving every historical version or deleting all traces can both be wrong. Resolve the applicable policy with authorised review and record the permitted disposition without publicly exposing private details.
9. A dashboard shows an apparently verified model estimate
A statistical model outputs a risk forecast with a narrow confidence interval, and its provenance log is complete. The log proves which code and inputs were used, not that the model assumptions are suitable or that the interval accounts for selection bias. Present the measurement definition, validation evidence and unresolved uncertainty separately. High-quality lineage makes scientific criticism possible; it should not be advertised as a guarantee of prediction accuracy.
10. An international dataset supplies incompatible units
One national table reports populations in thousands while another reports individual persons. A combined world total is off by a large factor because the import step preserves numbers but loses units. Field-level provenance and schema mapping should record each unit and conversion factor. Verify dimensional consistency before aggregating. A credible global comparison also needs compatible dates, geography and population definitions, not merely one numeric storage format.
11. An archive is retrievable but the computation cannot run
A research repository preserves the original input file and notebook. The notebook depends on a removed remote API and an undocumented package version, so reviewers cannot reproduce the reported output. Preservation of files is a necessary part of lineage, not sufficient computational reproducibility. Save appropriate dependency manifests, static permitted snapshots and explanatory methods without disclosing secrets. Mark any unavailable external step clearly rather than claiming every historical result can be regenerated.
12. A correction should follow the affected claim, not only the file
A correction changes the interpretation of one variable but leaves several source rows identical. A purely byte-based dependency checker may miss articles that paraphrased the old interpretation. Provenance at the claim and schema level helps identify affected explanations as well as calculated outputs. Reviewers should ask which assertions relied on the superseded definition and issue visible corrections where material. This closes the gap between data engineering and responsible publishing.
13. A public-source label hides a private intermediate dataset
A report lists only its public statistical source but silently enriched the result using restricted student records. The final output may reveal information or violate the intended data-use authority despite a harmless-looking citation. Record every relevant input in controlled internal provenance and publish an appropriate public methods statement. Disclosure boundaries can protect identities without concealing the existence of a material restricted contribution to authorised auditors.
14. A signed output arrives before a late source correction
An authorised report is digitally signed at noon, and an upstream correction is issued at 12:05. The original signature remains valid for the noon version; it does not establish that the report is the latest or should remain operationally authoritative. Publish a status and supersession relationship so readers can distinguish authentic history from current validity. Cryptographic proof and editorial currency are different obligations.
15. A dataset’s row count stays constant after a material correction
A transformation changes one record’s value but no rows are added or removed. A validation job checks only row counts and declares nothing changed. Lineage should track relevant field values, digests, versions and dependent calculations, not just record totals. Quality checks must correspond to the kinds of errors that matter for the stated scientific claim. A stable count can coexist with a completely reversed conclusion.
16. A training example is upgraded to operational evidence without review
An educational article explains provenance using an invented table and fictional records. Another publishing process extracts the table as if it were real official statistics because the ‘illustrative’ label was lost. Preserve synthetic-data status through extraction and reuse, and require validation before promoting any educational example into factual evidence. Provenance serves readers when it keeps clear boundaries among demonstrations, observed measurements and authoritative assertions.
Connected eduKate subject guides
Data and evidence foundations
Information rights and representation
- Knowledge Representation
- Data Governance
- Information Security
- Open Science
- Information Preservation Engineering
Primary provenance standards and research sources
- W3C — PROV-DM Recommendation
- W3C — PROV-O Ontology Recommendation
- W3C — PROV Primer
- W3C — PROV Constraints
- DataCite — Metadata Schema Documentation
All organisations, measurements and data values in the worked cases are invented for educational explanation. A real dataset may require permission, source-level rights checks, restricted review and protection of personal information.
Continue with The Gold Standard master, Education and Learning Transfers into Life, and How X Works.