The Gold Standard Reference | Evidence, Claims and Verification

A team can collect records for months without becoming better informed. The missing element is often a decision. “Collect feedback” does not specify whether the team needs to redesign an explanation, approve a new process, investigate an exception or substantiate a public claim. Each decision needs a different view of reality.

Write the decision before designing the evidence pack. A useful sentence is: “We need to decide whether this process can continue under these conditions, because these people depend on this outcome.” Name the conditions and people. The resulting evidence request becomes narrower and more useful than a general request for everything available.

Consider a fictional community workshop that promises to prepare the correct learning pack before each booked session. A manager deciding whether to continue a pilot needs evidence of correct packs, timely readiness and manageable staff workload. A participant asking why a pack was missing needs a case history and a repair. A reviewer considering a broad reliability claim needs coverage across ordinary and difficult operating conditions.

These views can use some of the same records, but they answer different questions. A resolved complaint does not establish general reliability. A good overall rate does not answer the affected participant's question. Evidence design begins by keeping those decisions distinct.

The decision brief

Record the decision owner, the claim under consideration, the population affected, the consequence of being wrong and the latest useful decision time. Then state what finding would change the action. If no possible finding could change the decision, the exercise may be documentation of a settled position rather than a genuine review. Make that limitation visible.

Contents · Next chapter

“Our process works” is too elastic to test. It can mean that the software opened, the operator followed instructions, the product was usable or the beneficiary received the intended outcome. A reviewer cannot evaluate all those meanings as if they were interchangeable.

A bounded claim identifies the outcome, scope, conditions, period and supporting evidence. For example: “In the recorded pilot sessions at this venue, the workshop checked pack identity against the booking list before release; the exception log identifies cases requiring repair.” This is less glamorous than “flawless delivery”, but it allows inspection.

Separate a claim about design from a claim about execution. A checklist can establish that a check was specified. Completed records may establish that someone recorded performing it. Observation or independent reconciliation may strengthen confidence that it actually occurred. Correct delivered packs support an outcome claim. None of these alone proves every other layer.

Match the public wording to the supported layer. Do not promote a successful demonstration into a claim of sustained capability without examining repeated operation. A demonstration is useful evidence when described accurately.

Contents · Previous chapter · Next chapter

The programme's evidence chain is requirement → observation → interpretation → decision → follow-up. Each step has a different job. The requirement establishes what should happen. Observation describes what was seen. Interpretation explains its significance. Decision chooses an action. Follow-up checks what that action produced.

In the workshop case, the local requirement might state that a pack's booking reference and session version must match before release. An observation records a particular mismatch. The interpretation identifies a failure of that criterion, with the cause still open. The immediate decision is to withhold and replace that pack. Follow-up checks both recipient resolution and whether the contributing process weakness persists.

Do not compress the chain into “staff error, fixed”. That phrase combines an untested explanation with an undocumented outcome. It omits the criterion, the specific event, the affected person's experience and the test of the repair.

Give cases stable identifiers. Keep the requirement edition attached to the finding. If the criterion changes later, the original decision should remain intelligible against the rule that applied at the time. A future reviewer should not need to reconstruct the past from the current version of a shared document.

The chain also works backwards. Start with a proposed conclusion and ask which decision it supports, which interpretation makes it reasonable, which observations support that interpretation and which requirement makes those observations relevant. A missing connection identifies where to narrow the conclusion or gather better evidence.

Contents · Previous chapter · Next chapter

Provenance is the history needed to understand where a record came from and what happened to it. For ordinary operational work, that often means identifying the source, creator or system, time, version and transformations. A graph copied into a presentation should still lead back to the records and calculations that produced it.

The W3C PROV Overview describes provenance through the things, activities and people involved in producing data. That provides useful context for this chapter. The simple record below is an original programme template; it does not claim compliance with the PROV technical specifications.

Preserve meaningful originals when correcting records. A correction should explain what changed and why, with enough history to distinguish a clerical repair from a changed interpretation. Silently replacing an inconvenient observation destroys the very evidence the review needs.

Do not retain every incidental trace merely because storage is available. Choose records that support a legitimate decision and manage access appropriately. The objective is a reconstructable account, not an indiscriminate archive. An index that locates a small set of dependable records is often more useful than thousands of files without ownership or context.

Contents · Previous chapter · Next chapter

Evidence can be authentic and still irrelevant. A real attendance sheet does not establish understanding. A genuine delivery timestamp does not establish correctness. A sincere testimonial does not describe everyone who used a service.

Test relevance first: does the item address the actual criterion? Then test credibility: is there a reason to trust its origin and meaning? Finally test sufficiency: does the collection cover enough of the claim's scope and conditions to support the decision? These questions should remain separate because a strong answer to one cannot automatically repair a weak answer to another.

For the workshop, a signed packing sheet is relevant to recorded preparation. Its credibility depends on when and how it was completed. Its sufficiency depends on whether it covers the sessions and failure modes included in the claim. If all sheets were reconstructed after a complaint, their status differs from contemporaneous records even if the signatures are genuine.

Use corroboration intelligently. A second source is useful when it can reveal something the first source misses. Two reports generated from the same mistaken booking file may repeat one error. A recipient's description, a physical pack check and a booking change history may provide different observations of the event.

More evidence is not always better evidence. Ten near-identical screenshots can add less than one carefully chosen observation of a previously unexamined handoff. Ask what new uncertainty each additional item resolves. Stop collecting when the remaining uncertainty is adequately bounded for the decision, or explicitly limit the decision when it is not.

Contents · Previous chapter · Next chapter

Sampling begins with the population, not with the number of records a reviewer has time to open. Define what could have been selected. Does the population include cancelled sessions, reopened cases, substitute staff, late booking changes and records that failed to save? Omissions in the population definition can make a tidy sample misleading.

Distinguish representative estimation from purposeful investigation. A sample chosen to estimate ordinary performance needs a defensible selection method for that purpose. A sample deliberately focused on difficult cases can help discover weaknesses, but its failure proportion should not be presented as the overall failure rate. Label the selection purpose.

Build a coverage map before deciding size. Identify conditions likely to change performance: workload, experience, shift, location, input complexity and process version. Select cases to answer the review question across those conditions. If a condition is excluded, limit the claim accordingly.

No universal sample size is prescribed here. The needed design depends on population structure, the consequence of error, expected variation and the inference required. A statistical assurance claim needs an appropriately justified statistical method. A small exploratory review should present itself as exploration.

Keep a record of how cases were selected, replaced or found unavailable. Otherwise, a reviewer may unknowingly replace difficult records with convenient ones and remove the very weakness under investigation. Missing cases are part of the evidence picture. They should not disappear from the account simply because they are awkward to inspect.

Contents · Previous chapter · Next chapter

An absent record does not automatically prove that the activity failed. Equally, it does not prove that the activity succeeded. The immediate conclusion may be that execution is unknown. That uncertainty can itself prevent a broader conformance statement when the requirement demands demonstrable control.

Suppose a readiness check has no retrievable record. If retaining that record was itself required, the recordkeeping criterion is not met. Whether the pack was actually checked may remain unknown. These are two findings about two different questions. Reporting them separately avoids both unjust blame and unjust confidence.

Distinguish “not observed”, “not recorded”, “record unavailable” and “record contradicts the claim”. Each suggests a different next step. A retrieval failure may call for access repair. Contradictory records call for reconciliation. An activity never observed under difficult conditions may need a targeted observation before the scope expands.

Do not fill gaps from habit. “The operator normally does this” is background information, not direct evidence of a particular event. A reconstruction can be useful, but label its date, sources and uncertainty. Keep it distinct from a contemporaneous record.

The programme's Unknown status protects the boundary between fact and assumption. It is not a neutral pass. Where the unresolved issue is material to the decision, the reviewer must narrow, defer or condition that decision. The action should follow the significance of the uncertainty, not the discomfort of admitting it.

Contents · Previous chapter · Next chapter

A plausible account becomes stronger when it survives serious alternatives. If failures fell after a new checklist, several explanations may remain: the checklist helped, workload fell, experienced staff took over, the case mix changed or reporting weakened. A before-and-after improvement does not by itself isolate the cause.

Write the leading explanation and at least one credible alternative. Ask what observation would distinguish them. If both predict the same dashboard result, that result is not enough to choose between them. A focused check of actual checklist use under comparable work may be more informative.

Keep contradictory evidence visible until resolved. A manager may remember that all packs were ready while the session log shows a late replacement. Clarify whether “ready” meant assembled, checked, released or usable by the participant. Apparently conflicting accounts sometimes use different boundaries. Sometimes one account is simply wrong. Both possibilities deserve examination.

Separate a reason for doubting a record from a reason for disliking its implication. A timestamp generated by an unreliable clock may need correction. A valid exception that weakens the preferred conclusion still belongs in the review.

Use a short unresolved-questions register: question, competing explanations, distinguishing evidence, owner and decision deadline. This turns uncertainty into work. Avoid endlessly collecting information that cannot discriminate between explanations. The next observation should have a purpose and a realistic chance of changing the conclusion.

Contents · Previous chapter · Next chapter

Experiences, explanations and observed behaviour often reveal problems that counts miss. A participant can explain why a technically complete pack remained unusable. An operator can show the moment two instructions conflict. Qualitative evidence belongs in the system when its collection and interpretation are clear.

Use questions that invite a description rather than a desired answer. “What happened when you tried to use the pack?” is more informative than “Was our excellent new pack helpful?” Record the context and distinguish a direct quotation, a paraphrase and the reviewer's interpretation.

When looking for recurring themes, describe how comments were grouped. Preserve unusual cases that challenge the dominant theme. Five similar comments from one session should not silently become a statement about every session. A theme identifies a pattern in the material reviewed; its wider prevalence remains a separate question.

Protect the person's meaning when shortening an account. An excerpt praising the tutor may coexist with a serious complaint about confusing materials. Removing the complaint changes the evidence. Likewise, do not expose personal detail that the review does not need.

Bring qualitative and numerical views together through a shared question. If counts suggest timely delivery but interviews describe unusable packs, examine the definition of completion. The disagreement may reveal that the metric measures release rather than successful use. The task is to improve the model of the work, not to force every source into agreement.

Contents · Previous chapter · Next chapter

A mathematically correct rate can describe the wrong thing. Check the arithmetic, but also verify the unit counted, inclusion rules, denominator, time boundary and treatment of reopened cases. These are part of the claim, not technical footnotes.

In a fictional review of 40 booked sessions, records show 36 on-time releases, two late releases and two sessions with unavailable timing records. The observed on-time count is 36 of all 40 sessions. Among the 38 sessions with timing information, the proportion is 36/38, approximately 94.7%. Those statements use different denominators and must not share an unexplained label.

Calling the result “94.7% on time” without disclosing missing timing information overstates what the records establish about the full population. A clearer report shows all three counts, explains the denominator and leaves the two unresolved cases visible. It does not invent their outcomes.

Trace one reported number back to individual cases, and trace selected cases forward into the summary. The first check tests whether the total can be reconstructed. The second can reveal excluded or duplicated cases. Both directions matter.

Volume 6 develops measurement definitions and thresholds in more detail. Here, the verification rule is to ensure that the evidence pack permits the calculation to be repeated and its interpretation to be challenged. A number that only one person can explain is a fragile foundation for a shared decision.

Contents · Previous chapter · Next chapter

A reviewer needs enough competence to understand the criterion and enough freedom to report an unwelcome finding. Being outside the immediate task can help, but distance alone does not create judgement. A person unfamiliar with the process may overlook the precise failure the review was intended to detect.

Define what the reviewer may inspect, who resolves disputed interpretations and what happens when evidence is unavailable. Disclose relevant involvement in designing or operating the work. A small team may need proportionate safeguards rather than an elaborate review hierarchy: another competent person checks material conclusions, and unresolved disagreements remain visible to the decision owner.

Ask operators to explain the work before judging a record against an imagined process. Then compare that explanation with requirements and observed practice. Familiarity should improve the reviewer's understanding without becoming an excuse to accept unsupported claims.

Record the finding and its basis in language another competent reader can follow. “Unsatisfactory” is too vague. “The reviewed release record does not identify which booking-list revision was checked, so the claimed version match cannot be verified” identifies the gap and the scope of the conclusion.

Give the process owner a chance to correct factual errors. Keep that separate from negotiating away a valid finding. A review becomes dependable when challenge improves accuracy while the decision standard remains stable. Volume 8 explains the corresponding decision rights and escalation arrangements.

Contents · Previous chapter · Next chapter

Use the programme's established statuses: Met; Partly met; Not met; Unknown; Not applicable with justification. Apply them against an identified criterion and scope. Do not use a favourable average to erase an unresolved critical gap.

Do not invent a percentage conversion for these statuses. They represent different logical findings, not equally spaced points on a scale. A requirement that cannot be verified should not receive half marks merely to make a dashboard convenient.

An unqualified internal conformance statement requires every applicable core requirement to be met within the stated scope, no unresolved critical gap and adequate evidence for the conditions claimed. A narrower statement about a successful pilot can still be useful when broader conformance is not established. Explain the boundary rather than stretching the finding.

Close the review with an action, owner and follow-up condition. Evidence has completed its job only when the decision is traceable and any required next step has somewhere to go.

Contents · Previous chapter · Next chapter

A claim ages when its supporting conditions change. New staff, a revised template, a different venue or a surge in demand may weaken the connection between old evidence and current performance. The records remain historical evidence; their ability to support today's decision must be reconsidered.

Attach review triggers to material claims. A scheduled review may be appropriate, but event triggers are often more revealing: a repeated exception, a change to the source system, an altered definition or the loss of a key control. Name the person who must notice the trigger and decide whether the claim should narrow, pause or be re-examined.

Keep a claim register with wording, scope, supporting evidence, owner, approval date and next review condition. Separate current claims from withdrawn or superseded claims. Preserve why the wording changed so future teams do not accidentally restore a stronger statement that the evidence no longer supports.

Public wording deserves the same discipline as an internal report. The reader should understand what is being claimed without needing to find a hidden qualification elsewhere. A headline that promises universality cannot be repaired by a distant paragraph describing one small pilot.

Evidence stewardship is ongoing work. The objective is not to defend yesterday's language indefinitely. It is to keep today's language aligned with what can presently be demonstrated, while making it clear what further work would justify a stronger conclusion.

Contents · Previous chapter · Next chapter

Use this fictional case. A workshop reviewed 12 sessions. Ten readiness sheets were completed before release. One was reconstructed the following day from a staff interview. One is missing. A participant reported that a pack contained an obsolete page even though its sheet was marked complete. The team proposes the claim: “Our checking process ensures every participant receives correct materials.”

First, identify the claim's scope and strength. Next, separate what each item establishes from what it leaves unknown. Then draft a review finding and a narrower statement that the available evidence can support. Finally, identify the next observation most likely to improve the decision.

A worked response

The proposed claim is universal and implies dependable prevention. The records describe only 12 sessions and include a contradictory outcome. Ten timely sheets support recorded use of a check; they do not alone prove that every page was correct. The reconstructed record has a different evidential status. The missing sheet creates an uncertainty about execution and may also breach a recordkeeping criterion.

The obsolete page is evidence of at least one outcome failure. The investigation should trace the applicable version rule, the source template and how the check was actually performed. It should not assume the operator ignored an adequate process. The checking method itself may have omitted page-version comparison.

A defensible statement is: “We reviewed checking records for 12 sessions. Ten were completed before release, one was reconstructed and one was unavailable. A reported obsolete page remains under investigation. The review does not establish consistently correct materials.” The next evidence request should examine the reported pack against the intended template and reconstruct that case's preparation and release sequence.

The reusable pack

Build a compact pack containing the decision brief, scoped criterion, case index, provenance notes, selection method, observations, unresolved questions, finding and follow-up. Ask a competent reader to reconstruct one conclusion without verbal assistance. If the reader can locate the rule, inspect the evidence and understand the decision boundary, the pack is beginning to serve its purpose.

Contents · Previous chapter · Continue through the reference program