Gold Standard Practical Case Study 06 | When a 90% Response Figure Misleads

A fictional membership-information desk receives twenty enquiries during one review period. Its supervisor, Ren, publishes a summary headed 90% of enquiries resolved. The source spreadsheet counts eighteen acknowledgement messages. Sending an acknowledgement tells a person their request has arrived; it does not necessarily answer the request.

The desk’s local definition M0 says resolved means a supported answer or agreed alternative has been communicated, with the relevant completion evidence retained. Initial acknowledgement is separately recorded as received and owned. The review period, included channels and twenty unique enquiry identities are fixed before the calculation.

The distinction matters because a reader may use the report to judge whether people received useful answers. Ren’s task is not simply to perform division correctly. It is to ensure that the numerator, denominator and headline describe the same meaningful event.

Contents · Next chapter

M1 is the register of twenty unique enquiries in scope. M2 contains eighteen acknowledgement records. M3 contains twelve resolution records meeting M0. M4 identifies three enquiries waiting for specified information and five others with work still open. The three plus five account for all eight unresolved enquiries.

The acknowledgements and resolutions are overlapping event sets, not separate populations to add together. For this exercise, all twelve resolved enquiries are among the eighteen acknowledged ones. Six further acknowledged enquiries remain open. Two open enquiries have no acknowledgement record. These relationships are explicit simulated facts.

M5 is the original public summary containing the resolved headline and the eighteen-over-twenty calculation. M6 is the analyst’s reconciliation note linking every included enquiry to its current state. There are no records of satisfaction, learning, long-term retention or other outcomes beyond the enquiry decisions described.

Contents · Previous chapter · Next chapter

The recorded acknowledgement proportion is eighteen divided by twenty: 0.9, or 90%. The recorded resolution proportion under M0 is twelve divided by twenty: 0.6, or 60%. Both calculations use the same included enquiry population; they answer different questions.

A corrected report can therefore say that eighteen of twenty enquiries had recorded acknowledgements and twelve of twenty had recorded resolutions by the review point. It should also state that eight remain open, with three awaiting information and five in other open work. These numbers reconcile with the twenty identities.

Do not label the two without acknowledgement evidence as definitely ignored. The dataset establishes the absence of an acknowledgement record, not every possible interaction with the person. If the desk wants to determine whether a message was sent, it must examine an appropriate communication source and preserve any resulting qualification.

Contents · Previous chapter · Next chapter

Under GS-R08, the original resolution claim is Not met against M0: its numerator counts a different event. The error is one of meaning and measurement, not rounding. Recalculating eighteen divided by twenty with more decimal places would not repair it.

Under GS-R02, the supplied completion definition is usable because it distinguishes acknowledgement from resolution and names the required evidence. This case does not show whether all staff applied that definition consistently across earlier periods. It supports examining this report and these twenty enquiries.

GS-R12 concerns review and controlled change. Ren needs to revise the claim where readers encounter it, record what changed and maintain the link to M6. Quietly changing only a private spreadsheet would leave the misleading public account in place.

Contents · Previous chapter · Next chapter

In M7, Ren replaces the headline with the two bounded figures and adds the open-case breakdown. The correction note explains that the earlier summary counted acknowledgements as resolutions. M8 records a reviewer comparing the revised report to M1–M4 and confirming the arithmetic, event definitions and scope match.

This supplies evidence of a corrected report for the examined period. It does not demonstrate that a future report will avoid the same error. Ren assigns action MA1 to put the event definition beside each report field and require a reconciliation before publication. No later reporting cycle is included.

The open enquiries remain real work within the fiction. Ren cannot treat publication of the correction as resolving them. Each needs an owner and next action appropriate to its state. The report repair and the enquiry-response work should remain connected without being counted as the same completion.

Evidence-to-decision register

Contents · Previous chapter · Next chapter

Now consider a different proposed headline: 100% of enquiries we finished were resolved. If finished is defined by M0, the denominator selects the twelve already resolved enquiries. The arithmetic would be twelve out of twelve, but it would no longer describe completion across the twenty received requests.

That narrower statement might be mathematically true and practically unhelpful. The reader needs to understand the population excluded from it. The reviewer should ask whether the chosen denominator answers the stated question or removes unfinished work from view.

Another variation adds five enquiries received after the review period. They belong to a later scope. Changing the denominator midway without explaining the period would prevent comparison. Stable identities and scope protect the meaning of a trend before a chart or percentage is drawn.

Contents · Previous chapter · Next chapter

Exercise: replace the original headline with a paragraph that preserves all supplied facts. Include the two fractions, the eight open cases and the evidence limit for the two unrecorded acknowledgements. Keep the wording accessible to a visitor.

A reasoned answer states: among twenty enquiries in the stated period, eighteen had recorded acknowledgements and twelve had recorded resolutions under M0; eight remain open, including three awaiting information and five in other open work. Two enquiries lack acknowledgement evidence, so their communication status needs examination.

The wider lesson is useful for a learner, manager or reviewer reading any performance claim. Ask what happened, which people or cases were counted, what was left out and what the record establishes. A percentage becomes trustworthy when its meaning survives those questions.

Contents · Previous chapter · Gold Standard hub