The Gold Standard of Data

Data becomes useful in relation to a question. A count of visitors, a set of temperatures and a record of transactions each answer different things. Begin by stating what someone needs to understand or decide and which observations could inform that decision.

A vague instruction to collect everything often produces unnecessary work and responsibilities. It may also distract from a small piece of missing information that actually changes the answer. Ask what evidence would support or challenge the proposed conclusion.

The Data Analysts branch develops the relationship between questions, evidence and decisions. This root follows the information through its full life. The standard is not a large dataset by itself; it is an accountable connection between a question, the observations gathered and the claim eventually made.

Contents · Next chapter

A row may represent a person, a visit, a household, a transaction or a measurement. These units are not interchangeable. Counting visits does not necessarily count distinct visitors, and counting submitted applications does not necessarily count the people who tried to apply.

Write the unit in plain language. Then explain whether one entity can appear more than once and how related events are connected. This prevents a later analyst from making an apparently reasonable calculation on the wrong basis.

In a fictional school library dataset, a borrower can make several loans. The number of loans measures activity, while the number of distinct borrowers answers a different participation question. Both can be useful when labelled correctly. The first defence against a misleading total is knowing what each record actually represents.

Contents · Previous chapter · Next chapter

Metadata explains information such as meaning, units, collection method, time, location and relevant limitations. It helps another person interpret the values without relying on the memory of their original collector.

A column called score leaves important questions open. Which assessment? Which scale? Was an absent student recorded as zero, missing or not included? A comparison can fail when those conditions differ, even if the spreadsheet looks tidy.

Document definitions where they can travel with the dataset and remain accessible to authorised users. The Library and Information Science branch connects discovery, metadata and access. For live services, Environmental API Station Metadata Change Notifications demonstrates how changed context can affect interpretation. Data quality includes preserving meaning through time, not merely preserving the characters in a file.

Contents · Previous chapter · Next chapter

The population is the group or set of events the question concerns. The available records are what the collection process actually captured. They may overlap closely or differ in important ways. State that relationship before making a general claim.

A survey conducted through one digital channel may omit people who cannot use it. A service dataset may record completed transactions while excluding unsuccessful attempts. Increasing the number of responses within the same process does not automatically remove that coverage problem.

Ask who or what could be missing and why. Where possible, improve the design or use suitable additional evidence. The Market Research Companies and Inclusive User Research branches develop specific collection contexts. The root habit is to keep the intended population visible rather than let a convenient file silently define the whole world.

Contents · Previous chapter · Next chapter

Every collected field should have a reason. Extra information can increase burden, create privacy responsibilities and introduce opportunities for error. A project should be able to explain how each field supports its task.

Consider whether less identifying or less detailed information would be sufficient. The answer depends on the purpose and circumstances. Anonymous classroom examples can often use invented data, while operational services may need verified identities for particular actions.

Plan access, retention and appropriate handling alongside collection. These decisions should not wait until a file has already been shared. The Data Protection Impact Assessments branch examines proportionate processing and risk. A clear purpose helps both technical quality and responsible governance because it gives the team a basis for deciding what belongs in the dataset.

Contents · Previous chapter · Next chapter

A collection method should make important distinctions easy to record correctly. Clear questions, suitable response options, appropriate units and useful validation reduce ambiguity. Pilot the process before treating the resulting values as dependable evidence.

Validation can check format or range, but it cannot establish every aspect of truth. A valid date may still be the wrong date. A number within the allowed range may still refer to the wrong person or instrument.

Record unusual conditions and changes to the method. If a question is rewritten halfway through a survey, later responses may not be directly comparable with earlier ones. Collection quality is therefore a combination of design, competent practice and documentation. The information should remain understandable after the original team has moved on.

Contents · Previous chapter · Next chapter

Zero is a value. Missing means that the relevant value is not present or not known in the record. A missing observation should not automatically become zero, because doing so changes the meaning of later calculations.

Different reasons for missingness can matter. A participant may not respond, an instrument may fail or a question may not apply. The appropriate treatment depends on the analytical question and the collection process.

Report the amount and relevant pattern of missing information. Where a method fills or estimates values, identify that intervention and its assumptions. The School Scientific Missing Outcome Data Analysis and Environmental API Missing Data Treatment articles provide focused routes. Good data practice makes the absence visible enough for someone to judge its consequences.

Contents · Previous chapter · Next chapter

Two identical-looking records can be a duplicate or two legitimate events. Deciding requires an understanding of the unit of observation and the rules for identity. Deleting similar rows without that context can remove real information.

A person may submit a form twice because the first attempt gave no confirmation. Two family members may share an address and surname. An instrument may correctly record the same value at consecutive times. Each situation needs a different interpretation.

Define the matching criteria and preserve a suitable audit trail for changes. Where uncertainty remains, flag the records rather than silently force a conclusion. The Database Administrators branch develops identity and integrity within maintained systems. The root principle is that cleaning should improve fidelity to the underlying events, not merely reduce the number of inconvenient-looking rows.

Contents · Previous chapter · Next chapter

Measurements need units and a defined scale. Mixing metres with centimetres or rates based on different periods can produce a neat but meaningless total. Identify conversions and preserve the original context where necessary.

Time also needs a clear interpretation. Does the timestamp record when an event happened, when it was observed or when it entered the system? Time zones and reporting periods can affect which records belong in a comparison.

Students can practise by deliberately mixing two simple units and explaining why the first calculation fails. Professional systems extend the same discipline through shared definitions and controlled transformations. The Computing root connects representation with system behaviour. A correct calculation requires quantities that can legitimately be combined, not merely values stored in compatible numerical fields.

Contents · Previous chapter · Next chapter

Accuracy concerns closeness to an appropriate reference or true value, while precision concerns the closeness of repeated results under the relevant conditions. Consistency can also mean that definitions and methods remain aligned. These ideas answer different questions.

An instrument can produce tightly grouped readings that share a systematic offset. A dataset can be consistently formatted while containing incorrect classifications. More decimal places do not resolve those problems.

Identify the quality characteristic needed for the decision and the evidence that can assess it. In specialised measurement, qualified methods and uncertainty evaluation may be necessary. In a classroom exercise, students can compare repeated invented readings with a stated reference. The learning objective is to avoid using one attractive quality word as a substitute for a complete account of how the data was obtained.

Contents · Previous chapter · Next chapter

A summary reduces detail so that a pattern can be understood. Mean, median, counts, rates and distributions serve different purposes. Choose the summary according to the question and the structure of the observations.

An average can hide a small group experiencing a very different outcome. A total can rise because the population grew, even when the rate stayed similar. A percentage needs its denominator and relevant period.

The Statistics root develops these relationships. In practice, describe what was summarised, which records were included and what the summary leaves out. Add appropriate detail when a single figure would distort the decision. The goal is a compact explanation that preserves meaning, rather than a number selected because it makes the result look favourable.

Contents · Previous chapter · Next chapter

Comparing two groups or periods requires compatible definitions, collection methods and boundaries. A difference may reflect real change, changed measurement or both. Ask which explanation the evidence can support.

An organisation may improve a response rate by changing who is counted as eligible. A sensor series may change when the station moves. Without context, the graph can suggest a performance change that the underlying service did not produce.

Record revisions and method changes close to the comparison. The Environmental Sensor Relocation Data Continuity and Historical Metadata Reprocessing articles examine specific forms of this problem. A credible comparison can acknowledge a break in the series. Preserving that limitation is more useful than forcing a smooth trend through incompatible observations.

Contents · Previous chapter · Next chapter

Two variables can change together without one causing the other. Other factors, selection and timing can influence the relationship. An observed association is a useful starting point for investigation, but it does not automatically establish an intervention’s effect.

Ask how the data was generated and which alternative explanations remain plausible. A stronger causal claim requires an appropriate design and reasoning about the context. A sophisticated chart or model does not remove that requirement.

In a fictional school example, students receiving extra support may have lower initial results than those not receiving it. A simple comparison of final scores can miss that starting difference. The lesson is to identify the question and design before choosing the conclusion. Specialist statistical guidance belongs in the relevant research context, while the root habit is disciplined interpretation.

Contents · Previous chapter · Next chapter

A model simplifies a situation to make a prediction or explanation possible. Its usefulness depends on the assumptions, the data and the conditions of intended use. A good fit to past observations does not guarantee useful performance in a changed setting.

Separate the data used to develop a model from the evidence used to evaluate it where the method requires that distinction. Inspect relevant groups and failure cases rather than relying on a single overall score.

State what the model output means and what decisions remain with people. Generated or synthetic records should be labelled appropriately and should not silently replace observations of the real world. The Computing and Statistics roots provide connected foundations. The data root asks whether the input meaning and the output claim have remained aligned.

Contents · Previous chapter · Next chapter

A chart should help the intended audience understand a question and the evidence. Titles, units, labels, scales and relevant notes carry essential meaning. Colour should not be the only means of distinguishing information needed for interpretation.

Provide suitable text or data alternatives and consider assistive technologies and different viewing conditions. Accessibility concerns whether the reader can obtain the same important information, not merely whether a chart image has a generic description.

The Accessible Data Visualisation Services and Accessible Dashboard Design branches develop these communication tasks. Test the actual interpretation: what conclusion does a reader take away, and is it supported? A chart can be technically accurate in its points and still mislead through its framing or omitted context.

Contents · Previous chapter · Next chapter

A database supports rules about identity, relationships, valid changes and access. Its design should reflect the underlying task rather than simply gather every field in one place. Different users may have different legitimate permissions.

Concurrent actions require particular care. Two requests can each appear valid in isolation while jointly violating a rule, such as reserving the same final place. The system needs an appropriate mechanism to preserve that rule.

The Database Administrators branch connects integrity with controlled access and tested recovery. A backup is useful evidence only when the required information and tasks can actually be restored. The root standard follows the record through creation, change, use and recovery so that its meaning remains dependable over time.

Contents · Previous chapter · Next chapter

Data lineage describes where information came from and how it changed on the way to a result. A report should have enough traceability for a competent reviewer to understand its important transformations and reproduce the relevant reasoning.

Corrections are part of responsible data work. Identify what changed, why and which outputs may be affected. Avoid leaving an old download and a new dashboard with conflicting figures and no explanation.

The Corrected Dataset Release Validation and Bank Risk Data Aggregation articles show how lineage matters in specialised services. Their requirements differ, but the common principle is a reliable chain from source to decision. A correction becomes most useful when the people who relied on the earlier result can recognise and respond to it.

Contents · Previous chapter · Next chapter

An open licence concerns particular rights to use material. It does not automatically establish that every piece of information can responsibly or lawfully be published. Free-text responses, unusual combinations of attributes and attached files can still identify people.

Review the content and the disclosure context, not only the licence label. Removing a name may leave other identifying details. Appropriate safeguards depend on the material, purpose and applicable duties.

The open research data and free-text identifiers case study illustrates this distinction. The Legal Document Redaction branch addresses another specialised disclosure setting. A root guide should teach the question clearly: permission to reuse and permission to reveal personal or protected information are different matters that may require separate assessment.

Contents · Previous chapter · Next chapter

Information should have named responsibility and an appropriate lifecycle. Define who can access it, how use is reviewed, when retention should be reconsidered and how authorised disposal is verified. Different records may be subject to different obligations.

A generic delete-after period is not suitable for every context. Legal preservation duties and other requirements may affect what can be removed. Equally, keeping information indefinitely because storage is cheap can create unnecessary responsibilities.

The Retention and Disposal Governance, Secure Data Erasure and Data Processing Agreement Governance articles develop distinct parts of the lifecycle. The root connects them to the original purpose. Responsible data work explains why the information exists and how its handling remains justified as that purpose and the surrounding conditions change.

Contents · Previous chapter · Next chapter

Official statistics support public understanding through defined methods and institutional responsibilities. UNECE’s Fundamental Principles of Official Statistics provides an authoritative reference for that professional setting. It is a framework for official statistical work, not an automatic quality certificate for every dataset.

When reading a public indicator, inspect the definition, coverage, period and revision notes. A rate and a total may each be correct while answering different questions. International comparisons may also depend on harmonised concepts and remaining differences.

Students can learn by locating the explanatory notes before repeating a headline number. That habit strengthens civic understanding: it allows someone to respect professional evidence while asking precise questions about what it means. The goal is informed use of the published information, rather than acceptance or dismissal based on the appearance of a single chart.

Contents · Previous chapter · Next chapter

A fictional service reports average completion time falling from ten minutes to eight. The calculation is correct for completed cases. A review finds that the most difficult cases increasingly stop before completion and are excluded from the measure.

The average answers a narrower question than the announcement suggests. Examine completion, abandonment and the distribution of times together. Investigate whether the service changed and whether the users in each period are comparable.

The correction is to preserve the useful calculation and improve its interpretation. Do not label every abandoned case with an invented duration. Report what is known and what needs further investigation. This joins missing data, denominators and service design in one example. The right next action depends on the actual cause of the incomplete journeys.

Contents · Previous chapter · Next chapter

In a fictional environmental dashboard, the screen refreshes regularly while the underlying observation has stopped updating. Users see a familiar number and assume that the conditions are current. The display is functioning, but the meaning of latest has become misleading.

Separate the time of observation, receipt and display. Detect stale values and communicate their status so that a user can judge whether they are suitable for the present task. A system should not make the reader infer freshness from page movement.

The Data Freshness Monitoring and Stale Data Alerting branches develop this service problem. The root lesson transfers widely: current presentation does not prove current evidence. A trustworthy system preserves the history needed to understand what the value actually represents.

Contents · Previous chapter · Next chapter

A useful learner task starts with a small invented dataset and a clear question. Define the rows, identify missing values, calculate an appropriate summary and explain what the result establishes. Then change one condition and ask whether the interpretation still holds.

For an emerging learner, the next step may be units or denominators. For a confident learner, it may be coverage, comparison or a causal claim. Diagnose the first unstable relationship rather than assign an undifferentiated collection of more calculations.

Return to Education: learning that transfers into life. Data literacy transfers when a person can question a number, preserve its conditions and explain its relevance without becoming either blindly confident or reflexively distrustful. The directory below provides the specialist routes for that next stage.

Contents · Previous chapter · Continue to the subject directory