eduKate Singapore

VIEW THIS AS

Auto mode follows the Route Engine until you choose a viewpoint.

YOU ARE HERE

ROUTE CHECK

CONNECTED TO

WHAT NEXT

Use the canonical route for this room, or HELP if you are unsure.

Next useful routeCanonical hubHELP

The Gold Standard of School Citizen Science Duplicate Observation and Dataset Reuse Control

—

by

eduKate Asia
Students work with books, notes and a tablet at a shared library table beside tall windows overlooking the city.

A class uploads observations from a supervised butterfly survey. One pupil photographs the same butterfly twice, another pupil photographs it from a different angle, and a teacher later exports both the app records and a spreadsheet. Suddenly the research dataset reports five butterflies where the class may have observed only two. Nobody has deliberately invented data. The records have simply travelled through different hands and systems. The gold standard of school citizen science duplicate observation and dataset reuse control is to preserve what was actually observed, make the relationships among records explicit, and prevent repeated submissions from becoming false ecological evidence.

Quick answer: Define the survey event and observation unit before collecting data; assign persistent local identifiers; distinguish multiple photographs from multiple organisms; recognise exact copies, possible duplicates and genuinely independent sightings; review candidates with appropriate expertise; preserve originals and a reversible decision log; protect children and sensitive species; and publish only a dataset whose purpose, permissions, provenance and uncertainty are clear. Removing every similar-looking entry is no more scientific than counting every upload as a new sighting.

This guide builds on The Gold Standard of School Citizen Science Data Contributions. The earlier guide explains responsible collection. This one follows observations after capture, when evidence is copied, reconciled, combined, corrected and reused.

Quick navigation

The first question is: what is one observation?

A photograph is a media object. An observation is a claim supported by evidence. A sampling event is an organised occasion on which people followed a particular method in a particular place and time. An individual organism may appear in several photographs or observations, while an individual observation can contain more than one organism. Unless those levels are separated, a perfectly ordinary school field trip can create a remarkably convincing but scientifically useless spreadsheet.

Imagine two groups walking the same garden path. One group takes three photographs of a butterfly without losing sight of it. The second group sees a butterfly ten minutes later in the same flowerbed. The first three photographs may belong to one observation; the second group’s record may or may not be a separate encounter. Without continuous tracking or distinctive evidence, the data cannot prove whether two unique butterflies were present.

Before fieldwork, the teacher should write one sentence defining the counting unit, such as ‘one independent sighting event at a fixed survey station within a stated time window’. That is a study-specific rule, not a universal convention. Counts of sightings must not quietly become counts of distinct organisms. The published variable should say what was measured.

Three identities prevent one fact becoming three facts

Give each scheduled survey its own event ID, each field observation a durable observation ID, and each image its own media ID. Where a wider biodiversity platform is used, retain the platform’s published record identifier separately. These fields solve different problems. An event ID groups observations collected under shared effort; an observation ID identifies the record across exports and corrections; a media ID lets researchers connect photographs without treating an extra image as an extra sighting.

For example, EVT-2026-10-CLASSA-01 could be a survey session, OBS-001 a sighting, and IMG-001A and IMG-001B two photos from that sighting. The teacher’s later spreadsheet import might get IMPORT-02, but that is an import transaction rather than another ecological observation. Never overwrite the observation ID each time a row changes.

The GBIF occurrence-data requirements include an occurrenceID and eventDate among the core fields for occurrence datasets. Their exact field conventions apply when publishing to that infrastructure. A classroom can borrow the underlying identifier discipline without pretending every school spreadsheet already meets an international data standard.

Exact copies are not the same as suspected duplicates

An exact copy can arise when identical rows are uploaded twice, an export is imported again, or a record is mirrored between systems. A suspected duplicate arises when two different records may describe the same real-world encounter. A legitimate repeat occurs when the same species is observed in a new, scientifically meaningful event. These need different handling.

A file checksum may identify identical image files, but cropped or recompressed versions can have different checksums. Matching time, place and species can generate candidates, but may also match truly independent records. Two students standing next to each other could independently document the same event. Two separate species individuals could be near one another at the same moment. The reviewer must inspect the observation context, not merely obey a fuzzy-matching score.

Use three outcomes: confirmed same event, confirmed separate event, and unresolved. Keep the unresolved category visible rather than forcing confidence. A decision to link two records is often safer than permanently deleting one, especially when the classroom intends to release evidence for further review.

Set a field protocol that reduces confusion before data collection

A sensible field protocol assigns roles: observer, recorder, identifier and supervising adult. It specifies where the station begins and ends, which clock is authoritative, how repeated photos are labelled, and what happens when one pupil loses sight of an animal. Children need a short usable routine, not a twenty-page database manual. The teacher can turn the procedure into a pocket card and practise it indoors before the trip.

For example: record the event code; write the approximate observation time; identify the survey station; capture permitted evidence; note whether the organism remained in sight; create a new observation only when the method justifies one; and flag uncertainty. A teacher can then challenge a pair of fictional records during rehearsal and ask, ‘What would we need to know to decide whether these are the same sighting?’

The protocol should not invite pupils to pursue wildlife, enter restricted areas or reveal classmates’ locations. Scientific improvement never requires children to take unnecessary risks. When the programme has its own established recording convention, follow that convention rather than inventing a parallel school rule that breaks compatibility.

Worked case: six records, but how many sightings?

Consider a fictional survey in one school garden. Six incoming rows describe butterflies. The labels below are invented for teaching and are not data from NParks, GBIF or any real school.

Rows A and B were two photographs taken thirty seconds apart by the same pupil while following one butterfly. Row C is a separate child’s upload of the same original photo shared in a group chat. Row D is a sighting fifteen minutes later with no continuous tracking. Row E was recorded the next day. Row F is a teacher’s re-import of row E during spreadsheet cleanup.

A and B belong to one sighting record with two images. C is a copied image attached to that event, not another sighting. F is an exact import duplicate of E. D remains a potential independent encounter; the correct count depends on the survey’s stated measure. If the report counts documented encounter events, A/B/C, D and E can yield three events, provided D meets the study’s encounter rule. If the report counts distinct individual butterflies, the evidence does not establish three individuals. Report the uncertainty plainly.

A transparent reconciliation table

A useful review table needs the incoming row, its event, suspected relationship, evidence and decision. The row should not disappear from the audit just because its contribution to a published count becomes zero. Retain a reversible map between source and accepted records.

In the fictional example, an accepted-sightings table can store canonical IDs OBS-001 for A/B/C, OBS-002 for D and OBS-003 for E/F. Preserve every source ID in a separate relationship table. That design allows a teacher to correct a mistaken merge without losing the original upload or repeating the whole survey.

Where a data platform supplies its own occurrence identifiers, do not reuse a classroom canonical ID in a way that collides with another publisher’s records. Document the mapping from school IDs to platform IDs. This is especially important when combining annual school cohorts, where ‘OBS-001’ may recur unless the survey or dataset namespace is included.

How to identify duplicate candidates without over-claiming

Start with safe, deterministic checks: same persistent identifier, same imported file, or clearly repeated platform transaction. Next compare context: event, approximate timestamp, observation station, observers, original image lineage and taxon. Candidate rules should be reviewed against examples of genuine repeats, not tuned only to maximise the number of removed rows.

Some information is unreliable for automated matching. A mobile phone clock may be wrong. Two devices may record the same encounter in different time zones. The nearest location pin might be a school gate rather than the observation site. A species name may be corrected after expert review. Matching on one field alone creates false merges, while overly strict matching misses copies.

Keep candidate confidence separate from final decisions. A rule can say ‘possible duplicate—teacher review required’; it should not silently label a child careless or delete a contribution. The reviewer should record why the evidence supports each link, and an appropriately authorised person should sign off on changes to a released dataset.

Preserve effort and non-observations when removing repeat records

A common statistical mistake is to clean the species table while accidentally changing the denominator. Suppose a class conducts four independent 10-minute survey periods. One period contains several photos of one bird and the other periods record no birds. The number of uploaded images is not the number of survey periods, and the absence of bird sightings in the other periods is not evidence that those periods never happened.

Sampling-effort records should remain intact even when duplicate photos are linked. A zero count under a valid completed protocol differs from a missing survey or an abandoned period. Scientific reporting may compare sightings per unit effort only when methods and effort are sufficiently comparable. It cannot infer population density or trend from a raw upload count alone.

School Scientific Sampling Bias and Detection Bias in Observations are useful next readings. Both explain why perfectly tidy records may still be unrepresentative of the world.

Species names can change without creating new observations

A pupil might submit ‘butterfly’, a teacher might identify the family, and a reviewer might later confirm a species. These are successive identifications of the same evidence, not necessarily new observations. Preserve the identification history: who proposed each name, when, on what evidence, and whether it remains tentative.

A changed scientific name can also reflect accepted taxonomy rather than discovery of a new individual. Avoid a deduplication rule that requires the species labels to match exactly before considering two records. Conversely, a photo assigned to a new species cannot simply be merged with an older record because both were initially labelled ‘unknown butterfly’.

The appropriate data structure keeps the observation identity stable while maintaining separate determination records. This lets downstream users distinguish what the student saw from what a later expert concluded. The distinction is a practical example of scientific honesty: we are allowed to improve an answer without pretending that the original observation happened twice.

Protect children and sensitive ecological locations

Precise coordinates, timestamps, faces, school identifiers, device EXIF metadata and rare-species locations may create privacy or ecological risks. These concerns become more serious when scattered photos are combined into a searchable public dataset. A classroom’s consent to participate in a supervised lesson is not automatically permission to publish every image and location field worldwide.

Apply the governing programme’s safeguarding and publication rules. Remove unnecessary personal identifiers, use suitable geographic generalisation, and restrict sensitive images or locations where warranted. Record that generalisation or withholding has occurred rather than presenting blurred coordinates as precise GPS evidence. The GBIF data-publishing recommendations explicitly identify publisher responsibility for sensitive species information.

Do not use public internet photographs to ‘test’ whether a student’s home or school can be inferred. A teacher can demonstrate metadata risks on an invented example. The objective is to train responsible data stewardship, not to turn a children’s science project into an exposure experiment.

Publishing a dataset requires a licence and a provenance trail

A reusable dataset needs more than a CSV download link. It needs a title, description, date range, geographic scope, method, responsible organisation, data contact, licence and citation instructions suitable for the chosen repository. Explain exactly what a count means, how duplicates were adjudicated, how missing fields are represented and whether any locations have been deliberately generalised.

The licence must be compatible with rights in images, student work, third-party material and the destination’s rules. Open access to a photograph is not proof that a school can redistribute it. Check rights and permissions before publishing. If a project cannot obtain suitable clearance, it may still teach pupils rigorous data science with restricted data or a privacy-safe aggregate rather than pushing identifiable records into the public domain.

GBIF’s occurrence-dataset guidance specifies required metadata and accepted publishing conditions for that system. Schools should not imply GBIF acceptance until the chosen dataset is actually validated and published through the appropriate route.

Version the dataset, not just the filename

Once others use a dataset, a corrected record can change a published graph or a classroom claim. The original release must remain identifiable. A good release note distinguishes added observations, linked duplicates, reidentified species, corrected coordinates and changed publication permissions. It should say which results were affected.

A version number such as 1.1 is only useful when readers can find what changed from 1.0. Keep an immutable release copy where allowed, a current canonical version, and a change log mapping old record IDs to their present status. Do not quietly reuse a URL to serve incompatible data without explaining the revision. For a small school project, a controlled spreadsheet, read-only archive and teacher-approved change log may be enough.

A simple reuse statement might read: ‘Version 1.1 links two duplicate photograph uploads to the same field observation and corrects one identification; original survey effort and event count remain unchanged.’ That is far more informative than ‘data updated’.

Count the right thing when writing a graph

Three different figures can all be true: six uploaded rows, three accepted encounter events and an unknown number of unique individual butterflies. A chart labelled ‘Number of Butterflies: 6’ would be false if six actually refers to uploads. A chart labelled ‘Accepted sighting events under the classroom protocol: 3’ makes a narrower and defensible claim.

Use descriptive denominators. If two of six rows are exact copies, the import-duplication rate by incoming row is 2/6 = 33.3% under that specific classification. That does not mean a third of all children’s observations were invalid, and it certainly does not measure error in the local butterfly population. A record that has multiple photographs may remain a perfectly valid contribution.

Teach children to annotate every chart with time interval, location or generalised location, unit, data-release version and major uncertainties. That habit transfers to Mathematics, Science and public data reading long after the project ends.

A review queue should have owners, priorities and an appeal route

Not every possible duplicate needs equal attention. A record that changes a public conservation claim deserves closer review than an extra classroom image never destined for publication. Define who may merge or separate observations, which changes require expert identification, and who handles privacy escalation. If children contribute uncertain labels, a reviewer should provide supportive feedback about the evidence rather than treating uncertainty as a disciplinary problem.

A simple queue may contain: awaiting review, linked as same event, accepted as independent event, uncertain, withdrawn for safeguarding, and corrected in release. Reviewers should be able to reverse a mistaken action. Keep a record of disagreement and the rationale for the final decision. The existence of a formal-looking status does not make the science reliable; evidence and governance do.

Parents and pupils should know how to correct their own contribution where permitted. A correction route builds trust because it makes the dataset accountable to people as well as to downstream researchers.

Reuse questions matter as much as collection questions

A school project designed to teach observation is not automatically suitable for measuring changes in biodiversity over ten years. Ask prospective data users: was the route stable, was the observation duration consistent, were observers trained similarly, did identification methods change, and were repeat uploads handled under the same rules in every year? If those conditions differ, compare cautiously or restrict the claim.

A biodiversity atlas may accept opportunistic occurrence records without supporting a population trend claim from them. A standardised sampling-event dataset may allow a different kind of inference because it explicitly describes effort and non-detection. GBIF’s dataset-class guidance distinguishes occurrence records from sampling-event records. Classroom scientists should learn why both exist before selecting a publishing structure.

Good reuse begins with the question being asked. The same clean table can be strong evidence for ‘this species was observed here’ and weak evidence for ‘this species is becoming more common’. Explain the distinction in the article abstract, not in tiny footnotes.

A practical end-to-end school workflow

A robust classroom workflow moves through eight clear checkpoints. First, write the research question and counting unit. Second, obtain safeguarding permissions and choose the sanctioned platform. Third, train pupils on event and observation identifiers. Fourth, collect evidence with supervised field notes. Fifth, reconcile import duplicates and suspected repeated encounters without deleting originals. Sixth, validate species identifications, dates, effort and uncertainty. Seventh, review privacy, licences and release metadata. Eighth, publish or share only the version approved for its actual purpose.

At each checkpoint ask a different question: ‘What happened?’ ‘What evidence supports it?’ ‘Could this already be counted?’ ‘What do we not know?’ ‘Who may see it?’ ‘What can a future reader responsibly infer?’ Repeating these questions may sound slower than uploading everything, but it greatly reduces later disputes over whether the graph measures nature or simply enthusiasm for photography.

A teacher can complete the full teaching cycle with six invented rows and one class period. The point is not advanced software. It is careful thinking about evidence, identity, rights and interpretation.

How to judge whether the standard has been met

Assess the project on demonstrable outputs: a written observation-unit definition; example event/observation/media IDs; a tested review rule for duplicate candidates; a sample audit trail; consistent denominators; explicit uncertainty; privacy review; a versioned release; and a chart whose title correctly describes the measured quantity. A perfect-looking map without these controls is not a high-quality result.

Students should be invited to explain a difficult case. Ask why two photos are not necessarily two organisms, why two sightings need not prove two individuals, or why a correction does not erase the original observation. A pupil who can defend the distinctions understands more science than one who has merely filled in a template correctly.

Teachers can also check that the data are usable by someone who did not attend the trip. If an independent reader cannot tell what was counted or how a duplicate was handled, the project is not yet ready for reuse.

Common mistakes and better alternatives

Mistake: Delete every row with the same species and date. Better: review event, time, location and observation lineage; legitimate repeat sightings may share those fields. Mistake: count photos as individuals. Better: separate media count, encounter-event count and organism count. Mistake: round all GPS positions to the school gate and then treat them as precise. Better: document intended geographic generalisation and its analytical limits.

Mistake: treat an unresolved identification as a failed submission. Better: preserve the best supported taxonomic rank and the uncertainty. Mistake: let a cleaned spreadsheet overwrite the archive. Better: preserve the source, version changes and reversible decisions. Mistake: publish a public link because the project is educational. Better: check consent, rights, platform terms and sensitive-species policy first.

Each repair is an opportunity to teach a transferable idea: data quality is not an attractive spreadsheet. It is the ability to explain which claims the evidence actually supports.

Frequently asked questions

Should two photos of the same butterfly be deleted?

Usually not. Keep appropriate evidence, link both images to the same observation event when that relationship is established, and avoid counting the second image as an additional sighting. Different photos may improve identification even when they do not increase event count.

Does the same species recorded twice mean a duplicate?

No. A species can be sighted many times. Compare the protocol’s unit and the encounter evidence. Similar species, timing and location may flag a candidate for review, not prove that two rows are duplicates.

What if the class cannot identify the organism to species?

Record the best defensible identification level and the uncertainty. A higher taxonomic level may be appropriate. Keep the photographic and field evidence so a qualified reviewer can improve the determination later.

Can school observations be uploaded directly to GBIF?

Publishing to GBIF depends on the appropriate dataset, metadata, licence and recognised publishing route. Follow the programme’s rules and current GBIF documentation; a class spreadsheet or app account is not itself proof of a GBIF-validated dataset.

Does removing duplicate rows make a trend analysis reliable?

No. Trend analysis also depends on comparable locations, observation effort, seasonal timing, species detectability, identification quality and many other conditions. Deduplication corrects one type of problem, not every source of bias.

Why retain original records after a correction?

A reversible trail lets reviewers explain changes, recover from a mistaken merge and reproduce an earlier release. Retain originals under appropriate access and retention rules, without needlessly keeping private child information in public releases.

Official sources and connected Gold Standard reading

The GBIF occurrence-dataset requirements explain occurrence identifiers, dates, location fields and publisher metadata; the GBIF dataset-class guide distinguishes occurrence and sampling-event datasets; and the GBIF quality recommendations address sensitive-species information and consistency. Use the relevant programme’s current rules, including the NParks community citizen-science programme information, rather than treating this teaching example as a universal field standard.

Continue with School Citizen Science Data Contributions, School Scientific Observation Record Reconciliation, Observer Agreement Checks, Sampling Bias and Multi-Study Evidence Synthesis. Each addresses a different part of evidence quality.

Final principle: A good citizen-science dataset can trace every published claim back to a defensible observation, preserve uncertainty, explain repeated evidence and protect its contributors. One butterfly photographed three times should teach students three ways to think carefully—not become three butterflies.