The Gold Standard of Anthropology | Reviewer Withdrawal, Substitute Voices and Attrition Bias in Community Panels

1. Treat withdrawal as evidence about the process

Back to contents

Panel turnover is often filed as an administrative event: one person leaves and another person joins. Anthropologically, it can reveal whose time the project consumes, which forms of speech feel safe, whose knowledge is treated as optional and whether participation has become costly. That does not mean every departure is caused by the project. People leave for private reasons that they need not disclose. The useful rule is to examine patterns without converting individuals into explanations. Record when participation stopped, what obligations preceded it and whether burden, access or conflict changed across the panel. Ask what the panel can still claim after the change. A stable headcount can conceal a sharp loss of lived experience, local memory or minority dissent.

2. Define the panel population before anyone leaves

Back to contents

A panel needs a stated composition logic: which communities, roles, places, ages or forms of experience must be present, and why. That description should not imply that one reviewer represents an entire community. It sets the sampling frame for the review. Record recruitment routes, term length, expected workload, decision rights and whether alternates were invited from the start. Without this baseline, attrition cannot be interpreted because there is no clear denominator. “Two reviewers withdrew” means something different in a panel of six than in a rotating pool of sixty. The same is true when both departures come from one language group or caregiving role. Define the panel that was intended, the panel that participated and the panel that reached each decision.

3. Protect an unconditional right to withdraw

Back to contents

A reviewer should be able to pause or leave without pressure, loss of owed payment or a demand for personal justification. Exit procedures can offer choices: a private conversation, a brief anonymous form, a representative speaking on the person’s behalf, or no explanation. Researchers should separate consent to withdraw from permission to use earlier contributions. If a person asks for their identifiable comments to be removed, the team must know what can still be changed after deliberation or publication. Never frame a replacement plan as a reason the person must stay until a successor is found. Ethical departure protects the reviewer; methodologically, it also reduces coerced attendance that produces silence rather than genuine participation.

4. Build a participation history, not an attendance score

Back to contents

Keep a time-ordered record of invitations, attendance, materials received, accessibility support, speaking opportunities, recusals and decisions. The purpose is not to rank “good” and “bad” reviewers. It is to know which evidence each person could influence. A reviewer who misses a meeting may still annotate documents or consult their community; another may attend every meeting but have little room to speak. Record modes of contribution and the stage at which withdrawal occurred. Distinguish temporary absence, deliberate recusal, non-response, pause and confirmed exit. These states have different implications. A panel decision should identify the active membership and quorum rule at that moment instead of borrowing legitimacy from everyone ever named on the roster.

5. Understand why substitution changes the sample

Back to contents

A substitute is not a statistical copy of the person who left. Even when both share a broad demographic label, they bring different networks, biographies, trust relationships and interpretations. Replacement also changes inclusion probabilities: people available late in a project may differ from those able to commit at the start. Selecting only highly experienced “professional contributors” can restore procedural fluency while narrowing community diversity. Selecting through the same organisation can preserve an institutional link but not the withdrawn person’s viewpoint. State the property a replacement is intended to restore—language access, lived experience, locality, technical knowledge or an accountable connection—and report what remains missing.

6. Worked example: eight seats, two departures

Back to contents

An eight-person housing review panel includes two private renters, two public-housing residents, a disability advocate, a youth worker, a landlord and a local service provider. After three long evening meetings, one renter with shift work and the disability advocate withdraw. The team replaces them with a retired tenant leader and a national disability charity officer. The panel again has eight members, but its composition has changed: direct current experience of shift-work renting and local access barriers is weaker, while policy experience is stronger. The team records the change, pays all completed work, shortens meetings, offers asynchronous review and invites two alternates selected through the original local routes. Earlier and later decisions are reported with their actual membership rather than presented as one unchanged community verdict.

7. Use alternates without creating a shadow panel

Back to contents

An alternate system can improve continuity when it is designed openly. Alternates should know the role, receive materials, be compensated for preparation and understand when they may vote or speak. They should not do unpaid standby work or appear only when a convenient opinion is needed. A handover can cover project history and procedural facts, but it cannot transfer another person’s lived experience or presumed consent. Keep the withdrawn reviewer’s earlier words attributed only as agreed. If alternates attend throughout, state whether they are observers, contributors or members; otherwise the project may operate with an invisible second panel whose influence is omitted from reports. Continuity is valuable, but it must not erase who actually deliberated.

8. Changed-context transfer check: a school parent council

Back to contents

A school parent council loses two members midway through a policy review. One parent has a child with additional needs; the other works nights and joined remotely. Replacing both with daytime volunteers may fill the seats but remove the access constraints the review is meant to understand. The transferable method is to map participation by stage, protect withdrawal and define what each new member can and cannot restore. The school context adds safeguarding and child privacy: no one should infer a family’s reason for leaving. A changed membership notice can be factual and minimal, while the analysis tests whether recommendations made after the departures rely on perspectives no longer present.

9. Diagnose attrition bias with patterns and counterfactuals

Back to contents

Attrition bias becomes plausible when leaving is related to the burdens or outcomes being reviewed. Compare participation patterns across roles, recruitment routes, language needs, payment timing and meeting formats without publishing identifiable details. Ask a counterfactual question: would the conclusion plausibly differ if those who left had remained able to contribute? Revisit their previously authorised comments, unresolved objections and agenda items. Sensitivity analysis can present conclusions under alternative interpretations, such as treating unrepresented concerns as unresolved rather than assuming agreement. Small panels rarely support formal adjustment, and inverse-probability weights cannot manufacture missing viewpoints from a handful of people. Qualitative traceability is often more honest than a decorative statistical correction.

10. Audit membership, burden and decision lineage

Back to contents

Retain versioned rosters, recruitment pathways, role definitions, accessibility offers, workload estimates, payment records, attendance and contribution modes, withdrawal categories, replacement criteria and decision logs. Access to personal reasons should be restricted. For each major recommendation, record which panel configuration reviewed it and whether earlier objections were carried forward. Recalculate simple composition summaries before and after departures, but do not imply that demographic matching makes people interchangeable. Ask an independent reviewer or community governance group to examine whether the replacement process protected both the departing person and the panel’s stated inclusion goals. Publish limitations where continuity could not be restored.

11. Frequently asked questions

Back to contents

Must a reviewer explain why they leave? No. The project may invite feedback, but withdrawal must not depend on disclosure. Can earlier comments still be used? Only under the participation agreement and any later request that can practicably be honoured. Is a demographically similar replacement equivalent? No; similarity on one characteristic does not reproduce biography, networks or judgement. Should every absence trigger replacement? Not necessarily. Temporary absence, recusal and exit are different states. Can attrition be corrected with weights? Sometimes in large, well-modelled samples, but small deliberative panels usually need transparent qualitative analysis rather than pseudo-precision.

12. The gold standard keeps changing voices visible

Back to contents

A credible community panel is not a fixed label attached at recruitment. It is a changing body whose membership, burden and influence must remain visible through the work. The gold standard protects withdrawal, pays completed labour, distinguishes absence from exit, prepares alternates ethically and reports the panel that actually made each decision. It does not claim that a substitute inherits another person’s voice. By tracing what was lost, what was restored and what remains uncertain, the project turns turnover from hidden procedural noise into an honest limit on community accountability.