The Gold Standard of Anthropology | Free-Listing Tasks, Salience Scores and Order Effects

1. Define a domain people can actually list

Back to contents

A free-listing question works only when participants recognise a reasonably bounded domain. “List everything important in life” mixes too many frames, while “name the illnesses that families here commonly treat at home” identifies a setting, activity and population. Pilot the prompt in the language of use, ask what participants think the category includes and record examples that reveal boundary disagreements. A domain is not valid merely because the researcher has a label for it. If some respondents list symptoms, others diagnoses and others treatments, the resulting frequency table combines different tasks. The first design decision is therefore conceptual: specify what counts as a member, who is expected to know the domain and which distinctions matter to the research question.

2. Treat a list as recalled knowledge, not an inventory

Back to contents

A participant reports items that are available during a particular memory search. They may know more than they mention, stop when the obvious answers are exhausted, avoid embarrassing terms or infer that the interviewer wants only a few examples. Long lists can reflect expertise, effort, talkativeness or extra prompting; short lists can reflect fatigue, uncertainty or a narrow interpretation. Absence from one list is not proof that an item is unknown, and absence from the sample is not proof that it is culturally irrelevant. Free lists are strongest as evidence about accessibility and distribution of recalled terms under a documented task, not as complete catalogues of everything a community knows.

3. Standardise the prompt without silencing local categories

Back to contents

Use the same core wording, introduction and neutral encouragement for every participant, while allowing clarification rules that were decided in advance. Translating word for word may change the implied category, so document the source wording, local versions and back-translation or collaborative review. Interviewers should not seed examples after the task begins unless that design choice is intentional and analysed separately. Record whether participants wrote, spoke, used cards or answered in a group, because mode affects pace and recall. Also note whether the list was spontaneous or followed discussion: a list produced after examples, observation or earlier survey questions is a prompted response, not the same measure as unaided recall.

4. Use a stopping rule that can be repeated

Back to contents

“Anything else?” can be a neutral invitation or a strong demand, depending on tone and repetition. Set a protocol: for example, ask the prompt, allow silence, use one standard neutral probe and close after the participant confirms completion. Record time, interruptions and whether a participant chose to stop. Avoid forcing a fixed number, because the final positions then become guesses; also avoid giving some respondents unlimited coaching while others receive none. If a second pass or category cue is useful, preserve it as a separate phase. A transparent stopping rule does not remove memory differences, but it prevents interviewer persistence from masquerading as cultural salience.

5. Build a visible coding dictionary

Back to contents

Raw lists often contain spelling variants, synonyms, nested categories and terms whose meaning depends on context. Keep the verbatim response and its original order, then create a separate coded field. Decisions to combine “herbal tea” with a named plant, or to split a broad term into several remedies, change both frequency and rank. Use bilingual or domain-informed coders where needed, record definitions and adjudications, and preserve uncertain cases. Do not merge terms merely because they look similar in translation. Report results at more than one defensible coding resolution when conclusions are sensitive. The coding dictionary is part of the evidence, not housekeeping performed after data collection.

6. Worked example: five lists of home remedies

Back to contents

Five participants list remedies for an ordinary cough. The verbatim lists are: honey–ginger–rest; ginger–warm water–honey; honey–lemon; warm water–rest; and ginger–honey–herbal tea–rest. Honey appears in four of five lists and usually early; ginger appears in three and also tends to be early. A Smith-style salience score therefore places honey above ginger, while rest may have similar frequency but lower average rank. Before interpreting the order, the team checks whether “warm water” and “herbal tea” were prompted by the preceding nutrition interview and whether two language variants of ginger were combined consistently. The result supports a claim about readily recalled remedies in this small sample—not treatment effectiveness, universal use or the beliefs of all households.

7. Read frequency and rank as different signals

Back to contents

Frequency asks how widely an item is mentioned across lists. Mean or median rank asks how early it appears among people who mentioned it. Smith’s S combines occurrence with relative list position, giving early items in shorter lists more weight than late items. That can summarise central tendency, but it also inherits variation in list length and memory order. Publish the component information alongside the combined score: number of participants, mention proportion, ranks, list-length distribution and uncertainty or resampling where appropriate. A small numerical difference between two items should not be turned into a sharp cultural hierarchy without stability checks.

8. Test order effects instead of assuming a natural sequence

Back to contents

Early answers can be highly accessible, but they can also be primed by the introduction, a visible object, previous questions or the first response itself. Later answers may require deeper search, yet fatigue and perceived interview expectations can suppress them. Randomise preceding modules when feasible, separate aided and unaided phases and compare ranks across interviewers, languages and settings. If participants complete repeated lists, counterbalance task order and allow enough time between them. “Mentioned first” should be interpreted as first under these conditions, not as an automatic measure of importance, preference or frequency of real-world use.

9. Changed-context transfer check: ways to revise for school

Back to contents

Ask students to list ways they revise for an examination. Flashcards, practice papers and rereading may dominate because the interview happened beside a study-skills poster. The same students might list teaching a friend after a classroom demonstration. The transferable method is to separate spontaneous accessibility from environmental cues, retain original order and compare prompt contexts. The substantive frame changes too: a salient method is not necessarily an effective one, and a list of named strategies does not show how often or how well they are used. Follow-up questions or observation are needed before making claims about behaviour or learning outcomes.

10. Know the limits of salience scores

Back to contents

Salience is sample-, prompt-, language- and coding-dependent. Scores from two communities are not automatically comparable if list instructions, interview modes or term aggregation differ. A high-scoring item may be memorable because it is feared, controversial or recently publicised rather than typical. Small samples may give unstable rankings, and unequal knowledge opportunities can make dominant groups appear to define the domain. Do not use an arbitrary salience cut-off as if it discovered a natural boundary. Use qualitative accounts, subgroup displays and alternative codings to understand why items appear, and state clearly what the score cannot establish.

11. Audit the path from speech to ranking

Back to contents

Archive the exact prompts and translations, recruitment frame, interview order, stopping protocol, raw ordered lists, timing, interviewer identifiers, coding dictionary and analysis code. Recalculate results with plausible synonym groupings, without obvious primed items and within important subgroups. Inspect list length by interviewer and setting. Show how frequently the top-ranked set changes under bootstrap resampling or leave-one-out checks when the sample is small. A reader should be able to reconstruct every score from the preserved lists and see which judgement calls materially affect the result.

12. The gold standard keeps recall conditions attached

Back to contents

A strong free-list study begins with a domain participants recognise, elicits lists under a repeatable protocol, preserves verbatim order and makes coding decisions inspectable. Frequency, rank and combined salience are then descriptive tools rather than cultural truth machines. The best interpretation asks what was readily recalled, by whom and under which conditions, then checks that finding through interviews, observation or another elicitation method. Free listing is valuable because it opens a domain efficiently; its credibility comes from keeping the route from prompt to claim visible.