The Gold Standard of Anthropology | Pile-Sorting Tasks, Similarity Judgments and Dimension Recovery

1. Define the domain before preparing the cards

Back to contents

A pile sort works on a bounded set of items. Those items may come from earlier free-listing, observation, interviews or an established inventory, but their selection fixes what participants can express. State the domain, population and item-selection rule. A threshold that removes rarely named items can make the task manageable while erasing specialist or minority categories. Preserve the full candidate list, selection frequencies and reasons for exclusions. If participants repeatedly request a missing item, treat that as evidence about the domain rather than a nuisance to be ignored.

2. Make the sorting prompt answer one question

Back to contents

“Put together the things that go together” is not neutral. Participants may sort foods by taste, meal, origin, price or moral value. Give a prompt tied to the research question without dictating the categories you hope to find. Explain whether one-item piles are allowed, whether all cards must be used and whether an item may appear in more than one group. A standard free pile sort usually requires each item to enter one mutually exclusive pile, so overlapping classifications need another design or a follow-up interview.

3. Treat presentation as part of measurement

Back to contents

Card order, print size, pictures, colour, spelling and handling space can alter attention and grouping. Randomise initial order, use consistent card design and pilot the task with the intended language and literacy range. Photographs may reduce reading demands but introduce background, scale and recognition cues. Online drag-and-drop interfaces change the available space and can make large piles easier or harder to see. Record the mode and layout so apparent cultural differences are not merely differences in presentation.

4. Record piles and the meanings participants give them

Back to contents

A pile membership table says which items were grouped; it does not say why. Ask participants to name or describe each pile and to explain difficult choices. Record uncertainty, rejected alternatives and items that could fit elsewhere. These comments help distinguish a taxonomic category from an occasion, value judgement or temporary strategy. Keep the participant’s wording beside the sort rather than replacing it immediately with an analyst label. Qualitative explanations are part of the evidence used to interpret later dimensions.

5. Build a similarity matrix transparently

Back to contents

For each participant, an item pair receives a one when both items occupy the same pile and zero otherwise. Summing or averaging across participants produces a co-membership similarity matrix. With missing or unfamiliar items, divide each pair by the number of people who actually judged both, not automatically by the full sample. Decide whether every participant contributes equally and whether repeated or successive sorts are combined. Publish the item-by-item matrix or code that generates it; the final map cannot reveal denominator or coding mistakes.

6. Worked example: six everyday foods

Back to contents

Six cards show rice, noodles, bread, mango, banana and orange. Four participants place the three fruits together; three also place rice and noodles together, while bread shifts between staples and breakfast food. Fruit pairs therefore receive similarities near one, rice–noodles receives 0.75 and bread has weaker ties. A two-dimensional map separates fruits from staples, but the second axis is ambiguous until participants’ labels reveal “cooked meal” versus “ready to eat.” The analyst reports that interpretation as a supported reading, not as a dimension mechanically discovered by the software.

7. Use maps and trees as models, not photographs

Back to contents

Multidimensional scaling places frequently co-sorted items nearer together while approximating many pairwise relations in a few dimensions. Cluster analysis joins items into a hierarchy according to a linkage rule. Stress, dimensionality, rotation, local minima and cluster method all affect the picture. Inspect one-, two- and three-dimensional solutions, report fit and compare maps with the raw similarities. Axes have no inherent names, and a dendrogram branch is not proof that participants possess a natural hierarchical taxonomy.

8. Changed-context transfer check: revision strategies

Back to contents

Students sort revision activities such as rereading, self-testing, summarising, teaching a peer and spacing sessions. Some group by social setting, others by effort or by when the activity occurs. The transferable workflow is to keep the item set and prompt explicit, compute co-membership and recover meaning from explanations. The context changes the claim: the map describes perceived relationships among study practices, not their effectiveness. Evidence about learning outcomes must come from a different design.

9. Look for subgroup structure without manufacturing it

Back to contents

Age, role, language or experience may shape sorting, but subgroup comparisons need enough participants and comparable item familiarity. Compare similarity matrices, individual coordinates or agreement patterns using prespecified groups and uncertainty. A pooled map can hide two coherent systems; small subgroup maps can also exaggerate random variation. Begin with the participant explanations and sampling design, then test whether the apparent difference survives alternative distances, dimensions and influential participants.

10. Know what the task cannot establish

Back to contents

A fixed card set cannot reveal excluded concepts, and co-sorting does not prove that items are linguistically synonymous, causally linked or equally valued. Forced single membership hides cross-cutting categories. Successive pile sorts may encourage a hierarchy that the domain does not otherwise have. A visually tidy map can arise from sparse data or dominant items. Pile sorting elicits relational judgements under a particular task; it does not read an untouched cultural structure directly from the mind.

11. Audit the path from cards to dimensions

Back to contents

Archive the item-generation rule, card faces, randomisation method, prompt, setting, pile memberships, pile labels, missing-item handling and analysis code. Recalculate similarities under plausible denominator rules, remove the most salient items and compare alternative MDS dimensions and cluster linkages. Examine stress, residuals and respondent-level variation. A robust interpretation should remain recognisable when display and modelling choices change, and every dimension label should be traceable to participant language or independent ethnography.

12. The gold standard keeps the sorting act visible

Back to contents

Pile sorts are strongest when a simple activity opens a disciplined conversation about classification. The gold standard defines the domain, standardises presentation, preserves explanations, constructs pairwise similarities transparently and treats maps as fallible summaries. It then checks subgroup variation, omitted items and alternative solutions. The result is not a declaration that one picture is the culture; it is a documented account of how particular people related particular items under a stated task.