Six pupils answer a class survey about why group projects become difficult. Their sentences are different, some mention two problems, and one pupil says there was no problem at all. A student researcher wants a bar chart. It is tempting to replace every response with the first category that comes to mind. That would make the graph tidy, but it could quietly change what classmates actually reported.
Coding open-ended survey answers means reading the original words and assigning clearly defined labels to ideas they contain, so patterns can be examined while the original responses remain recoverable. The useful procedure is to decide what one coded unit represents, create a small codebook with inclusion and exclusion rules, pilot it on a few answers, resolve ambiguous cases, count the codes correctly and report limits honestly. Do not invent meanings or force a mixed answer into a single category just to make the percentages add to 100.
Did you know? The code is an interpretation, not the original answer
A multiple-choice survey already gives participants predefined boxes. An open-ended prompt gives people space to explain in their own words. Coding helps the researcher organise those explanations, but every code is a decision about what a sentence means. Two pupils can use very different vocabulary to describe the same obstacle; one pupil can describe two obstacles; and a reply such as “I don’t know” may not describe any obstacle at all.
Professional qualitative research treats code definitions and the unit of analysis as part of the method. The US CDC’s published discussion of qualitative analytic rigour describes why coders should agree on meaningful units and check whether they are applying a codebook consistently. A school project can use a much simpler version without pretending that six sentences form a representative research study.
Worked example: six fictional responses, seven justified labels
Imagine an invented anonymous practice dataset from six fictional group members. The prompt is: “What, if anything, made your group homework difficult this week?” The answers are below. They are teaching examples only, not reports about real pupils, classes or school outcomes.
- A: “We could not find a meeting time.”
- B: “The instructions did not say what we had to hand in.”
- C: “Two people edited the same slide.”
- D: “Finding a time was hard, and nobody knew who owned the slides.”
- E: “Nothing was difficult this week.”
- F: “A teammate did not reply to the group message.”
Instead of making six different labels that merely repeat the answers, write a short working codebook. For Scheduling, include an explicit difficulty arranging a time; exclude complaints only about the task deadline. For Requirements, include unclear instructions or deliverables; exclude disagreement about how teammates distribute work. For Coordination, include clashes in editing or assignment of roles; exclude difficulty arranging a meeting unless a role problem also appears. For Communication, include difficulty getting messages or responses. No reported difficulty applies when the respondent explicitly states that nothing was difficult and does not also mention a problem.
Now apply the codebook conservatively. A gets Scheduling. B gets Requirements. C gets Coordination. D gets both Scheduling and Coordination because its two clauses clearly express two different obstacles. E gets No reported difficulty. F gets Communication. Preserve every response next to its assigned label or labels, and record why D was coded twice.
Hand-check the totals. Scheduling appears in 2 of 6 responses, or 33⅓% of respondents. Coordination also appears in 2 of 6, again 33⅓%. Requirements, Communication and No reported difficulty each appear in 1 of 6, or 16⅔% each. The total number of code assignments is 2 + 2 + 1 + 1 + 1 = 7, although there are only six respondents. The percentages therefore sum to 116⅔% when calculated from unrounded fractions. This is not a counting mistake: D legitimately contributes to two separate categories.
The accurate reporting sentence is: “In these six fictional replies, two mentioned scheduling and two mentioned coordination; one reply included both.” It would be wrong to say that a third of all pupils in the school struggle with scheduling. Six deliberately invented examples do not estimate a school population. It would also be wrong to label the bar chart’s vertical axis “number of students with a unique single problem” when multi-coding is allowed. Label it “number of responses mentioning each theme.”
A quick diagnostic: what kind of mistake is the learner making?
- Vocabulary-only coding: the learner treats “could not find a meeting time” and “finding a time was hard” as unrelated because the words differ. Practise recognising the shared meaning.
- Overconfident inference: the learner interprets “did not reply” as proof of laziness or dishonesty. The answer supports a communication difficulty, not the teammate’s motive.
- Forced single choice: D is assigned only Scheduling, hiding the coordination issue. Review whether multiple codes are permitted by the study’s design.
- Unsafe denominator: seven code assignments are divided by six people and presented as one exclusive pie chart. Clarify whether counting people, mentions or individual code assignments.
- Missing-response confusion: a blank answer is coded as “No difficulty.” An unanswered item is not the same as an explicit statement that nothing was wrong.
Build a codebook another person can use
Give every label a short name, a plain-language definition, one positive example and one borderline non-example. Decide whether one response can receive several labels and whether a code is applied to a sentence, a phrase or the entire answer. In a classroom, a small codebook with four to six relevant labels is often easier to test than an elaborate scheme that nobody remembers.
If a new idea appears that does not fit, create an explicit “Other: specify” note or consider revising the codebook rather than mislabelling the response. Record the codebook version. If analysis has already begun, review earlier responses under the revised definition so the same rule applies throughout. An “Other” bucket must not become a way to avoid reading difficult answers.
Progressive practice: from shared decision to independent analysis
1. Model one decision with its evidence
A teacher codes A as Scheduling and underlines the exact phrase “find a meeting time”. Explain why this is not a Requirements problem. The point is to connect the label to the respondent’s actual wording.
2. Compare a borderline answer together
Next examine D as a class: two clauses, two possible codes. Ask pupils to name the specific words supporting each label. One alternative codebook might split Coordination into “editing conflict” and “role ownership”, but only if the research question and number of responses justify that added detail.
3. Independently code a fresh sample
Give the learner four new fictional replies: “The deadline was moved,” “We could not agree on who would present,” “We couldn’t meet, but the directions were clear,” and “I am unsure.” Using the original codebook, the first does not automatically fit Requirements (unclear directions) and may need a separate “Changing deadline” label; the second fits Coordination; the third fits Scheduling but not Requirements; and the fourth is uncertainty, not “No difficulty”. The important achievement is an explained classification, including justified refusal to force a fit.
4. Test consistency without overclaiming
Two learners can independently code the same small, consent-safe practice sample and compare assignments. If they disagree, inspect the words and definition before deciding. Agreement helps reveal clear instructions, but it does not prove that the categories capture every possible interpretation. For a small classroom activity, document disagreements plainly rather than announcing a sophisticated reliability statistic from a handful of examples.
What students, parents and teachers can do next
Students can keep a two-column sheet containing the unedited response and its codes, plus an optional decision-note column. Parents can ask, “Where in the answer did you find that idea?” Teachers can assess the accuracy and transparency of the coding rather than rewarding the most colourful chart. Ask whether a new pupil could understand the definitions and reproduce the classification with minimal prompting.
Avoid collecting personal or sensitive details just to make practice look realistic. Use invented or appropriately consented, anonymised text; do not publish identifiable peer comments. If respondents have been told their words will be used only for a limited purpose, respect that. Six fictional sentences are sufficient to teach the mechanism safely.
Limitations worth writing below the chart
Coding compresses language and can lose tone, sarcasm or context. A single response may contain contradictory ideas. Different wordings of the survey prompt can elicit different answers. Choosing categories after seeing results may influence the interpretation. If a theme appears in two of six examples, do not extrapolate it to every class. When the original answers are available, readers should be able to inspect anonymised representative examples under the applicable privacy rules.
Frequently asked questions
Should categories add up to the number of respondents? Only if each respondent is assigned exactly one category under a valid exclusive coding rule. Multi-code analysis can have more assignments than people.
Can I correct a student’s spelling before coding? You can note an obvious spelling interpretation, but preserve the source wording. Never silently replace a respondent’s meaning with one you prefer.
What if two coders disagree? Revisit the exact text and codebook inclusion rules, document the resolution and apply the refined rule consistently to earlier examples.
Is a blank answer evidence of no difficulty? No. Record it as unanswered or unusable under clearly stated rules; do not invent a substantive response.
Can a short open-ended survey prove why groups fail? No. These answers describe perceptions and reported experiences. Causation and generalisability require additional design and evidence.
Listen first; count second
A useful chart should preserve the story that produced it. For related Core Aim practice, read why an unanswered survey item is not a No vote and how changes in survey wording affect comparisons. The eduKate Education Hub provides broader learning pathways, and the Clementi Secondary 1 Mathematics reference models diagnosing a specific misunderstanding, practising it and checking a fresh example. The aim is not to turn every sentence into a number; it is to organise what people actually said without taking their meaning away.