Vocabulary | What Is a Corpus Sampling Frame? How the Texts You Include Shape Word Evidence

CHAPTER 01 OF 22 · A corpus sampling frame

1. Start with the claim you want to make

A corpus design begins with a sentence you hope the evidence can support. ‘Which words appear frequently in school notices?’ is narrower than ‘How do young people speak?’ Neither question is foolish, but they require different source populations. Write the intended conclusion before choosing convenient files; otherwise you may redefine the question to flatter the material already on your desk.

Read the evidence, not just the label

Imagine two folders. One has circulars sent to families; another has transcribed playground conversations. Both are language data. Neither is a universal sample of Singapore English. Their differences become informative only when the question names the audience, medium, place and communicative purpose.

The tempting mistake

An easy mistake is to collect a large number of readily available texts, analyse them carefully, then present the outcome as a portrait of the whole language. More text does not repair the absence of an explicit domain. A million tokens of official notices can remain uninformative about informal private speech.

What to do instead

State the domain in ordinary language: ‘English-language announcements issued by this school to families during one term.’ Say whether the goal is description, comparison, teaching-material selection or model evaluation. These uses may require different forms of coverage.

Try this: Does a word count from school circulars tell you how teenagers usually talk with friends?

Explanation: No. The sources are institutional written messages. A conclusion about informal conversation requires data selected from that domain, with ethical collection and proper consent.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Start with the claim you want to make is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Next

CHAPTER 02 OF 22 · A corpus sampling frame

2. Distinguish population, frame and sample

The target population is the set of language events or texts to which you want the conclusion to apply. The sampling frame is the operational set or rule by which eligible units can actually be reached. The sample is what you selected. These layers can differ even when your spreadsheet uses one column called ‘documents’.

Read the evidence, not just the label

Suppose the target is every announcement emailed to families at a school during Term 2. The school archive may contain ninety eligible messages, while a student downloads twenty. The ninety-message archive can function as a frame if it is complete enough for that purpose; the twenty downloaded messages are the sample.

The tempting mistake

Confusing the list you happen to possess with the entire target population hides access bias. Deleted notices, inaccessible attachments or messages stored in another system might be missing. The frame therefore needs a coverage statement, not simply a filename.

What to do instead

Write all three on the first page of your project: intended population, accessible frame and actual selected units. Record known exclusions, uncertain coverage and a plan for checking the archive against another source such as a log of issued notices.

Try this: If ninety texts are eligible and twenty are selected, which number is the frame size?

Explanation: Ninety, provided those ninety are the defined accessible eligible units. Twenty is the sample size. The total target population could still be larger if the archive is incomplete.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Distinguish population, frame and sample is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 03 OF 22 · A corpus sampling frame

3. Decide what counts as one unit

One text may contain several pages, one email may carry an attachment, and one webpage may be updated repeatedly. Sampling by document, by author, by page or by message can produce different distributions. The choice is not cosmetic: one very long document can contribute thousands of tokens while a one-line note contributes only a few.

Read the evidence, not just the label

Consider a newsletter with a cover letter, three pasted announcements and a timetable. A document-based frame could treat the PDF as one unit; a message-based frame might separate its announcements. For a study of announcement wording, separating meaningful message units might make more sense.

The tempting mistake

A seemingly fair random sample of twenty files may become unbalanced when one file is a fifty-page handbook and nineteen are small notices. Even if every file had the same selection probability, the token distribution would be strongly unequal.

What to do instead

Write a unit definition that another researcher could apply: ‘each independently dated announcement, excluding repeated copies of the same text’. Keep both document IDs and token counts because source selection and language weighting are two different issues.

Try this: Is one PDF always one language sample?

Explanation: No. The unit depends on the question and the structure of the data. A PDF can contain multiple communication events or revisions, and identical material may appear in several files.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Decide what counts as one unit is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 04 OF 22 · A corpus sampling frame

4. Set inclusion rules before seeing the favourite words

Inclusion rules answer whether a text is eligible without checking whether its vocabulary is interesting. Specify source, language, genre, period, audience and document status. If a text must mention a particular word to enter the sample, any later estimate of that word’s prevalence will be circular.

Read the evidence, not just the label

A school-notice corpus might include official notices distributed to parents from 1 January through 31 March, and exclude newsletters written by external advertisers. It should say how bilingual documents, corrected notices and duplicate links are handled.

The tempting mistake

A researcher who searches only for documents containing ‘assessment’ and then announces that assessment is common has selected on the outcome being measured. That can be useful for collecting examples of usage but not for estimating its rate among all notices.

What to do instead

Create a one-page eligibility checklist with yes/no/uncertain options. Apply it before calculating target word counts. Keep a log of decisions so that later corrections can be made without rewriting the study’s history.

Try this: Could a set of texts found by searching one target word estimate the general frequency of that word?

Explanation: Normally not across the full target domain: selection depends on the word’s presence. It may serve as a concordance sample, but the sampling design changes the allowed claim.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Set inclusion rules before seeing the favourite words is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 05 OF 22 · A corpus sampling frame

5. Choose a time boundary that means something

Vocabulary patterns can shift with examinations, holidays, school closures, policy announcements and changing digital tools. A corpus without a time window might mix years when a word had different meanings or communicative roles. A window is part of the frame, not an afterthought.

Read the evidence, not just the label

Suppose January notices focus on opening routines and September notices focus on examinations. Counting ‘registration’ in one period and ‘revision’ in the other does not by itself demonstrate a permanent language change. The texts serve different activities.

The tempting mistake

A frequent mistake is to report a ‘new trend’ when the new period contains a different kind of material. Topic and time can move together. Even a perfect date field cannot separate those explanations automatically.

What to do instead

Record the date of the event, the writing date and the publication date when they differ. Decide which date governs inclusion. If two periods are compared, use similar genre and source rules or explicitly explain the mismatch.

Try this: If one text was written in March but emailed in April, which term does it belong to?

Explanation: It depends on the declared temporal rule. For a study of messages received by families, send date may be more appropriate than drafting date; specify the choice consistently.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Choose a time boundary that means something is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 06 OF 22 · A corpus sampling frame

6. Name the genres that belong in scope

Genre is not decoration on a document: it is a recurring communicative purpose and pattern of language. A homework instruction, permission form, newsletter and promotional poster may all come from school, but their vocabularies serve different tasks.

Read the evidence, not just the label

Permission forms often contain ‘consent’ and ‘acknowledge’, while student-facing task sheets may repeatedly use ‘explain’ and ‘justify’. A pooled count can hide both patterns, particularly if one genre has more material.

The tempting mistake

A misleading conclusion declares a word ‘typical of school English’ when it is frequent only because the corpus contains dozens of permission forms. Within-genre inspection might show the word scarcely appears elsewhere.

What to do instead

List genres before sampling and provide a few observable indicators for each. Keep borderline texts in an ‘uncertain’ log for review, rather than forcing them into whichever category makes the result neat.

Try this: Would a single general category called ‘school documents’ be enough?

Explanation: Only for a question that really concerns that whole mixed category and can tolerate its internal variation. For comparisons or teaching decisions, the genre breakdown often matters.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Name the genres that belong in scope is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 07 OF 22 · A corpus sampling frame

7. Specify intended speakers, writers and readers

A message written by a teacher to parents is not the same population of language as a student’s reflective journal. Writers, recipients and institutional roles influence which words are useful or expected. The frame should identify whose language practices are being represented.

Read the evidence, not just the label

A science teacher’s feedback might use ‘hypothesis’, ‘variable’ and ‘justify’. A parent enquiry might use ‘arrange’, ‘clarify’ and ‘appointment’. Pooling both could obscure distinctions that are essential for teaching formal email writing.

The tempting mistake

The common mistake is to infer students’ active vocabulary from words that teachers wrote in worksheets. Exposure to a word is not evidence that each student understands, remembers or uses it independently.

What to do instead

Where feasible and ethical, record writer role and intended readership separately. Do not infer identities from a name or anonymised style. If role is unknown, mark it unknown rather than guessing.

Try this: Can a high frequency of ‘evaluate’ in a worksheet prove students know the verb?

Explanation: No. It demonstrates presence in instructional materials, not individual learner knowledge. A different task must elicit student understanding or production.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Specify intended speakers, writers and readers is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 08 OF 22 · A corpus sampling frame

8. Choose a language variety consciously

English varies across countries, institutions and communities. A frame designed for Singapore school notices may justifiably include local terms and spelling conventions, but should not claim to measure all world English usage. Multilingual switching raises further classification choices.

Read the evidence, not just the label

A document may contain English paragraphs and Malay or Chinese names. Removing all non-English strings can damage the context; treating them as ordinary English tokens can distort a strictly monolingual frequency list. Both choices have consequences.

The tempting mistake

One shortcut is to call any unfamiliar construction an error because it differs from a dictionary example from another region. A corpus records observed use; evaluation requires a stated audience, purpose and language norm.

What to do instead

Describe the variety, setting and code-switching policy. Preserve original text when lawful; create an analysis layer for token handling rather than silently rewriting the source. Name the audience for any teaching recommendation.

Try this: Does an unfamiliar expression become incorrect because it is not found in a small international corpus?

Explanation: No. Absence may reflect the corpus’s scope, sampling, period or search settings. Check authoritative usage evidence and communication context separately.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Choose a language variety consciously is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 09 OF 22 · A corpus sampling frame

9. Recognise access and coverage gaps

The most accessible language is not automatically the most typical. Public notices are easy to obtain; private peer conversations and spoken classroom exchanges are harder and may require consent. A sample biased toward available texts may overrepresent institutional language.

Read the evidence, not just the label

A class preparing a communication project can collect public school website messages but cannot freely record classmates’ conversations without appropriate agreement. The resulting corpus may still be useful as a corpus of public messages.

The tempting mistake

Calling missing private conversations ‘irrelevant’ solely because access is difficult turns an ethical and practical limitation into an unsupported claim about the population. The gap should remain visible.

What to do instead

Publish a coverage statement: which subgroups are represented, which are not, and why. Where privacy prevents collection, narrow the claim rather than trying to bypass the restriction.

Try this: Can we fix a lack of peer speech by doubling the number of school website pages?

Explanation: No. More of the same source genre increases text volume but does not cover the missing communication setting.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Recognise access and coverage gaps is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 10 OF 22 · A corpus sampling frame

10. Stratify by a meaningful category

Stratification divides a target domain into specified groups before selection. It can help ensure that small but relevant categories are not absent merely by chance. The strata must be relevant to the research question and should be recorded independently of the words later counted.

Read the evidence, not just the label

For a study of school communication, strata might be parent notices, student task instructions and public event announcements. If the question is about all three, selecting some texts from each helps reveal differences.

The tempting mistake

Creating dozens of overlapping strata after viewing the results risks an illusion of precision. A paper with three genres and two terms cannot reliably support hundreds of fine categories without sufficient units.

What to do instead

Use a small number of defensible strata, describe how units are assigned, and decide whether a text may belong to more than one stratum. If categories overlap, state whether counts are duplicated or assigned by priority.

Try this: If parent notices are rare but important, why might stratification help?

Explanation: It can secure examples from that genre, allowing a separate description. But the resulting mixture may need weights if you estimate the overall population distribution.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Stratify by a meaningful category is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 11 OF 22 · A corpus sampling frame

11. Separate proportional sampling from equal quotas

A proportional design samples strata roughly in line with their presence in a known target population. An equal-quota design assigns similar numbers of units to each stratum. Either can be appropriate, but they answer different questions without weighting.

Read the evidence, not just the label

Imagine ninety eligible notices: sixty routine notices, twenty event invitations and ten examination notices. A sample of thirty using proportional allocation might select twenty, about seven and about three. An equal allocation would choose ten from each, intentionally amplifying the smallest category.

The tempting mistake

A student who uses ten texts of each type and reports the pooled word rate as the average for all ninety eligible notices silently substitutes an equal genre mixture for the original population composition.

What to do instead

State the allocation. If you oversample a group to compare vocabulary, report group-specific results and use justified weights for population-wide estimates. Do not invent weights when the population shares are unknown.

Try this: Does a balanced-looking table of ten texts per genre represent the archive’s natural proportions?

Explanation: Not necessarily. It is equal by design and useful for comparing genres, but not the same thing as proportional representation.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Separate proportional sampling from equal quotas is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 12 OF 22 · A corpus sampling frame

12. Understand the limits of random selection

Random selection can protect against some forms of subjective cherry-picking when the frame is complete and the sampling procedure is genuine. It cannot repair missing groups, inconsistent units or incorrect labels. A random seed and a clear rule help reproduce the draw.

Read the evidence, not just the label

If ninety notices have stable IDs, one could number them and randomly select twenty without examining whether they contain the target word. That reduces the chance of consciously favouring documents with interesting language.

The tempting mistake

A common error calls ‘the first twenty files returned by search’ random. Search engines often sort by relevance, date or prominence. Alphabetical filename order is also not a random draw.

What to do instead

Record the method and, when relevant, the random seed. If purposive selection is better for rare genres or qualitative interpretation, call it purposive and avoid generalising a statistical rate beyond the chosen cases.

Try this: Is a sample random because the teacher selected examples without a particular plan?

Explanation: No. Haphazard is not the same as probabilistic sampling. A random design specifies a procedure giving eligible units known or describable chances of selection.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Understand the limits of random selection is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 13 OF 22 · A corpus sampling frame

13. Make text length part of the design

Equal numbers of documents do not imply equal amounts of language. Longer items can dominate token-based frequencies. The right approach depends on whether the inference concerns documents, writers, passages or words in the total collection.

Read the evidence, not just the label

Suppose five letters contribute 1,000 tokens altogether and one handbook contributes 12,000. The handbook may determine most of the pooled vocabulary list even though it is a single document. This is not inherently wrong if the question concerns all tokens, but it is wrong to describe it as an average document.

The tempting mistake

A misleading report says ‘six documents show the word often’ when almost every instance comes from the single handbook. Without per-document counts, readers cannot see the concentration.

What to do instead

Record token totals per text. Compare document prevalence and token frequency separately when useful. Consider bounded excerpts or appropriate weighting if the question calls for equal contribution by document rather than by token.

Try this: A word appears forty times, thirty-eight inside one handbook. What should be reported?

Explanation: Both the total count and its concentration. The claim should not imply the word is widespread among the six sources.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Make text length part of the design is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 14 OF 22 · A corpus sampling frame

14. Handle repeated and syndicated text

A notice may appear on a website, in an email, as a PDF and in an updated archive. Repeating the same text three times inflates its vocabulary contribution unless the study intentionally tracks exposure across channels.

Read the evidence, not just the label

A sports-day reminder appears verbatim on a school homepage and a PDF download. If the target is distinct messages, count it once and record both appearances; if the target is pages encountered by readers, counting both may serve another declared question.

The tempting mistake

A mistake is to deduplicate all near-identical messages automatically, eliminating meaningful revisions or repeated exposure. Another is to count accidental copies as independent evidence that a phrase is common across different writers.

What to do instead

Define exact duplicates, near-duplicates and deliberate reissues. Keep a provenance link between copies and choose rules consistent with the research question. Review the impact on high-frequency words.

Try this: Should a reminder printed twice always count once?

Explanation: No universal answer. A unique-text study may deduplicate, while an exposure study may count each separate communication event. The policy must be explicit.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Handle repeated and syndicated text is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 15 OF 22 · A corpus sampling frame

15. Link the frame to metadata

A source ID without context is difficult to audit. Minimum metadata might include source, date, genre, audience, language, unit length, selection pathway and any processing applied. Sensitive personal details are rarely needed for a vocabulary count.

Read the evidence, not just the label

For text N14, a row could state ‘student instruction; Year 2026; 143 tokens; website; selected under rule R2; punctuation normalised; one duplicated attachment removed’. That is enough to reconstruct many choices without publishing a person’s private identity.

The tempting mistake

An apparently scientific chart becomes fragile when titles are omitted and no one can tell whether the same document entered twice or whether the denominator includes table headings and boilerplate.

What to do instead

Keep one metadata row per sampling unit and stable anonymised identifiers. Separate original-source fields from derived analysis fields so a later correction does not silently alter the record.

Try this: What does a token count tell us without the sampling unit and source?

Explanation: It tells us about a particular processed text count, but not whether the text belongs to the intended domain or whether two records represent independent examples.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Link the frame to metadata is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 16 OF 22 · A corpus sampling frame

16. Document exclusions and uncertain cases

Some documents will be ambiguous: an attachment straddles two periods, a screenshot lacks its publication date, or a bilingual message is not easily segmented. A mature frame has a process for uncertainty rather than pretending every eligibility decision is obvious.

Read the evidence, not just the label

An undated screenshot of a parent notice may be useful for teaching how a phrase is written, yet unsuitable for a strict comparison of Term 1 and Term 2. These are two different uses of evidence.

The tempting mistake

Deleting ambiguous records without leaving a trace creates invisible selection. Including them anyway can contaminate a time-bound estimate. Both decisions can bias results if made after inspecting the vocabulary outcome.

What to do instead

Keep an exclusion log with ID, reason, decision-maker and review status. If the disputed material could change the answer, run an explicitly labelled sensitivity analysis with and without it.

Try this: A notice’s date is unknown. Should it be assigned to the most likely month?

Explanation: Not as a definite fact. Seek verification; otherwise keep its date unknown and exclude it from analyses that require a confirmed month.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Document exclusions and uncertain cases is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 17 OF 22 · A corpus sampling frame

17. Check privacy, consent and permission

Corpus construction often happens in schools, where texts can reveal minors’ names, family circumstances and confidential conversations. Eligibility for linguistic analysis does not itself authorise publication. Collection, storage and access rights are separate decisions.

Read the evidence, not just the label

A teacher may analyse anonymised classroom instructions already authorised for study but should not upload named student essays into an open public corpus merely because they are informative. Privacy protections can coexist with rigorous language research.

The tempting mistake

A frequent mistake treats a document’s appearance in a shared folder as proof that it may be redistributed, indexed or used to train a public model. Access and reuse have different conditions.

What to do instead

Apply the relevant school and institutional policies, minimise personal data, obtain appropriate permissions, restrict sensitive records and publish synthetic examples when real texts cannot be shared safely. Explain what a reader can audit without disclosing protected material.

Try this: Does a text being useful to a language model mean it is free for public reuse?

Explanation: No. Educational usefulness does not determine privacy rights, copyright permission or legitimate data access.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Check privacy, consent and permission is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 18 OF 22 · A corpus sampling frame

18. Build a transparent mini-frame

A miniature example is valuable because every unit and exclusion can be inspected. Imagine a fictional twenty-four-item archive: twelve parent notices, eight student instructions and four event posters. The research question concerns formal school notices to families, so most student instructions are ineligible.

Read the evidence, not just the label

The original exercise below uses IDs F01–F08 to represent examples rather than real records. Count eligibility first. Then choose a sampling procedure among the eligible units. Only after those decisions should you run a word-frequency search.

The tempting mistake

Do not mistake the miniature demonstration for a report on what Singapore schools actually send. Its purpose is to make the logic visible and let a learner identify which features change when the research question changes.

What to do instead

Write the eligibility rule at the top of the table; mark each example yes/no/uncertain and explain the choice. After deciding, compare whether another student reaches the same decisions from the rule alone.

Try this: Why is it useful to include an ineligible record in a teaching table?

Explanation: Because it tests the boundary. A rule that only labels obvious eligible examples may appear clear while failing on ordinary edge cases.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Build a transparent mini-frame is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 19 OF 22 · A corpus sampling frame

19. Evaluate coverage before claiming representativeness

A frame can be well documented but still exclude major parts of the target domain. Representativeness depends on the research purpose, coverage of meaningful variation and the way texts are distributed. It is not a certificate awarded to any dataset larger than a threshold.

Read the evidence, not just the label

A corpus with many official circulars, no informal speech and no student-authored text may be excellent for studying formal notices. It is not a general measure of children’s everyday vocabulary. The difference is the target question.

The tempting mistake

An attractive pie chart can disguise missing genres because only observed categories enter the denominator. Absence from the sample is not evidence of absence from the world.

What to do instead

Write an audit paragraph that names the supported target, known omissions, the likely effect of those omissions and what kind of new source would test the conclusion. Prefer a modest accurate statement to a universal one.

Try this: What is stronger: ‘These notices frequently use enrolment’ or ‘Every student uses enrolment frequently’?

Explanation: The first, if supported by the observed notice counts. The second requires student-language evidence and a much broader population claim.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Evaluate coverage before claiming representativeness is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 20 OF 22 · A corpus sampling frame

20. Revise a design without rewriting the past

A design may improve as missing sources or poor category rules become apparent. Changes are legitimate when recorded. The danger is silently altering inclusion after seeing which version yields the most impressive percentage.

Read the evidence, not just the label

Suppose an audit finds that twelve circulars were all copies of one original. A correction may remove duplicates from the unique-text corpus and change every rate. A reproducible report should preserve the original version and explain the revised count.

The tempting mistake

Pretending the earlier result never existed prevents readers from learning what went wrong. Conversely, refusing any change because the first design was published confuses consistency with accuracy.

What to do instead

Version the frame and corpus; record added or removed sources, modified rules, reasons and how outputs change. A table comparing version 1.0 with 1.1 can be more educational than hiding the correction.

Try this: Does discovering a design error mean every conclusion must be abandoned?

Explanation: Not necessarily. Recalculate affected claims, narrow scope if required and explain which results remain robust.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Revise a design without rewriting the past is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 21 OF 22 · A corpus sampling frame

21. Teach the reasoning before the terminology

School vocabulary learning benefits when students move from a concrete communication problem to an explicit evidence rule. ‘What does this sample allow me to say?’ matters more than memorising ‘sampling frame’ in isolation.

Read the evidence, not just the label

Give a class six fictional notices and two fictional text messages. Ask them to answer two questions: ‘How do notices word requests?’ and ‘How do teenagers ask friends for help?’ The same eight texts do not answer both questions equally well.

The tempting mistake

A definition-only exercise might produce students who can recite that sampling is selecting units yet continue to make broad claims from convenient data. That is vocabulary recognition without transfer.

What to do instead

Ask learners to identify population, frame, sample, unit and exclusion in a short paragraph. Then switch the question and require them to revise only the frame. The revision demonstrates that they understand the term’s role.

Try this: What is evidence of understanding rather than rote recall?

Explanation: The student can justify a source boundary, identify a missing group and rewrite an overclaim as a narrower supported statement.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Teach the reasoning before the terminology is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Next

CHAPTER 22 OF 22 · A corpus sampling frame

22. Use a final claim audit

Before publishing a vocabulary finding, inspect five questions: which population, which reachable frame, which selected texts, which preprocessing rules, and which meaning of the resulting count? A claim should remain truthful when the reader sees all five answers.

Read the evidence, not just the label

A well-formed conclusion might say: ‘In our constructed sample of eligible parent notices, the phrase please submit appeared in five of eight selected texts; the sample does not describe student-to-student speech.’ It is modest but independently checkable.

The tempting mistake

A weak conclusion replaces the exact sample with ‘people usually say’ or implies national prevalence from one institution. That extra phrase turns observed description into unearned generalisation.

What to do instead

Keep the five-question audit beside every chart and add an invitation to check the source log. If a conclusion cannot be revised into a defensible sentence, do not use the numerical result as a teaching fact.

Try this: What is the first thing to fix when a chart is tidy but its claim is too broad?

Explanation: Narrow the claim to the sampled domain or improve the sampling frame. Another decimal place is not a solution.

Reader check: Can you name the unit being counted, the evidence needed and the boundary of the conclusion? If any of those are missing, revise the claim before applying it to a new word, school subject or reading passage. Use a final claim audit is useful because it changes a decision, not because it adds another impressive-sounding term.

Chapter contents · Previous · Worked evidence

Worked evidence: a complete miniature dataset

Here is a deliberately small, fictional source register for a class project whose target is English-language messages issued by school administration to families during one declared term. The frame includes the administration archive; student peer messages, outside advertising and undated material are not automatically eligible. The rows illustrate how an eligibility rule differs from an interesting vocabulary example.

Record	Source description	Declared audience	Eligibility	Reason
F01	Term-opening administrative letter	Families	Yes	Official administration, within term
F02	Student’s personal reflection	Teacher	No	Not an administration message to families
F03	Attendance reminder email	Families	Yes	Matches source, audience and period
F04	External shop advertisement	Customers	No	External commercial writer
F05	School event invitation	Families	Yes	Official family communication
F06	Undated screenshot	Unknown	Uncertain	Date and origin need verification
F07	PDF copy of F03	Families	Duplicate	Same underlying notice; separate appearance
F08	Private peer chat	Friend	No	Outside the declared domain

Under the chosen unique-text rule, F01, F03 and F05 are confirmed eligible unique texts. F06 remains unresolved until its date and origin can be verified; F07 is a duplicate of F03 and should not be treated as an independent unique message. F02, F04 and F08 lie outside the defined target. This is not a statement about which items are linguistically interesting: all eight could support different questions.

The eligible count is three confirmed distinct items in this miniature display, not eight. That does not imply the real frame has three units; the table is a constructed teaching subset. A learner could draw a sample from a larger archive only after applying the same eligibility rule to every candidate.

Suppose the question changes to ‘What written instructions do students receive?’ F02 might still be ineligible because it was written by a student, while classroom instructions previously excluded could become important. The rule changes because the research question changes. Do not move individual records merely because they contain the words you hope to find.

A complete report would include the population definition, a record of how many units were screened, the number excluded by each reason, how duplicates were handled, and the token totals for eligible material. The omission of one field may make a later frequency comparison impossible to audit.

Back to the contents

Twelve decisions: transfer the idea into real reading and writing

These short situations are not a substitute for a complete dataset. They demonstrate how to identify the question, notice a limitation, and write a narrower conclusion. Treat each situation as a separate exercise.

1. Exam-week vocabulary

Situation. A sample contains ten examination notices and one general newsletter. A word for revision appears frequently.

Risk. The exam-heavy mixture may reflect the chosen event period more than general school communication.

Decision and reason. Describe this as exam-week administrative language, or sample multiple periods for a broader claim.

Transfer question. Which missing period could test the conclusion?

2. Two school websites

Situation. One school exposes all circulars publicly; another has a private archive.

Risk. Availability may be mistaken for a school-wide linguistic difference.

Decision and reason. Treat the accessible webpages as two uneven frames; do not compare schools as if the sources have equivalent coverage.

Transfer question. Which coverage log would make the comparison fairer?

3. One long handbook

Situation. A handbook contributes eighty percent of corpus tokens.

Risk. A pooled rate could be driven by a single author or document.

Decision and reason. Report token-weighted and document-level results separately.

Transfer question. What would change if each document had equal influence?

4. Repeated reminders

Situation. The same message is emailed three times.

Risk. A unique-text estimate and an exposure estimate will diverge.

Decision and reason. Choose the question first, then count unique texts or delivery events accordingly.

Transfer question. Which question justifies including repetitions?

5. Bilingual circular

Situation. A school circular mixes English and another language.

Risk. Deleting portions silently changes the denominator and context.

Decision and reason. Record the language policy; preserve source text securely and make analysis transformation explicit.

Transfer question. Would the conclusion concern English passages or total message content?

6. Undated screenshot

Situation. A screenshot carries an unfamiliar administrative word but no timestamp.

Risk. Assigning an assumed date can fabricate a time trend.

Decision and reason. Use for qualitative discussion only or verify metadata before chronological analysis.

Transfer question. What extra information is needed to include it?

7. Student essay collection

Situation. Only the highest-scoring essays were supplied.

Risk. The frame excludes weaker and average writing.

Decision and reason. Describe vocabulary in selected high-scoring essays, not all students.

Transfer question. Which sampling change would support a whole-class question?

8. Term 1 versus Term 4

Situation. Different subjects and activities dominate the two terms.

Risk. Genre composition and topic are confounded with time.

Decision and reason. Compare within common genres and report the changed composition.

Transfer question. What comparison would isolate time better?

9. Search-result corpus

Situation. Files were downloaded only when the target term appeared in search.

Risk. The word’s apparent prevalence is guaranteed by selection.

Decision and reason. Use the files for usage examples, not an unconditional rate estimate.

Transfer question. What broader frame would permit a frequency estimate?

10. School chat archive

Situation. Messages are confidential and contain identifiable names.

Risk. Linguistic interest does not create authority to publish private text.

Decision and reason. Seek permission, minimise information, or use original synthetic teaching cases.

Transfer question. What claim can be made without exposing private records?

11. Genre quotas

Situation. Ten instruction sheets and ten public notices were selected despite an uneven archive.

Risk. Equal quotas may distort pooled population shares.

Decision and reason. Use strata-specific findings or documented weights for relevant overall estimates.

Transfer question. When are equal quotas actually advantageous?

12. Frame revision

Situation. An archive later reveals five missed eligible documents.

Risk. A previously published frequency uses an incomplete denominator.

Decision and reason. Version the sample, report the correction and assess whether the claim changes.

Transfer question. How would you explain the revision to a reader?

Practice: eight questions with explained answers

Try each question without looking at the answer. Then explain your choice aloud in one or two complete sentences. A correct word without a defensible reason is not yet the skill this guide is teaching.

Exercise 1: Population or sample?

Question. A class wants to know how parent circulars word reminders. It selects twelve from a list of seventy eligible notices. Name the three levels.

Worked answer. Target population: the specified parent circulars in scope; operational frame: the seventy eligible archived notices, assuming adequate coverage; sample: the twelve selected notices.

Why the shortcut fails. Calling the twelve ‘all school language’ erases the frame and broadens the target without evidence.

Exercise 2: Eligible source

Question. A personally written diary shares the target term with official notices. Include it?

Worked answer. No, not for the declared administration-to-family notice population. Its language can be studied separately or under a different frame.

Why the shortcut fails. Word presence does not override an eligibility criterion.

Exercise 3: Duplicate notice

Question. A PDF repeats an email exactly. Does that create a second independent text?

Worked answer. Not under a unique-text design. Keep a record of the second distribution channel and count both only for a declared exposure question.

Why the shortcut fails. Silently pooling copies exaggerates the number of independent sources.

Exercise 4: Long document

Question. A handbook supplies 7,000 of 10,000 tokens. What is the risk?

Worked answer. The pooled token count is heavily driven by one unit. Report concentration and inspect document-level prevalence.

Why the shortcut fails. Equal file counts do not mean equal word exposure.

Exercise 5: Equal quotas

Question. A frame contains 80 notices of genre A and 20 of genre B. A student selects ten from each. What changed?

Worked answer. The sample is 50/50 even though the frame is 80/20. Equal quotas may aid genre comparison but cannot be assumed to estimate the natural pooled distribution.

Why the shortcut fails. The superficially balanced selection does not mirror the frame’s proportions.

Exercise 6: Missing date

Question. A message is relevant but its publication period is uncertain. What is honest?

Worked answer. Mark uncertain, seek metadata and avoid assigning it as definite chronological evidence.

Why the shortcut fails. Guessing a date can create a false trend.

Exercise 7: Overclaim audit

Question. A report says ‘Teenagers prefer formal requests’ based on thirty circulars. Repair it.

Worked answer. For example: ‘The thirty sampled formal school circulars often used direct request language’; avoid generalising to teenagers’ preferences.

Why the shortcut fails. The sampled writers and readers differ from the claimed population.

Exercise 8: Privacy

Question. An unpublished student chat would make the sample more diverse. Must it be collected?

Worked answer. No. Ethical access and permission govern collection. Narrow the claim or design a permitted collection, including synthetic examples for teaching.

Why the shortcut fails. A desirable sample composition does not excuse collecting confidential data.

Frequently asked questions about a corpus sampling frame

Is a sampling frame the same as the corpus?

Not necessarily. The frame specifies eligible reachable units or the mechanism used to identify them; the corpus is the selected and processed collection. A census of all eligible units can make the two coincide, but keep the distinction visible.

Must a frame be a complete list?

An explicit complete list is ideal for some designs, but operational source rules and access mechanisms also exist. Describe what was actually reachable and assess omissions rather than claiming completeness without evidence.

Is a bigger corpus always more representative?

No. Scale may help with precision and rare occurrences but does not automatically cover absent varieties, people or genres. The research purpose defines what adequate coverage means.

What is stratification?

Dividing the target domain into meaningful groups before selection, such as genres, periods or writer types. It helps examine coverage but requires clear allocation and accounting.

Is equal representation of genres always fair?

Equal quota sampling can help comparison but may not reflect the population’s proportions. Fairness is relative to the inference, not to how symmetrical the chart looks.

How should I treat repeated texts?

Set a unique-text or exposure-count policy that fits the question. Track copies even if you exclude them from the analytic count.

Can I use publicly available online text?

Public visibility is not itself a blanket licence for reuse, redistribution or publication of private details. Follow lawful access, permissions and institutional guidance.

Why does metadata matter?

It makes eligibility, source diversity, timing and transformations inspectable; without it, the observed words may lose their interpretive context.

What if two reviewers disagree on a genre?

Document the case, clarify the rule, review the evidence and retain the decision history. Uncertainty is preferable to false unanimity.

Can a sample answer a different question from the one it was built for?

Sometimes, but only after auditing how the new target overlaps with the sampled sources. Do not assume one corpus is universal.

What can a Primary or Secondary student do without software?

Build a six-to-twelve-text fictional example, create an eligibility checklist, mark cases and justify a narrow claim. The reasoning matters more than database size.

What should I write below a chart?

Name the domain, the units and period, source coverage, analysis denominator and known missing groups. That compact caption can prevent an exaggerated headline.

Glossary for careful vocabulary evidence

• Target domain: The language setting or kind of communication a study intends to describe.
• Population: The full set of eligible events, texts or other units about which an inference is intended.
• Sampling frame: The accessible source listing or operational eligibility mechanism from which a sample may be selected.
• Sample: The set of units actually selected for analysis.
• Unit: What is treated as one selectable item: for example a message or document.
• Stratum: A predeclared group within a domain used for sampling and analysis.
• Coverage bias: A distortion caused by relevant sources missing or being overrepresented in the reachable frame.
• Duplication: Repeated text that may not constitute independent language evidence.
• Provenance: A record of where an example came from and how it was handled.
• Representativeness: The extent to which a corpus supports meaningful generalisations about a specified domain.

How to use this guide with students or colleagues

A productive first reading is selective. Choose a question that really occurs in your own work, read the chapters needed to answer it, and write down the smallest useful decision. For a school project, this might be a sampling rule or a denominator. For a writing task, it might be a phrase whose apparent popularity changes across genres. For a language tool, it might be a validation check that prevents a misleading output.

On a second reading, reconstruct the worked table with different original examples while keeping its method constant. Ask one learner to propose the claim and another to inspect the evidence. The inspector must explain what is counted, what is excluded, which context is missing, and what would make the claim stronger. Both should be able to amend a sentence without turning the lesson into an argument about who sounds more confident.

On a third reading, revisit a failure case after changing just one assumption. If a count changes when the tokenisation policy changes, explain why. If a school-text sample excludes spoken conversation, say so. If an association score was computed using a tiny frequency, show the raw count before presenting a ranking. These habits make a vocabulary lesson more transferable than memorising a list of technical labels.

Further reading and methodological sources

• Cambridge: Egbert, Biber and Gray on domain considerations and sampling
• Cambridge: approaches to representativeness
• Oxford: Biber on representativeness in corpus design

Research sources are offered for methodological depth; classroom cases and numerical examples in this article remain constructed illustrations unless explicitly attributed.

Continue through the vocabulary evidence library

• Corpus metadata: learn which source details explain a count
• Corpus comparison: test whether two text collections support the same claim
• Gold-standard corpus: understand a checked reference for annotations
• Hapax legomenon: handle a once-seen word without overstating it
• Vocabulary learning hub
• Vocabulary articles index

Return to the chapter index · Vocabulary Learning Hub