Vocabulary | What Is a Subcorpus? How Smaller Text Collections Reveal Differences in Word Use

CHAPTER 01 OF 22

1. Why a corpus is rarely one kind of English

A corpus can contain articles, messages, transcripts, instructions and stories. These texts have different purposes and therefore favour different words and constructions. The overall count answers a question about the combined collection, but it cannot automatically describe the habits of every component.

A concrete example

Imagine twenty school stories, thirty science instructions and forty administrative notices. Even if these were all published in one year, the sources would invite different kinds of language. Stories might narrate actions; instructions might specify procedures; notices might request a response.

Where an attractive answer goes wrong

A strong-looking generalisation says that a term is ‘typically used by students’ because it occurs often in a mixed school archive. The statement ignores who wrote the texts and why. Teacher instructions are not evidence of student production.

A better decision

Name each major text type and its producer before calculating a frequency list. Ask whether the current question concerns the full archive or only part of it. A subcorpus is a useful response to genuine variation, not a way to hide inconvenient examples.

Check your understanding: Do science worksheets and parent circulars automatically reflect the same communication habits?

Explained answer: No. Their audiences, purposes and writers differ. They can share a corpus but merit separate descriptions for many vocabulary questions.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Next chapter

CHAPTER 02 OF 22

2. Define a subcorpus in one operational sentence

A subcorpus is not merely a topic that someone has in mind. It needs a selection rule that another analyst can run against the available texts. ‘All teacher-written science worksheets dated from January to June’ is checkable; ‘the interesting science bits’ is not.

A concrete example

One project may store a science subcorpus, a 2026 subcorpus and a written-English subcorpus. The same science worksheet dated in 2026 may belong to all three. In a capable corpus tool, these named selections can be saved and reused.

Where an attractive answer goes wrong

A common mistake creates a handpicked list after viewing the target word’s counts, then calls it the natural science subcorpus. Selection based on the outcome can create a misleading apparent advantage.

A better decision

Record the exact inclusion conditions, source fields and chosen date definitions. Preserve a machine-readable membership list or a clear manual register so the subset can be reconstructed later.

Check your understanding: What must be saved besides the name ‘science subset’?

Explained answer: The rule, the relevant source IDs or metadata fields, and enough information to rebuild which texts qualified.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 03 OF 22

3. Understand corpus, subcorpus and sample

The parent corpus is the whole collection being analysed. A subcorpus is a defined subset of that parent. A sample is a set of selected items; it may be drawn from a subcorpus, or the subcorpus itself may be the chosen analytic sample. The terms answer different design questions.

A concrete example

Suppose a parent corpus contains one hundred school texts. Forty are science-related. A random draw of ten of those forty is a sample from the science subcorpus. The parent corpus has one hundred units, the subcorpus forty and the selected sample ten.

Where an attractive answer goes wrong

The error calls every subset ‘representative’ because the filter uses a precise label. A filter can be precise and still miss important types of science communication or favour one school’s materials.

A better decision

State the parent, selected subset and any further sampling step in three separate lines. This is especially important when comparing results obtained from different interfaces or downloaded files.

Check your understanding: Can a subcorpus be an analysis set without being a random sample?

Explained answer: Yes. It may be selected by metadata or purpose. Statistical generalisation depends on the underlying collection design, not simply the presence of a named filter.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 04 OF 22

4. Select by genre and communicative purpose

Genre captures recurring tasks such as explaining, narrating, requesting, reporting and persuading. A subcorpus formed around a genre can reveal phrase patterns that look weak in a pooled word list. But genres need consistent operational boundaries.

A concrete example

A teacher comparing explanation writing with stories might create one subset for subject explanations and another for narratives. ‘As a result’ may appear more in the former, while ‘suddenly’ may feature in the latter; the teaching question concerns their functions.

Where an attractive answer goes wrong

A misleading shortcut assigns each text to a genre based solely on the presence of the target word. If ‘therefore’ triggers the label explanation, a later count of therefore in explanations is partly circular.

A better decision

Define genre from independent evidence such as text purpose, intended audience and document type. Keep ambiguous records marked for review instead of forcing them into convenient categories.

Check your understanding: Why is selecting all documents containing ‘therefore’ a poor test of explanation frequency?

Explained answer: Because the selection already depends on the word being measured. A genre definition should be established independently of the target occurrence.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 05 OF 22

5. Use time-based subsets without inventing trends

A year, term or month can define a subcorpus if dates are reliable and the analytic question warrants time comparison. Word-use changes can reflect changing topics or genres rather than a permanent language shift.

A concrete example

A school might send many examination notices near an assessment period and many excursion notices near an activity week. The frequency of revise or permission can therefore rise and fall in different monthly subsets.

Where an attractive answer goes wrong

The mistake labels a spike in a term as a generational change in language. The corpus may contain a different mixture of purposes, and only a limited number of texts may be available in each period.

A better decision

Keep genre and source descriptions alongside period labels. Check whether the subcorpora cover comparable kinds of texts and whether the time field represents writing, issue or event date.

Check your understanding: Can a word becoming common in one term prove it is a new permanent English trend?

Explained answer: No. It is initially a local, time-bound observation that requires broader comparable evidence to support any trend claim.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 06 OF 22

6. Separate spoken and written evidence

Medium matters because speech and writing operate with different constraints. Spoken dialogue can include hesitations, repairs and interaction cues; edited articles can use longer planned structures. A mixed corpus may be valuable, but the analyst should not mistake its average for each medium.

A concrete example

A speaking-practice corpus might contain transcribed discussions and school presentations. A written subcorpus might contain teacher notices and public articles. The word well can serve different functions across these contexts.

Where an attractive answer goes wrong

An attractive table claims spoken language is simpler because its frequency of a chosen technical term is lower. That does not establish complexity or competence, and transcription methods may also change token counts.

A better decision

Build spoken and written subsets from reliable medium metadata. Record transcription conventions, privacy permissions and whether conversational fillers or overlapping speakers are included in the eligible token total.

Check your understanding: Is a transcribed conversation the same as a written article because both appear as text on screen?

Explained answer: No. Its production conditions and transcription rules differ. The screen format should not erase the original medium.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 07 OF 22

7. Use speaker and writer roles carefully

The words in a document do not necessarily belong to the person who read it. Teacher instructions, student answers and administrative messages have distinct authorship and audiences. A subcorpus can isolate each role when those fields are available and lawful to use.

A concrete example

An archive containing assignment prompts and student compositions may contain the word analyse in the prompts more frequently than in the submissions. That does not prove students can independently use the word.

Where an attractive answer goes wrong

A weak conclusion infers a learner’s vocabulary breadth from the teacher’s worksheet. Exposure is not equivalent to recognition, comprehension or fluent independent production.

A better decision

Keep writer and audience separate. If identity metadata are sensitive, use appropriate aggregate roles or anonymised codes. When role is unknown, retain uncertainty instead of guessing from style.

Check your understanding: Can a list of frequent words in homework instructions measure what pupils can write unaided?

Explained answer: No. It measures the instruction texts, unless the study includes separate evidence of student production and understanding.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 08 OF 22

8. Allow overlap without double counting totals

Subcorpora can overlap. A text can be part of the science group and the 2026 group at the same time. That is perfectly legitimate for separate analyses, but summing overlapping totals as though they partition the corpus can count the same tokens more than once.

A concrete example

Suppose the science subset contains 12,000 tokens and the 2026 subset 18,000, of which 5,000 belong to both. Their union has 25,000 unique tokens, not 30,000. The overlapping five thousand are already present in both reported subsets.

Where an attractive answer goes wrong

The error places each subset’s total in one stacked chart labelled ‘whole corpus’ despite overlap. The chart suggests a larger dataset than actually exists and can inflate word counts if repeated tokens are summed.

A better decision

Choose whether the analysis requires disjoint categories, a union, an intersection or separate overlapping views. State the construction and count unique source IDs or tokens when calculating a union.

Check your understanding: What is the union size of 12,000 and 18,000 tokens with 5,000 overlapping?

Explained answer: 12,000 + 18,000 − 5,000 = 25,000 unique tokens, assuming the overlap was defined at the same token-unit level.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 09 OF 22

9. Understand subset size and denominators

A smaller subcorpus offers fewer opportunities for word occurrences. Comparing 80 hits in one group with 20 in another requires knowledge of their token totals. Normalised rates help make unequal corpus sizes comparable on a common scale.

A concrete example

A constructed science subset of 12,000 tokens with 60 matches of charge has 5,000 occurrences per million tokens. A notices subset of 8,000 tokens with eight matches has 1,000 per million. These figures describe the exact selected materials.

Where an attractive answer goes wrong

The wrong conclusion says that science uses charge six or seven times as often as everyone because the raw counts differ by more than seven to one. The valid rate comparison here is five to one, limited to the toy subsets.

A better decision

Always show each subset’s eligible tokens, raw hits, search form and normalised rate. If a subset is very small, add a caution that one or two matches can change the scaled rate substantially.

Check your understanding: Which is necessary for a rate comparison: total documents, token totals or both?

Explained answer: Token totals are essential for per-token rates; document totals help interpret source spread and uneven lengths. A complete report often includes both.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 10 OF 22

10. Keep the same search rule across subsets

Comparing groups requires a shared definition of the search target. An exact spelling count in one subset and a lemma count in another are not equivalent. Some words are homographs, and their senses can differ by genre.

A concrete example

In a study of charge, science worksheets might use the noun for electrical property while notices use the verb meaning to ask payment. The exact written form can be counted in both, but the semantic interpretations are not interchangeable.

Where an attractive answer goes wrong

The tempting shortcut compares the combined written-form counts then announces that one meaning is dominant everywhere. Form counts do not determine the distribution of senses.

A better decision

Preserve the identical query syntax and processing settings across subsets. If meaning is central, annotate context or inspect concordance lines by subset and report sense-specific results separately.

Check your understanding: If charge appears five times per thousand in science and one per thousand in notices, is the meaning necessarily the same?

Explained answer: No. The rate comparison concerns the declared form unless a checked sense analysis shows that the uses are comparable.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 11 OF 22

11. Recognise metadata errors and missing labels

A text-type filter can only be as accurate as the metadata used to construct it. Missing genre fields or unreliable dates may exclude legitimate units, and a mislabelled text can contaminate a comparison.

A concrete example

A science experiment instruction saved under ‘general notices’ could be missing from the science subset. If many similarly misfiled texts exist, the word profile may be distorted while the interface still displays a tidy filter.

Where an attractive answer goes wrong

A careless analyst treats the subset name as validation of every included text. Software executes the stored criteria; it does not guarantee the underlying metadata were assigned correctly.

A better decision

Inspect a sample of member documents, audit missing fields and record uncertain assignments. If possible, correct the metadata at its source or create a reviewed analysis layer with transparent versioning.

Check your understanding: Does a perfectly repeatable filter guarantee a scientifically sound subset?

Explained answer: No. Repeatability and metadata accuracy are separate qualities. A precise filter can faithfully reproduce a poor source classification.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 12 OF 22

12. Use topic-based subsets cautiously

Topic can be useful when clearly defined but may be particularly prone to circular selection. A researcher interested in words for climate events should distinguish documents independently classified as climate reports from documents chosen only because they contain climate terms.

A concrete example

An index of school projects may mark a text’s curriculum subject before the vocabulary analysis. Those subject labels can form topic subsets. If no labels exist, a topic model or hand coding may help, but its classification decisions need validation.

Where an attractive answer goes wrong

The mistake creates a ‘flood’ topic by searching for the word flood, then declares flood a keyword of that topic. The method makes the key conclusion largely predetermined.

A better decision

Explain how topic membership was assigned and whether the word being studied contributed to that assignment. Use a holdout or independent classification if the same vocabulary feature is a central outcome.

Check your understanding: When does topic filtering create circular evidence?

Explained answer: When the target word determines which texts enter the topic subset and is then used to show the target word is characteristic of that subset.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 13 OF 22

13. Learn to compare two subcorpora

A useful comparison table shows raw and normalised frequencies in each subset, followed by context examples and a clear description of their differences. A keyword or word-sketch comparison can highlight possible patterns, but no tool removes the need to inspect the source domain.

A concrete example

A fictional school corpus could be divided into science teaching texts and family notices. The higher rate of charge in science suggests the value of investigating context. Concordance lines might reveal electrical, payment and general uses.

Where an attractive answer goes wrong

A flawed comparison calls the largest numerical difference ‘significant’ without an inferential method or ignores that one group includes a single very long textbook while the other has many short notices.

A better decision

Start by checking source comparability, collection rules, size and query. Then examine counts and sentence context. Use inferential or effect-size measures only with suitable design and explicitly stated assumptions.

Check your understanding: Is a higher normalised frequency proof that the word is more important to learners?

Explained answer: No. It describes observed use in the selected subcorpus. Teaching priority also depends on meaning, curriculum and intended tasks.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 14 OF 22

14. Read the miniature dataset completely

Our worked example separates a fictional 30,000-token school archive into fiction, science teaching and family notices. The groups do not overlap for this first calculation, so their counts can be summed. The target is the exact written form charge regardless of meaning.

A concrete example

Fiction contributes 10,000 eligible tokens and forty matches; science contributes 12,000 and sixty; family notices contribute 8,000 and eight. These are invented to make the denominators visible. The pooled corpus has 30,000 tokens and 108 matches.

Where an attractive answer goes wrong

The false shortcut reports a rate of 4,000 per million because fiction has 4,000, or averages the three subcorpus rates without considering how much text each provides. An unweighted average of rates is not the pooled token-level rate when sizes differ.

A better decision

Compute each subset’s per-million rate, then compute the pooled rate from summed raw hits divided by summed eligible tokens. For a weighted average of subcorpus rates, use the relevant token shares.

Check your understanding: What is the pooled rate of 108 matches in 30,000 eligible tokens?

Explained answer: 108/30,000×1,000,000 = 3,600 occurrences per million tokens. It is not the simple mean of 4,000, 5,000 and 1,000.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 15 OF 22

15. Use intersection and exclusion to test a question

A metadata system may support combining filters, such as science AND year 2026, or fiction excluding textbooks. Intersections and exclusions help target more exact contexts, but can rapidly produce very small analytic sets.

A concrete example

Suppose the science subset has 12,000 tokens, but only 3,000 are in texts from a chosen month. A special question about that month can use the intersection, provided the number of included texts and the selection reason are reported.

Where an attractive answer goes wrong

The temptation is to keep narrowing filters until a surprising rate appears, then present that rate as though the narrowest group had been the original research target.

A better decision

Plan key comparisons in advance when possible. Label exploratory subsets as exploratory and treat their patterns as hypotheses for checking on independent material.

Check your understanding: Can a striking result from a tiny intersection automatically represent the entire science subset?

Explained answer: No. It is a description of the filtered intersection until broader evidence supports a generalisation.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 16 OF 22

16. Avoid comparing a subset with its parent as independent samples

A science subcorpus and its full parent corpus share all science tokens. Comparing their rates can be useful descriptively, but the groups are not independent. The parent is partly composed of the subset, which affects any inferential calculation.

A concrete example

If science has 12,000 of the parent’s 30,000 tokens, its word pattern contributes to the parent baseline. A science-versus-whole comparison will usually show a smaller contrast than science-versus-remainder for a term concentrated in science.

Where an attractive answer goes wrong

The mistake treats the subset and parent as separate, independently observed populations and applies a test whose assumptions require independent groups.

A better decision

For a direct contrast, compare the science subset with the complement that excludes it when that fits the question. State which baseline is used, especially for keyword or statistical comparisons.

Check your understanding: Why may science versus the remaining texts be clearer than science versus all texts?

Explained answer: The complement contains no science tokens, so the two observed groups do not overlap. The distinction clarifies the comparison while not solving all sampling concerns.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 17 OF 22

17. Control multiple comparisons and exploration

With hundreds of possible genres, years and words, a researcher can obtain eye-catching differences by chance or by selecting one dramatic case. Subcorpus tools make exploration convenient; they also make post-hoc cherry-picking easy.

A concrete example

Imagine searching fifty target words across twenty tiny subsets and publishing only the largest ratio. That ratio may reflect ordinary sampling noise, a short document or a rare spelling rather than a stable group characteristic.

Where an attractive answer goes wrong

A sensational headline makes one discovered contrast look as though it was the single planned test. This conceals the many alternative combinations explored.

A better decision

Keep a search log. Distinguish preplanned comparisons from exploratory findings, and replicate important patterns on comparable material when possible. Avoid unnecessary claims of statistical significance.

Check your understanding: Why is it valuable to record queries that produced no dramatic difference?

Explained answer: The record shows the scope of exploration and helps readers understand how a reported contrast was found.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 18 OF 22

18. Preserve consent and access restrictions

Subcorpora based on classroom or student material may expose sensitive information even when aggregate counts appear harmless. Text filters are not permission filters. A learner’s private writing should not be redistributed simply because the words are educationally interesting.

A concrete example

A corpus of anonymised student reflections may be available for a limited educational purpose. Creating a publicly downloadable subgroup of personal writing could exceed that permission even if the frequency table itself is innocuous.

Where an attractive answer goes wrong

The shortcut assumes that a research tool’s export button authorises republication or unrestricted use of the underlying texts. Technical ability does not establish privacy rights.

A better decision

Apply the data-use rules of the collection, minimise personal identifiers and publish constructed examples or lawful aggregate results when individual texts cannot be shared. Consider small-group disclosure risks.

Check your understanding: Can a frequency chart be shared when the source essays cannot?

Explained answer: Possibly, subject to the relevant permissions and privacy safeguards. The rights and disclosure risks of aggregates and full texts should be evaluated separately.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 19 OF 22

19. Teach subcorpora through a genuine school task

Students learn the idea best when two filters produce visibly different contexts. Ask them to examine a shared word in a short constructed story and a constructed science explanation. The objective is to explain the text-purpose difference, not merely announce a larger count.

A concrete example

The word light might refer to illumination in a narrative and relative mass in a science sentence. Even if both subsets contain the form, the surrounding grammar and meaning differ. A two-genre lesson can connect vocabulary with subject learning.

Where an attractive answer goes wrong

A weak activity asks pupils to memorise ‘subcorpus means a small corpus’ without practising what is included, why the groups differ or how they may overlap.

A better decision

Have learners define the parent archive, two subset rules and one boundary case. Then let them compare a small set of context examples and write a two-sentence interpretation with a stated limit.

Check your understanding: What is stronger evidence of understanding than recalling the word subcorpus?

Explained answer: The learner can construct a subset rule, apply it consistently, recognise overlap and explain why a conclusion applies only to that subset.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 20 OF 22

20. Use subcorpora to improve teaching examples

A practical vocabulary list depends on what the learner will read or write. A science subcorpus may help identify verbs and nouns used for experiments, while a narrative subcorpus may show movement and description. Frequency should be interpreted with comprehension difficulty and transfer demands.

A concrete example

One teacher might prepare task-specific examples of observe, infer and justify; another might focus on dialogue verbs and descriptive adjectives. The aim is not to claim that a language form is inherently superior because one genre uses it frequently.

Where an attractive answer goes wrong

The false shortcut produces a universal ranked list from a highly specialised archive and assigns it to every student regardless of purpose, age or prior vocabulary knowledge.

A better decision

Connect the selected subset to a real reading or writing goal. Teach the word in sentences, invite meaningful variation and check whether students can use it in a new context.

Check your understanding: Can two students reasonably need different target words from the same parent corpus?

Explained answer: Yes. Their learning tasks may require different subsets and different kinds of productive vocabulary practice.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 21 OF 22

21. Document and version your subset definitions

A saved subcorpus is a reproducible selection only if the parent corpus version, metadata fields and filter logic remain known. New texts, corrected labels or altered date boundaries can change subset membership and every derived statistic.

A concrete example

Suppose a 2026 science subset originally contains twenty files. An audit finds that three belonged to 2025 and that two 2026 files were missing. The revised subset might still have nineteen files, but it is not the same collection.

Where an attractive answer goes wrong

The mistake quietly updates a filter and leaves old frequency charts alongside new ones with identical labels. Readers may believe the difference comes from language change instead of data management.

A better decision

Version the definition and membership list; record added and removed documents, reasons and changes to token totals. Recalculate comparisons rather than editing only their headings.

Check your understanding: Does a stable subcorpus name guarantee that its contents never change?

Explained answer: No. The underlying corpus or metadata can change. Versioning protects the meaning of earlier results.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Next chapter

CHAPTER 22 OF 22

22. Finish with a ten-point subset audit

Before making a subcorpus claim, inspect the parent population, selection question, filter, unit, metadata, overlap, text volume, query, denominator and meaning in context. This audit prevents a small filter from appearing more authoritative than it is.

A concrete example

A valid modest conclusion might read: ‘In this constructed set of science instructions, the exact form charge occurs at a higher per-token rate than in the constructed family notices; the contexts and senses may differ.’ That sentence shows what was compared and what remains open.

Where an attractive answer goes wrong

An overclaim says that all science learners use charge five times as often in speech. The toy data contain documents, not a measurement of student speech, and the rate is scoped to those invented collections.

A better decision

Publish the audit with an example table and source notes. Ask a peer to reconstruct the subsets and recompute one rate. If the peer cannot, improve the method description before broadening the interpretation.

Check your understanding: What makes a comparison responsibly usable in a learning guide?

Explained answer: Visible selection rules, compatible counts and denominators, context inspection and an explicit limit on generalisation.

This distinction is most useful when it changes what the reader can safely conclude. Write down the source or counting rule you would need to inspect, say what remains unknown, and resist upgrading a local observation into a claim about all writers, students or texts. Where a method offers several reasonable choices, keep those choices visible rather than presenting one decision as a universal rule.

Contents · Previous chapter · Worked dataset

Worked dataset: inspect every assumption

Our fictional parent corpus contains three non-overlapping types of school text. The query counts the exact written form charge, ignoring its meanings for this first comparison. The per-million rate is count divided by eligible tokens multiplied by 1,000,000. This is a constructed classroom illustration, not a claim about actual school vocabulary.

Disjoint subcorpus	Eligible tokens	Matches of charge	Occurrences per million	What this means
Fiction	10,000	40	4,000	Relatively common within this invented fiction set
Science teaching	12,000	60	5,000	Highest observed rate of the three
Family notices	8,000	8	1,000	Lower observed form frequency
Whole parent corpus	30,000	108	3,600	Token-weighted pooled rate

For fiction: 40/10,000×1,000,000 = 4,000. For science: 60/12,000×1,000,000 = 5,000. For notices: 8/8,000×1,000,000 = 1,000. These are equal-scale comparisons, but they do not distinguish payment meanings from electrical or other senses of charge.

Because the three teaching subsets are non-overlapping and together cover the parent example, the pooled total is 40+60+8=108 hits and 10,000+12,000+8,000=30,000 eligible tokens. The pooled rate is 3,600 per million, not the unweighted mean of 3,333.3.

The token-weighted calculation can be checked another way: 4,000×(10,000/30,000) + 5,000×(12,000/30,000) + 1,000×(8,000/30,000) = 3,600. This equality relies on matching units and a disjoint exhaustive partition.

If a later filter creates a subset called ‘2026 science’, its tokens overlap with science. The new view should be analysed on its own; do not add its totals again to the parent. No arithmetic can resolve which sense of charge appears without inspecting the sentences.

A reliable report would also include the number of documents, their lengths, metadata accuracy and a description of how the corpus was selected. Those fields prevent readers from generalising the constructed result to students or schools.

Twelve transfer cases: from tidy counts to defensible conclusions

These cases are purposefully varied across school communication, research, workplace writing and general reading. They ask for a decision, not simply a recited definition; each answer is bounded by its supplied evidence.

1. Two writing purposes

Situation. A word is frequent in science methods but absent from diary excerpts.

Hidden issue. Different communicative tasks shape the samples.

Reasoned response. Explain the source-specific usage without claiming the word is unknown to diary writers.

Further question. Would a sense-based concordance change the interpretation?

2. Two overlapping filters

Situation. All science texts are also tagged 2026.

Hidden issue. The same tokens are in two saved subsets.

Reasoned response. Report the intersection; do not add overlapping denominators as new evidence.

Further question. What would the unique union contain?

3. One oversized textbook

Situation. A science subset has one textbook and ten small worksheets.

Hidden issue. One source may dominate token-level counts.

Reasoned response. Show both per-token counts and per-document spread.

Further question. Would limiting text length answer the intended question better?

4. Unknown writer

Situation. Some records are labelled ‘student’ without verified authorship.

Hidden issue. The group name may misrepresent who wrote the text.

Reasoned response. Audit metadata and label ambiguous items unknown.

Further question. Which source field confirms authorship?

5. Spoken transcription

Situation. A comparison uses recordings and edited articles.

Hidden issue. Transcription conventions and medium differ.

Reasoned response. Document source processing and interpret the patterns in context.

Further question. What is counted as a word token in each?

6. Word-selected topic

Situation. A climate subset was created by searching climate.

Hidden issue. Its keyword claim is circular.

Reasoned response. Use independent genre or topic labels for the analysis.

Further question. Can held-out documents test the pattern?

7. Month-to-month contrast

Situation. One term contains exam notices; another contains excursion notices.

Hidden issue. Time and topic vary together.

Reasoned response. Compare like genres before claiming a time trend.

Further question. Which additional metadata field is needed?

8. Privacy boundary

Situation. Classmates’ essays could reveal sensitive personal details.

Hidden issue. Analysis relevance is not permission to distribute.

Reasoned response. Use lawful aggregated analysis or synthetic teaching cases.

Further question. What must remain unpublished?

9. Missing date

Situation. A text has no verified issue date.

Hidden issue. It may be assigned to the wrong time subset.

Reasoned response. Keep an unknown-date category and verify before chronological analysis.

Further question. What changes when it is excluded?

10. Small intersection

Situation. An impressive rate comes from a single short document.

Hidden issue. One match can swing scaled rates.

Reasoned response. Disclose raw count and document number; validate on more text.

Further question. Does a million-word scale mean one million observed words?

11. Parent comparison

Situation. A subset is compared against the whole parent and treated as independent.

Hidden issue. The subset contributes to its own reference baseline.

Reasoned response. Use the non-overlapping complement when the question needs separate groups.

Further question. How would the comparison baseline change?

12. Teaching vocabulary

Situation. A general list is built from only a laboratory corpus.

Hidden issue. Specialist vocabulary is mistaken for universal priority.

Reasoned response. Match the subcorpus to learner goals and inspect transfer tasks.

Further question. Would a narrative-reading student have the same needs?

Eight practice questions with explanations

Answer each question before opening the explanation. For a numerical question, show the denominator and unit; for an eligibility or annotation question, state the exact boundary of the decision. A correct number with a false generalisation is not a complete answer.

Exercise 1: Define the subcorpus

Question. A parent has fiction, science worksheets and family notices. You want verbs used in science instructions. State the selection.

Explained solution. Select independently identified science instructional texts; define the verb query and eligible token universe consistently.

Common wrong turn. Searching only for the target verb to define science texts would be circular.

Exercise 2: Compute a rate

Question. A subset has 30 hits among 15,000 eligible tokens. Find the per-million rate.

Explained solution. 30/15,000×1,000,000 = 2,000 occurrences per million.

Common wrong turn. Thirty occurrences are observed; 2,000 is a scaled rate, not an observed count.

Exercise 3: Join overlapping subsets

Question. Group A has 12,000 tokens, B 18,000, and 5,000 belong to both. Find the unique union.

Explained solution. 12,000+18,000−5,000=25,000 unique tokens.

Common wrong turn. Simply adding to 30,000 double-counts the overlap.

Exercise 4: Find the pooled rate

Question. Disjoint groups contain 40 hits in 10,000 tokens and 60 in 12,000. Compute the pooled rate.

Explained solution. 100/22,000×1,000,000≈4,545.45 per million.

Common wrong turn. A simple mean of the two group rates ignores unequal token denominators.

Exercise 5: Check an unclear genre

Question. A newsletter includes a science experiment instruction. How should it be labelled?

Explained solution. Apply the predeclared genre unit rule; decide whether the unit is the whole newsletter or an independently coded instructional section.

Common wrong turn. Labelling only from the target word can create biased membership.

Exercise 6: Rewrite an overclaim

Question. A chart says ‘all students use this word’ based on staff newsletters.

Explained solution. Rewrite as a frequency finding about the sampled staff newsletters and distinguish it from student production.

Common wrong turn. Source audience and writer identity are not interchangeable.

Exercise 7: Compare one word’s senses

Question. The form charge is frequent in finance and science subsets. What remains unknown?

Explained solution. Which senses, grammatical roles and constructions drive the counts; inspect concordance evidence before a meaning-specific conclusion.

Common wrong turn. Identical spelling does not prove identical meaning.

Exercise 8: Version a change

Question. A document date correction moves it from 2025 to 2026. What must be updated?

Explained solution. The membership register, token totals and affected queries and charts; log the old and new subset definitions.

Common wrong turn. A changed dataset should not silently inherit the same analysis version.

Frequently asked questions: Subcorpus

Is a subcorpus just a smaller corpus?

It is a selected portion of a parent corpus defined by a criterion. Size alone does not explain the selection.

Can one document belong to several subcorpora?

Yes. Overlapping filters are normal, but overlapping tokens must not be added as independent data.

What fields define subcorpora?

Common fields include text type, genre, medium, time, source, writer role and other relevant metadata.

Does a subcorpus represent its wider population?

Only to the extent justified by its source coverage and selection design; a filter alone does not guarantee representativeness.

Can I compare a subcorpus with its parent?

Yes descriptively, but remember the datasets overlap and may not satisfy assumptions of independent comparisons.

Why must I normalise word counts?

Unequal token totals provide unequal opportunities for occurrences; a common denominator helps compare rates.

Do subcorpora have to be disjoint?

No. They can overlap. Distinguish overlapping views from an exhaustive disjoint partition.

What if some documents lack genre metadata?

Audit, verify where possible and preserve an unknown category rather than silently pretending they fit.

Can a keyword search create a subcorpus?

Yes for some exploratory purposes, but selecting on the target word compromises later claims about how distinctive that word is.

Is a subcorpus the same as a sample?

No. It is a selected portion of a corpus; a sample may be drawn from it and representativeness requires additional reasoning.

What is the minimum useful subcorpus size?

No universal threshold fits all tasks. Rare-word, grammar and quantitative comparisons demand different kinds of coverage.

How do I make my subcorpus work reproducible?

Save eligibility rules, corpus version, source IDs, metadata, processing and analysis query settings.

Can subcorpora help teachers?

Yes. They can reveal task-specific word usage, provided lessons also teach meaning, context and transfer.

Short glossary for working readers

• Parent corpus: The full language collection from which a subset is selected.
• Subcorpus: A defined portion selected by metadata or other criteria.
• Genre: A recurring communicative purpose and text pattern.
• Metadata: Recorded properties such as source, period and text type.
• Overlap: Units appearing in more than one selected subset.
• Union: All unique units included in any of several sets.
• Intersection: Units meeting two or more selection conditions.
• Token: A counted text element under a stated segmentation policy.
• Normalised frequency: An occurrence rate scaled to a common token denominator.
• Keyword: A word statistically or descriptively salient relative to a defined comparison group.
• Concordance: A display of word occurrences with surrounding context.

A four-stage teaching and verification plan

First, begin with the specific claim that a learner wants to make. Reframe it as a question about observable language, not as a contest to find the largest number. The learner writes the target, units and intended audience on one line. That starting line is a useful quality control because it can be compared directly with every later result.

Second, work with a miniature source register and deliberate edge cases. Ask learners to label the obvious examples and then spend more time on duplicated files, ambiguous sentences, unbalanced genres or missing metadata. Differences of judgement are productive when each person can explain the rule and what evidence would settle the case.

Third, reconstruct the worked example without the answers. Keep the same declared rules but change one detail, such as a document’s genre, an inclusion condition, a window length or whether a repeated notice is counted twice. Record what changes in the numerical result. This makes the new vocabulary a working method rather than a definition to memorise.

Fourth, practise transfer. Have the learner write one cautious conclusion for a reading lesson, one for a research report and one for a public-facing explanation. Each should identify what the observed evidence supports and one limitation. If a conclusion overreaches, revise its scope before polishing its style. Clear reasoning is a more durable writing skill than a dramatic claim.

Methodological sources

• Sketch Engine: create and use a subcorpus
• Sketch Engine glossary: subcorpus
• Sketch Engine: compare corpora
• Cambridge University Press: approaches to corpus representativeness

Sources provide conceptual grounding; the fictional worked datasets in this guide are not findings reported by the linked authors.

Related vocabulary evidence guides

• What Is a Corpus Sampling Frame? — Define your source population
• What Is Normalised Word Frequency? — Compare different-sized corpora
• What Is Corpus Metadata? — Source details that make word evidence interpretable
• What Is a Corpus Query? — Know what the search actually finds
• What Is a Gold-Standard Corpus? — Keep checked references distinct from raw examples
• Vocabulary Learning Hub
• Vocabulary Article Library

Return to chapter contents · Vocabulary Learning Hub