Corpus source clustering: is the practice of recognising and, where useful, grouping language records that share an underlying author, publication, institution, template, conversation, event or other source process. For analysis it makes related observations visible, so a collection of many text files is not mistaken for the same number of independent speakers or writers.
Did you know that twelve documents can contain only three independent writers? Imagine a school archive with five notices from Writer A, four texts from Writer B and three from Writer C. A phrase might appear twenty-six times overall, yet most occurrences may come from one writer. Counting pages tells you one thing; counting independent voices tells you another.
This guide distinguishes source clusters from exact duplicates and genre-based subcorpora. It explains nested files, multiple authorship, original versus repeated source material, cluster-aware uncertainty, effective coverage and practical reporting. Its invented twelve-document table shows why counting source IDs matters before using statistics to make a claim about people.
All invented records and counts in this article are labelled constructed examples. They are intended to make the statistical or methodological steps reproducible, not to represent measured Singapore classroom data. The linked external references support the described methods, not the invented numerical results.
EDUKATE · VOCABULARY · EVIDENCE METHODS
Choose your reading route
- Understand the question and definition · Chapter 1 onward
- Build a reliable method · Chapter 7 onward
- Interpret the full evidence · Chapter 13 onward
- Use the insight in school or research · Chapter 19 onward
All chapters · Full worked example · Twelve cases · Practice answers · FAQ
Complete chapter index
- 1. Start with what a corpus observation represents
- 2. Define a cluster operationally
- 3. Separate exact duplicates from related originals
- 4. Recognise repeated templates as another link
- 5. Understand why source dependence matters
- 6. Keep documents and clusters as two separate totals
- 7. Inspect source concentration
- 8. Know that document coverage is not author coverage
- 9. Recognise hierarchical sources
- 10. Allow crossed and overlapping memberships
- 11. Calculate per-cluster word rates
- 12. Distinguish pooled and author-average rates
- 13. Read the twelve-document miniature archive
- 14. Treat sampling selection as a separate quality issue
- 15. Interpret a cluster-aware bootstrap
- 16. Know when document bootstrapping is insufficient
- 17. Avoid pseudo-replication in findings about people
- 18. Use source clusters to make teaching lists less misleading
- 19. Protect writer identities while preserving useful structure
- 20. Document the source relationship and its version
- 21. Audit source diversity before claiming representativeness
- 22. Finish with a source-first interpretation
CHAPTER 1 OF 22
1. Start with what a corpus observation represents
A text collection may contain documents, word tokens, speakers, writers, publication events and source templates. These units are not identical. Before counting word occurrences, identify which unit matters for the question being asked.
A worked situation
A website archive with twelve pages may represent twelve publication records but only three authors. A study of page exposure and a study of writer preference therefore have different denominators.
What a tidy but misleading result would say
The misleading conclusion says twelve pages prove twelve independent writers prefer a phrase. The number of files alone cannot establish authorship or independence.
The evidence-first decision
Record document ID and any permitted author, source or template identifiers. Choose the analytic unit by the intended inference.
Check the distinction: Does twelve documents automatically mean twelve people?
Explanation: No. One writer or institution may produce many documents.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 2 OF 22
2. Define a cluster operationally
A source cluster is a group linked by a declared relationship relevant to the analysis. A cluster might be all documents from one author, all notices derived from one template or all utterances from one participant.
A worked situation
For a writer-specific analysis, our invented example groups D01–D05 under Writer A, D06–D09 under Writer B and D10–D12 under Writer C.
What a tidy but misleading result would say
The shortcut groups texts together because they discuss similar topics and then assumes that topic similarity proves common authorship.
The evidence-first decision
State the clustering criterion and evidence. Different relations can create different valid clustering schemes for distinct questions.
Check the distinction: Can genre similarity alone establish shared authorship?
Explanation: No. Similar purpose or vocabulary does not prove one source or writer produced the texts.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 3 OF 22
3. Separate exact duplicates from related originals
Two identical copies of one notice are duplicate text for a unique-wording study. Two genuinely different messages written by one person are related by author but are not duplicates. Clustering and deduplication address different dependencies.
A worked situation
Writer A may compose five distinct notices, all with similar vocabulary. Removing four notices as ‘duplicates’ merely because the author is the same would destroy genuine information.
What a tidy but misleading result would say
The mistaken workflow deletes every record belonging to the same author until only one remains. It confuses nonindependence with redundant data.
The evidence-first decision
Preserve different texts and record shared sources. Deduplicate only when the project identifies reused text under an explicit policy.
Check the distinction: Must every cluster contain identical wording?
Explanation: No. Related documents can be original but correlated in style, topic or source conditions.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 4 OF 22
4. Recognise repeated templates as another link
Institutional messages may share forms, footers, prompts or boilerplate. Their vocabulary can reflect a single underlying template rather than repeated independent choices by different authors.
A worked situation
Ten class notices might reuse the phrase ‘please submit by Friday’ from the same template. They can be ten delivered messages but not ten newly invented phrasings.
What a tidy but misleading result would say
The shortcut attributes every repetition to independent lexical preference while ignoring a copied template.
The evidence-first decision
Track the template identifier and decide whether the analysis concerns authored variety or actual reader exposure. Both questions can be legitimate with different counts.
Check the distinction: Can copied notices be meaningful reader encounters while providing little independent wording evidence?
Explanation: Yes. Exposure events and independent authoring are separate units.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 5 OF 22
5. Understand why source dependence matters
Words within the same writer’s texts may share habitual phrasing, curriculum, topic or audience. Treating those documents as independent draws can misstate uncertainty in a population-level estimate.
A worked situation
Writer A repeatedly writes submit in official reminders, while Writer B rarely uses it. More pages from A provide more word tokens but not necessarily proportionally more independent information about the variety of writers.
What a tidy but misleading result would say
The error applies an independent-document statistical method despite knowing that many pages share the same author and communication process.
The evidence-first decision
Use an inference method suited to the data hierarchy, and report source clusters alongside token totals and document counts.
Check the distinction: Are word tokens inside the same writer’s documents guaranteed independent?
Explanation: No. Shared source conditions can create dependence in linguistic choices.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 6 OF 22
6. Keep documents and clusters as two separate totals
A corpus can be large in token count, have many files and still contain few distinct authors or source processes. Reporting several totals provides a more honest picture of the evidence.
A worked situation
Our worked archive has twelve documents, 1,200 eligible tokens and three writer clusters. All three numbers are correct, but they answer different questions.
What a tidy but misleading result would say
The shortcut writes ‘n=1,200 participants’ because the corpus has 1,200 word tokens. Tokens are not participants.
The evidence-first decision
State document count, token total and distinct primary clusters with their definitions. Avoid using n without specifying the unit.
Check the distinction: How many writer clusters exist in the sample?
Explanation: Three, even though there are twelve documents.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 7 OF 22
7. Inspect source concentration
A word can be frequent overall because one source group contributes most of its uses. Source concentration is not inherently erroneous; it matters when the claim concerns widespread practice across speakers.
A worked situation
Writer A has twenty target hits, Writer B zero and Writer C six. The pooled count is twenty-six, but Writer A supplies 20/26≈76.92 percent of all observed matches.
What a tidy but misleading result would say
The misleading headline says the target is equally typical of all three writers because the archive contains twenty-six hits overall.
The evidence-first decision
Show counts per cluster and the number of clusters with at least one hit. Interpret the total alongside that concentration.
Check the distinction: What proportion of the constructed target hits come from Writer A?
Explanation: Twenty of twenty-six, approximately 76.92 percent.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 8 OF 22
8. Know that document coverage is not author coverage
A term may occur in five files by one writer and none from others. Its document frequency is five, whereas its writer prevalence is one. Neither number is a substitute for the other.
A worked situation
Writer A’s phrase might appear in all five notices. A document-presence metric gives five, but an author-presence metric gives one writer.
What a tidy but misleading result would say
The mistake reports ‘five independent speakers used the expression’ from the five file hits.
The evidence-first decision
Keep both measures and choose the one appropriate to exposure, authored text diversity or social distribution.
Check the distinction: Can a word appear in five documents but only one author’s work?
Explanation: Yes. The documents may all have the same author.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 9 OF 22
9. Recognise hierarchical sources
Documents may be nested in authors, authors in schools and texts in publications or time periods. This hierarchy can influence both vocabulary rates and the amount of independent evidence.
A worked situation
A school corpus could contain thirty paragraphs across ten essays written by five students in two classes. The hierarchy is paragraphs→essays→students→classes, if the sampling actually follows that structure.
What a tidy but misleading result would say
The shortcut treats each paragraph as a new independent learner, inflating the apparent participant count.
The evidence-first decision
Record the sampling structure and avoid treating nested units as independent when estimating variability or learner differences.
Check the distinction: What is the difference between five students and thirty paragraphs?
Explanation: One count concerns people; the other concerns text units produced by those people.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 10 OF 22
10. Allow crossed and overlapping memberships
Language data are not always a simple hierarchy. A document may have several authors, refer to an institutional template or be republished through multiple channels. Cluster rules must say how such relationships are represented.
A worked situation
A jointly authored report belongs to two writer identities but one publication record. A copied extract can appear in several outlets while retaining one underlying source lineage.
What a tidy but misleading result would say
The mistake forces every record into exactly one writer or publication cluster without an explicit policy, creating misleading independence claims.
The evidence-first decision
Use multiple source fields or a provenance graph when needed. If a simple cluster definition cannot capture the relationships, state the simplification.
Check the distinction: Can one document have multiple relevant source links?
Explanation: Yes. Joint authorship, republication and templates can create overlapping relationships.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 11 OF 22
11. Calculate per-cluster word rates
A pooled word rate is total occurrences divided by total eligible tokens. A cluster-specific rate uses that cluster’s counts and tokens. Averages of cluster rates represent a different target if clusters vary in text volume.
A worked situation
A contributes 20 hits in 500 tokens, or 40 per thousand. B contributes 0 in 400, or zero. C contributes six in 300, or 20 per thousand.
What a tidy but misleading result would say
The shortcut reports Writer A’s rate as the rate of all writers, or mistakes the simple mean of cluster rates for the pooled token rate.
The evidence-first decision
Present both the pooled token rate and cluster rates, with clearly labelled denominators.
Check the distinction: What is Writer C’s rate?
Explanation: 6/300×1,000=20 per thousand.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 12 OF 22
12. Distinguish pooled and author-average rates
Pooling weights writers by their observed token contribution. An unweighted average of writer-specific rates gives each writer equal influence. Both can be useful but answer different questions.
A worked situation
The pooled toy rate is 26/1,200×1,000≈21.67 per thousand. The mean of Writer A=40, B=0 and C=20 per thousand is (40+0+20)/3=20 per thousand.
What a tidy but misleading result would say
The mistake calls one result an arithmetic error because it differs from the other, without recognising the different estimands.
The evidence-first decision
Name the target: prevalence per token in the sampled text or typical writer-level rate. Avoid switching between them mid-argument.
Check the distinction: Why do 21.67 and 20 per thousand both appear?
Explanation: The first weights each writer by observed token volume; the second gives three writers equal weight.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 13 OF 22
13. Read the twelve-document miniature archive
The worked table contains twelve fictional 100-token documents, grouped under three invented writers. Every Writer A document has four target hits, every Writer B document zero and every Writer C document two.
A worked situation
Five A documents give 500 tokens and twenty hits; four B documents give 400 and zero; three C documents give 300 and six. Totals are 1,200 tokens and twenty-six hits.
What a tidy but misleading result would say
The misleading claim treats the constructed table as evidence that actual teachers or student writers differ in vocabulary preference. Its values were chosen to make the arithmetic visible.
The evidence-first decision
Compute per-writer, per-document and pooled totals without attaching them to real populations. Note that independence cannot be established solely from the apparent number of files.
Check the distinction: How many total target matches are in the toy archive?
Explanation: Twenty-six across the twelve constructed documents.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 14 OF 22
14. Treat sampling selection as a separate quality issue
Knowing source clusters does not guarantee that the sampled writers represent the population. An archive may overrepresent prolific authors or particular genres and omit groups with different language habits.
A worked situation
If Writer A contributes five documents and others less, the pooled rate gives A more influence. That can be suitable for a question about observed token exposure but not automatically about typical writers.
What a tidy but misleading result would say
The shortcut assumes that clustering automatically fixes sampling bias or produces a representative national estimate.
The evidence-first decision
Keep source-frame information, writer coverage and selection mechanism separate from clustering. Where relevant, consider appropriate design weights or broader collection.
Check the distinction: Does clustering three existing writers create evidence about writers absent from the sample?
Explanation: No. The observed group still lacks uncollected voices.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 15 OF 22
15. Interpret a cluster-aware bootstrap
A cluster bootstrap resamples higher-level units so dependence within a selected cluster is kept together. It may be appropriate for some corpus estimates, but only if the cluster definition matches the sampling design.
A worked situation
For an author-level study, a replicate could select Writers A, A and C, carrying all texts of each selected writer together. It would not select five random paragraphs from A and call them five authors.
What a tidy but misleading result would say
The mistake resamples every word token independently and assumes the interval reflects between-writer uncertainty.
The evidence-first decision
Choose the resampling unit from the design and preserve cluster membership. With only three clusters, an interval would be fragile and require strong caveats.
Check the distinction: Does resampling writers create new independent real writers?
Explanation: No. It constructs pseudo-samples from the observed writer clusters.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 16 OF 22
16. Know when document bootstrapping is insufficient
Document-level resampling can preserve intra-document context but fail to capture dependence between multiple documents by the same author. Source clustering indicates when a higher-level unit may be needed.
A worked situation
Five Writer A notices share habitual language. Resampling each notice as though it came from a separate independent writer ignores that relationship.
What a tidy but misleading result would say
The shortcut equates file boundaries with the true sampling units in every study.
The evidence-first decision
Use author or relevant higher-level grouping when the inferential question concerns independent writers, subject to actual sampling design.
Check the distinction: Can a document bootstrap be inappropriate even if its arithmetic is correct?
Explanation: Yes. The resampling unit may not match the dependence structure or intended population.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 17 OF 22
17. Avoid pseudo-replication in findings about people
Pseudo-replication occurs when related observations are analysed or described as though they were independent units, exaggerating the effective evidence for a claim. Text corpora are especially vulnerable when many sentences come from few sources.
A worked situation
A school analysis claims a result from ‘1,200 learner observations’ when its 1,200 units are tokens written by three people. It would overstate how many learners were actually observed.
What a tidy but misleading result would say
The error gives a narrow writer sample artificial authority by substituting token count for person count.
The evidence-first decision
State each sample size with its unit and use appropriate methods for correlated observations. More tokens do not automatically mean more independent voices.
Check the distinction: What is the pseudo-replication danger in this toy corpus?
Explanation: Calling 1,200 tokens or twelve files 1,200 or twelve independent writers.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 18 OF 22
18. Use source clusters to make teaching lists less misleading
A word list derived from one institution’s template may reveal that organisation’s preferred phrasing rather than every pupil’s natural language. Recognising clusters can help tutors choose broader, more relevant example sources.
A worked situation
The phrase ‘please submit’ may dominate administrative notices but appear rarely in narrative essays. A learner writing formal requests may still need it, while a fiction task requires different vocabulary.
What a tidy but misleading result would say
The shortcut labels the top words from one source cluster ‘the essential words all students use’. It ignores genre and author coverage.
The evidence-first decision
Match the examples to a learning goal and deliberately examine varied source styles, keeping each source’s provenance clear.
Check the distinction: Is a concentrated institutional phrase necessarily unimportant?
Explanation: No. Its task-specific usefulness can be high even if few independent writers generated it.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 19 OF 22
19. Protect writer identities while preserving useful structure
A source study may require identifying that texts share a writer without publishing private names. Privacy and data-minimisation rules still apply, especially to children’s writing and workplace communication.
A worked situation
Use anonymous Writer A, B and C codes in a published illustration or lawful aggregate analysis. An internal access-controlled mapping may be necessary for authorised audit without revealing personal details.
What a tidy but misleading result would say
The mistake assumes that statistical value automatically permits publishing identifiable student essays or author labels.
The evidence-first decision
Minimise personal data, respect permissions and use constructed examples for public teaching when real records cannot be shared.
Check the distinction: Can a report disclose source clustering without naming children?
Explanation: Often yes, through anonymous stable IDs and aggregates, subject to reidentification risk and relevant consent.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 20 OF 22
20. Document the source relationship and its version
Author records, templates and source attribution can change as an archive is corrected. A reproducible clustering analysis should record how membership was established and which version of the corpus was used.
A worked situation
A file initially attributed to Writer B is later found to be coauthored by A and B. The change affects cluster membership even when its text and word count remain identical.
What a tidy but misleading result would say
The mistaken workflow silently edits author IDs and leaves earlier graphs with unchanged labels, creating an apparent linguistic trend caused by a data correction.
The evidence-first decision
Version source metadata and document the effect of attribution corrections on counts and uncertainty analyses.
Check the distinction: Can source clustering change without any vocabulary changing?
Explanation: Yes. Correcting authorship or template lineage can change group assignments while leaving text intact.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 21 OF 22
21. Audit source diversity before claiming representativeness
A responsible report should state how many documents, tokens, authors or source clusters were observed; what counts as a cluster; how repeated text was handled; and whether the collection covers the intended domain.
A worked situation
A cautious statement might say ‘Twenty-six target hits occurred in twelve constructed text files from three assigned writer clusters, with twenty hits in the five files from one writer.’ This is precise and reproducible.
What a tidy but misleading result would say
The unsupported claim says the phrase is preferred by the population because most observed hits came from the most prolific writer.
The evidence-first decision
Give both word-level and source-level summaries, and state the inferential boundary. Look for missing voices before broadening the conclusion.
Check the distinction: What is the central limitation of the worked sample?
Explanation: Only three constructed writer clusters are represented, and the counts are not empirical observations.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
CHAPTER 22 OF 22
22. Finish with a source-first interpretation
The purpose of source clustering is to prevent one collection of related texts from pretending to be many independent language producers. It supports better descriptions of distribution, sampling and uncertainty without forbidding legitimate repeated exposures.
A worked situation
The same archive can support a statement about 1,200 observed tokens and another about three writer groups. Different questions may use different statistics; the right one depends on the intended conclusion.
What a tidy but misleading result would say
The error treats one choice of unit as universally correct and dismisses all other summaries. Token exposure, document presence and writer diversity are complementary quantities.
The evidence-first decision
End with explicit units and a claim small enough to be supported. Invite independent readers to check whether the source structure matches the language question.
Check the distinction: What makes a source-cluster analysis useful?
Explanation: It reveals where apparent sample size, independence and lexical concentration diverge.
For a transferable result, name the actual unit, the source evidence and the rule being used. A result can be calculated correctly while describing a much narrower population or task than the headline suggests. If an alternative defensible design changes the answer, describe that sensitivity. If the data are insufficient, keep the uncertainty visible rather than inventing an independent observation or a validated reference.
Worked evidence: inspect every input and total
This original table contains twelve fictional documents with 100 eligible tokens each, grouped by three fictional writers. The target word appears four times in each of five Writer A documents, zero times in each of four Writer B documents and twice in each of three Writer C documents. Files are distinct original messages for the exercise, not copies.
| Source group | Number of documents | Eligible tokens | Target hits | Rate per 1,000 | Distinct writers |
|---|---|---|---|---|---|
| Writer A | 5 | 500 | 20 | 40.00 | 1 |
| Writer B | 4 | 400 | 0 | 0.00 | 1 |
| Writer C | 3 | 300 | 6 | 20.00 | 1 |
| Pooled original sample | 12 | 1,200 | 26 | 21.67 | 3 |
| Equal-writer mean rate | 3 writer rates | Not a pooled token denominator | Not an observed count | 20.00 | 3 |
The pooled rate is 26/1,200×1,000≈21.67 hits per thousand tokens. The source counts are twelve document records and three writer identities, not twelve independent writers. Neither count is an error; they address different units.
Writer A contributes 20 of 26 matches, or approximately 76.92 percent of occurrences, from 500 of the 1,200 tokens (41.67 percent of token volume). Its own rate is 20/500×1,000=40 per thousand.
Writer B contributes 400 tokens and no target matches; Writer C contributes six among 300 tokens, a rate of 20 per thousand. These differences are constructed to make between-writer variation visible.
Giving each writer equal weight yields (40+0+20)/3=20 per thousand. This is not the same estimand as pooling all tokens. The equal-writer rate should not be represented as 24 invented observed hits or as a guaranteed population estimate.
If a claim concerns independent writers, its independent sample size is at most the three writer clusters under this constructed assignment, subject to any other shared institutional or template links. An author-level bootstrap with only three writers would be fragile.
Document-level deduplication, author grouping and genre stratification are complementary analyses. A real study should define source IDs, verify provenance, consider within-cluster dependence and report limitations before making population inferences.
Twelve transfer cases
Each fictional case tests a distinct decision. Read the problem and propose a correction before consulting the reasoned response.
1. One prolific editor
Situation. One writer publishes most of the archive.
Hidden issue. Pages overrepresent one person’s style.
Reasoned response. Show author share and per-writer rates.
New question. What is the intended unit of inference?
2. Shared template
Situation. Different people use one standard form.
Hidden issue. Repeated wording may derive from one source.
Reasoned response. Track template lineage separately from authors.
New question. Are these independent lexical choices?
3. Distinct original texts
Situation. One writer produces five genuinely different reports.
Hidden issue. Same author does not imply duplicate content.
Reasoned response. Retain texts and group by author.
New question. What would justify deduplication?
4. Joint authorship
Situation. Two colleagues coauthor one report.
Hidden issue. One-document-one-writer assumptions fail.
Reasoned response. Record overlapping source links.
New question. How will shared items be analysed?
5. Two classes
Situation. Essays are nested in pupils and classes.
Hidden issue. Paragraphs are not independent students.
Reasoned response. Respect sampling hierarchy.
New question. At which level was selection made?
6. Repeated publication
Situation. A report appears on three sites.
Hidden issue. Web locations differ from authored source.
Reasoned response. Track one original text and its distribution events.
New question. Is exposure or originality measured?
7. Unknown authors
Situation. Several files lack provenance.
Hidden issue. Independent-source count is uncertain.
Reasoned response. Use unknown codes and bounds, not invented names.
New question. Can metadata be verified?
8. Writer removal
Situation. Excluding A changes pooled frequency.
Hidden issue. One source drives the observed rate.
Reasoned response. Report sensitivity and source concentration.
New question. Was A included appropriately?
9. Cluster bootstrap
Situation. Documents by one writer are resampled separately.
Hidden issue. Inter-document dependence remains.
Reasoned response. Resample at a relevant higher level where justified.
New question. Which clusters are exchangeable?
10. Small number of groups
Situation. An interval is computed with three writers.
Hidden issue. Few independent source groups limit inference.
Reasoned response. Qualify or collect more relevant sources.
New question. How stable is the estimate?
11. Instructional corpus
Situation. Most word examples come from one tutor.
Hidden issue. The sample may be pedagogically useful but narrow.
Reasoned response. Add varied authentic contexts appropriate to the task.
New question. What does the learner need to write?
12. Sensitive student text
Situation. Authorship IDs could identify minors.
Hidden issue. Source audit and privacy need balancing.
Reasoned response. Publish aggregates or synthetic examples.
New question. Can the underlying texts remain restricted?
Eight practice questions and explanations
Answer before viewing the solution. Show source IDs and denominators for counts, and name the reference category and predicted category when checking annotations. A correct total without a defensible interpretation is incomplete.
Exercise 1: Count source units
Question. Twelve documents are assigned to Writers A, B and C. How many distinct writers?
Explained answer. Three, assuming each identifier refers to one distinct writer.
The common shortcut. Document count must not substitute for writer count.
Exercise 2: Pooled rate
Question. Compute 26 hits in 1,200 eligible tokens per thousand.
Explained answer. 26/1,200×1,000≈21.67 per thousand.
The common shortcut. This is a token-pooled rate, not the average writer rate.
Exercise 3: Writer A rate
Question. Compute 20 hits in 500 tokens.
Explained answer. 20/500×1,000=40 per thousand.
The common shortcut. Do not divide by twelve documents.
Exercise 4: Hit concentration
Question. What share of target matches came from A?
Explained answer. 20/26≈76.92%.
The common shortcut. A’s share of hits differs from its 500/1,200≈41.67% share of tokens.
Exercise 5: Writer average
Question. Average rates 40, 0 and 20 with equal writer weight.
Explained answer. (40+0+20)/3=20 per thousand.
The common shortcut. Averaging group rates asks a different question from pooling their token totals.
Exercise 6: Duplicate versus cluster
Question. Five original messages from one writer: how many duplicates?
Explained answer. Shared authorship alone does not establish duplicates. All five may be original.
The common shortcut. Related source units are not necessarily copied text.
Exercise 7: Statistical independence
Question. Would 1,200 tokens represent 1,200 independent authors?
Explained answer. No. The constructed texts come from three writers and may share institutional conditions.
The common shortcut. Tokens are not people or independent speakers.
Exercise 8: Source-sensitive conclusion
Question. Write a cautious finding about the toy data.
Explained answer. ‘Twenty-six target hits in 1,200 tokens came from twelve fictional documents across three writer groups; one group contributed twenty hits.’
The common shortcut. The toy sample cannot establish how real schools or populations use the word.
Frequently asked questions: corpus source clustering
What is corpus source clustering?
Grouping related source records by a defined shared origin or process such as author, institution, template or event.
Is clustering the same as deduplication?
No. Related original texts can share a writer without being identical copies.
Why can documents be statistically dependent?
They may share writer habits, topics, templates, institutions or collection conditions.
How does document count differ from author count?
Several documents can belong to one author; the two numbers answer different questions.
What is pseudo-replication?
Treating related observations as though they were independent units, potentially overstating evidence.
Can a text belong to more than one cluster?
Yes. Joint authorship, publication lineage and templates can create overlapping relationships.
Does a large token total guarantee many independent voices?
No. One author can produce millions of tokens.
What is a cluster bootstrap?
Resampling entire groups under a design where those groups are relevant independent source units.
Should you always resample authors?
No. The appropriate level depends on the selection process and the target inference.
Why report per-writer rates?
They reveal concentration and variation hidden by a pooled word rate.
Are repeated notices always bad data?
No. For reader exposure they can be real events; for independent wording they may be duplicates.
How should uncertain author IDs be treated?
Record uncertainty rather than inventing identities or assuming independence.
Can source metadata be private?
Yes. Use access controls and anonymous group IDs where appropriate.
What is the safest conclusion from the toy example?
The invented corpus contains twelve texts, 1,200 tokens and three writer groups with concentrated target hits; it measures no real population.
Short glossary
- Source cluster: A group of related text records under a declared criterion.
- Document: A source text unit, not necessarily an independent writer.
- Author: A person or role associated with producing source language.
- Template: Recurring shared text structure reused across documents.
- Deduplication: Handling repeated or copied content under an identity rule.
- Provenance: Recorded origin or lineage of a text.
- Pseudo-replication: Treating dependent observations as independent evidence.
- Nested units: Data organised in multiple levels such as words within documents within writers.
- Crossed membership: Source records linked to multiple groups or factors.
- Pooled rate: Total occurrences divided by total eligible token volume.
- Cluster-aware inference: An uncertainty method accounting for dependence within source groups.
- Writer prevalence: How many distinct eligible writers use a target.
A four-part learning and checking routine
Start with the communication question. Decide what would count as a useful answer before opening a word-frequency report or an automated label table. This is where a learner can move from memorising vocabulary to understanding how evidence supports an interpretation.
Next, reconstruct the smallest complete example. Write the original source item, the category or counted feature, and the output. If another reader cannot independently recover the result, clarify the method before scaling it up. A transparent three-row example often teaches more than a complicated uncheckable chart.
Then vary one condition deliberately. Change the number of source units, the classification rule or the sampling frame, while keeping the reason for each change explicit. Ask which conclusions survive and which need narrower wording. This builds the habit of treating language data as observations under defined conditions.
Finally, transfer the idea to a genuine subject task: reading a History source, comparing Scientific descriptions, or selecting a precise English word. Test comprehension in a new example after a delay. The learning outcome is the ability to make and explain a better decision, not simply recognising an advanced term.
Research and methodological sources
- Gries (2022), Toward more careful corpus statistics: uncertainty estimates for frequencies, dispersions and association measures
- Cambridge University Press, Designing and Evaluating Language Corpora — Domain considerations
- eduKateSG, Document-Level Bootstrapping
- eduKateSG, Corpus Deduplication
These references support the methodological distinctions. The numerical school examples are not results reported by the cited sources.
Connected vocabulary methods and hubs
Read the next four vocabulary evidence guides together
These four guides answer connected but different questions: what went wrong with a word choice, whether a replacement preserves meaning, which annotation mistakes recur, and whether multiple texts come from independent language sources.
- Vocabulary Error Taxonomy — Identify meaning, collocation, form and register errors before repairing a sentence
- Lexical Substitution Test — Test whether a proposed synonym works in the complete sentence
- Annotation Confusion Matrix — Read noun, verb and adjective label errors with per-class metrics
- Vocabulary Learning Hub — follow the main learning pathways
- Vocabulary Article Library — continue into related concepts and examples