Compression begins before the first bit is saved. It begins when we decide where one thing ends and another begins.
Take the string ABABABABABABABAB. If we look at it one character at a time, we see A, then B, then A, then B. If we look at it two characters at a time, we see AB repeated eight times. If we look at it as one uninterrupted block, we may discover an even shorter rule: repeat AB eight times.
The data did not change. The boundary did.
This is a deceptively large idea. Compression depends not only on what information exists, but on how we segment it into symbols, tokens, blocks, frames, chunks, phrases and contexts. A poor boundary can hide structure. A good boundary can make structure obvious.
Quick Read
- Compression operates on units, and somebody has to define those units.
- Different segmentations expose different repetitions and dependencies.
- Tokenisation can make recurring words or phrases cheap to represent.
- Block size trades compression opportunity against access, memory and resilience.
- Boundaries that are too small miss long-range structure; boundaries that are too large may become expensive to search or decode.
- Natural language, images, audio, genomes and databases all have boundary problems.
- There is rarely one universally best segmentation because the useful boundary depends on the task and receiver.
The One-Sentence Answer
Where we cut information determines which regularities become visible, which dependencies can be exploited and what costs appear elsewhere, so segmentation is part of compression rather than a neutral preprocessing step.
A Boundary Creates a Unit
Suppose we are compressing English text. We could treat each byte as a symbol. Or each character. Or each word. Or each recurring phrase. Or perhaps larger learned tokens that sometimes align with words and sometimes do not.
Each choice changes the alphabet the compressor sees.
At the character level, “information” is eleven letters. At a word-token level, it may be one unit. At a phrase level, “information theory” may be treated as a recurring pair. At a document level, whole templates may repeat.
The representation decides what counts as repetition.
Too Small and You Miss the Pattern
Imagine a tiled floor made from the repeating motif:
red-blue-green-yellow
If we inspect isolated tiles, the sequence may look like four unrelated colours. If we group every four tiles, the motif appears instantly.
Small units are not wrong. They simply may fail to expose a higher-order regularity.
This is one reason compressors often search across windows or dictionaries rather than considering only one symbol at a time. Useful structure can span many local units.
Too Large and You Pay for the Search
Now swing too far in the other direction.
If we treat an entire gigabyte file as one indivisible unit, then identifying reusable substructure becomes awkward. The unit is so large that local repetition disappears inside it.
Large contexts can also require more memory, more search time and more complex models. They may improve compression by exposing long-range dependencies, but the extra context must be managed.
The boundary problem is therefore not “smaller is better” or “larger is better.” It is a search for the scale at which useful regularity becomes economical to exploit.
Words Are Human Compression Boundaries
Look at the phrase:
the theory of evolution by natural selection
A reader does not normally process this as a flat stream of letters. We recognise words, then phrases, then a concept. Boundaries let the mind build larger chunks.
That chunking is partly compressive. Instead of treating every letter as a fresh decision, the reader activates familiar lexical and conceptual units.
But boundaries are language-dependent and context-dependent. Chinese writing does not mark word boundaries with spaces in the same way English does. Compound words can be written together, apart or hyphenated. Technical vocabulary creates units unknown to a beginner.
Segmentation is not simply given by nature. It is often a modelling choice.
Tokenisation Makes This Visible
Modern language-processing systems often break text into tokens. A token might correspond to a whole common word, part of a rarer word, punctuation or another recurring unit.
Why not use one token for every complete word?
Because the vocabulary would become enormous and rare or novel words would be problematic. Why not use only characters? Because many useful recurring sequences would require longer token chains.
Tokenisation therefore negotiates between vocabulary size and sequence length. That is a compression-like trade-off: a larger dictionary can shorten many messages, but the dictionary itself becomes larger and less universal.
Block Boundaries Can Hide Repetition
Suppose a phrase begins near the end of one compression block and finishes at the start of the next. If blocks are compressed independently, the compressor may fail to notice that the phrase repeats elsewhere as one unit.
Shift the boundary and suddenly the whole phrase sits inside a single block, where it can be recognised.
This is why block design matters. Fixed-size boundaries are simple, but content-defined chunking can sometimes identify boundaries that better survive insertions or capture recurring regions.
The exact techniques vary, but the principle is stable: boundary placement affects duplicate detection.
A Small Insertion Can Move Every Fixed Boundary
Imagine two almost identical files. File B has one extra byte near the beginning.
With rigid fixed-size chunks, that one insertion can shift every subsequent boundary. Regions that are otherwise identical may now be partitioned differently, so naive block-by-block comparison misses the similarity.
Content-defined chunking tries to make boundaries depend on the content itself, so unchanged regions are more likely to retain similar chunk boundaries despite insertions or deletions.
This matters in backup systems, deduplication and version storage because good boundaries can turn “almost the same file” into “mostly the same chunks.”
Images Have Boundaries Too
An image can be processed pixel by pixel, row by row, tile by tile, block by block, region by region or through transformed coefficients. Different boundaries reveal different structure.
A smooth sky may be highly predictable locally. A repeating building façade may reveal patterns across larger regions. An edge cuts across neighbourhoods and changes which pixels belong to the same visual regime.
When an image-processing method imposes block boundaries, those boundaries can even become visible under strong lossy compression as block artefacts. The representation’s segmentation leaves fingerprints on the reconstructed world.
Here we remain on the lossless conceptual side: boundaries influence what relationships the compressor can see, regardless of whether any later system chooses to discard information.
Audio Is Segmented in Time
Audio is continuous in experience but discretised in representation. Systems analyse frames, windows or blocks of samples. Window length changes what temporal and frequency structure becomes visible.
Short windows track rapid change better. Long windows provide finer frequency resolution for stable signals. Again, where we cut time changes the structure we can exploit.
Genomes and the Boundary Problem
Genomic sequences contain repeats at many scales: single bases, motifs, genes, transposable elements, duplicated regions and larger structural patterns.
A representation that sees only local bases may miss long duplicated regions. One that operates only on enormous segments may miss smaller motifs.
Compression therefore becomes a multiscale segmentation problem: which units reveal biologically and statistically reusable structure?
Databases Choose Boundaries Around Workloads
A database can store rows together, columns together, pages together or partitions together. Compression behaves differently under each arrangement.
Columnar storage often places similar data types and values near one another, which can expose repetition and predictable distributions. Row-oriented storage keeps complete records together, which may suit different access patterns.
Physical layout is therefore not merely about disk placement. It changes statistical neighbourhoods.
Education: The Wrong Boundary Creates the Wrong Lesson
Students also struggle when knowledge is cut at the wrong places.
A child may memorise “photosynthesis” as one word without seeing the process inside it. Another may memorise every step separately without seeing that the steps form a larger energy-conversion system.
Good teaching moves between boundaries. Zoom in when mechanisms matter. Zoom out when relationships matter.
Learning improves when the student can flexibly regroup information rather than being trapped by the first segmentation presented.
Primary School: Different Ways to Group
Give a child twelve counters arranged as red-blue-red-blue-red-blue-red-blue-red-blue-red-blue.
Ask them to describe the arrangement one counter at a time, then in pairs, then as a repeating rule.
The physical counters do not change. The description gets shorter because the grouping reveals repetition.
Secondary School: Paragraph, Sentence, Phrase, Word
Take a paragraph and ask students to identify repeated ideas at several levels. Which words recur? Which sentence structures recur? Which argumentative moves recur?
They will discover that pattern recognition depends on the size of the unit being observed.
JC and Beyond: Segmentation as Model Selection
At a more advanced level, boundary choice can itself be treated as a modelling problem. A segmentation is good when it exposes regularities that produce a compact, useful description without imposing excessive overhead.
We can think of the cost as involving at least three pieces:
cost of boundaries + cost of units + cost of residual irregularity
Too many boundaries increase bookkeeping. Too few can hide structure. The optimum depends on the source and task.
The Boundary Is Part of the Decoder Contract
The decoder must know how the stream was segmented or possess rules for recovering the segmentation. Otherwise a sequence of compact codes may be ambiguous.
Self-delimiting codes, length fields, markers, grammars and framing structures all solve versions of the same problem: where does one encoded object stop so the next can begin?
Boundaries cost information because boundaries are information.
The Deeper Point
We often imagine that data arrives already divided into its “natural” parts.
Sometimes it does. Often it does not.
A compressor must decide what counts as a symbol, a token, a block, a context or a reusable object. That decision changes which redundancies become visible and which costs appear later.
So before asking, “How can we compress this?” ask a quieter question:
What exactly have we decided this thing is made of?
Compression often starts there.
Explore the How Compression Works.