Sometimes the smallest version of a thing is not a smaller copy of it. It is a recipe for rebuilding it.
Write the letter A ten thousand times and you have ten thousand characters. Write “A repeated 10,000 times” and you have something dramatically shorter.
Yet nothing essential has been lost if the instruction is unambiguous. The receiver can recreate every A in exactly the right place.
That simple trick opens a much larger door. Compression is often not about squeezing the original object. It is about discovering a description that generates the original more economically.
Quick Read
- Repeated structure creates opportunities for shorter descriptions.
- Run-length coding replaces repeated symbols with symbol-plus-count descriptions.
- Dictionaries replace recurring sequences with references.
- Predictive methods describe what changed rather than restating everything.
- Grammar-like and procedural representations can describe large structures using compact rules.
- The best representation depends on which patterns actually occur.
- A short description is useful only if the receiver can interpret and reconstruct it.
The One-Sentence Answer
Patterns create compression because a repeated or predictable structure can often be described once as a rule, reference or difference instead of being independently restated every time it appears.
Literal Description Is Expensive
Imagine giving someone directions through a corridor containing 100 identical doors.
You could say:
door, door, door, door, door…
Or you could say:
100 identical doors.
The second description gains its economy from regularity. The moment every door becomes different, the shortcut weakens. You may need to describe the differences too.
This is the central relationship:
repetition + shared rule → fewer independent descriptions
Run-Length Encoding: The Child’s First Compressor
Suppose a black-and-white row of pixels looks like this:
WWWWWWWWBBBBBWWWW
Instead of recording every pixel separately, a run-length representation might describe eight whites, five blacks and four whites. When long runs occur, the representation can be substantially smaller.
But notice the condition: runs must actually exist. Apply the same idea to an alternating pattern such as WBWBWBWB and the bookkeeping may be worse than the original.
A compression method is therefore a bet about structure.
The Dictionary Trick
Now imagine a document repeatedly uses the phrase “Secondary English examination”. Instead of storing those characters afresh every time, a compressor can notice recurrence, keep a useful sequence in a dictionary or sliding history, and later represent another occurrence by referring to what is already known.
This family of ideas sits behind enormously influential dictionary-based methods. The exact engineering varies, but the intuition is wonderfully human: we have seen this before; point to it instead of saying all of it again.
References turn repetition across space or time into economy.
Prediction: Describe the Mistake, Not the Whole Answer
Suppose a sequence is:
1000, 1001, 1002, 1003, 1004, 1005
Once the pattern is visible, repeating every four-digit number feels wasteful. A model could predict the next value and store only the difference between prediction and reality.
This is a profound compression idea. Instead of encoding the world directly, build a model of the world and encode the model’s errors.
actual value = prediction + residual
If predictions are good, residuals tend to be small or statistically concentrated, which can make them cheaper to encode.
Suddenly compression begins to resemble intelligence: notice regularity, form an expectation, and spend information mainly on surprise.
A Map Is Not a Smaller Landscape
Think about a map. It does not shrink every tree, kerb, drain, doorway and blade of grass until Singapore fits onto paper. It chooses a representational language: lines for roads, shapes for buildings, symbols for stations, colours for categories, labels for names.
That is not necessarily lossless compression of the physical world; much is deliberately omitted. But it teaches an important representational lesson: economy often comes from changing the description language rather than physically miniaturising the described thing.
Digital lossless compression performs a stricter version of the same intellectual move. It seeks another description from which the required original can be recovered exactly.
The Recipe Can Be Smaller Than the Meal
A fractal gives us a dramatic visual intuition. A compact iterative rule can generate structures containing detail across many scales. Procedural graphics can generate forests, textures, terrains and patterns from algorithms and parameters rather than storing every final pixel or object independently.
The rule is not a physically squeezed copy of the output. It is generative.
This is why the phrase “shorter description” is more powerful than “smaller file.” It lets us see compression as a relationship between an object, a descriptive language and a reconstruction procedure.
But Which Language?
Here is where things become subtle.
“A checkerboard” is an extraordinarily short description if both of us know what checkerboards are. Without that concept, I may need to explain alternating squares, rows, columns, colours and dimensions.
A description can be short because the descriptive language already carries machinery.
This does not invalidate the compression. It reminds us to count honestly. Codebooks, algorithms, conventions, models and decoder software are part of the wider system even when they are not repeated inside every payload.
Kolmogorov’s Beautiful Question
Algorithmic information theory asks a wonderfully clean question: what is the length of the shortest effective description, often formalised as a program for a fixed universal computational framework, that can produce a particular string?
A million zeros has an obvious short generating description. A sufficiently irregular million-bit string may not possess any comparably short description.
The exact shortest-program quantity is not something a general algorithm can simply calculate for every arbitrary string. That limitation matters. “Find the ultimate compression” is not merely an engineering problem waiting for faster computers.
Compression Depends on What You Notice
Suppose I show three people the same data.
- One notices repeated characters.
- One notices repeated words.
- One notices that entire paragraphs follow a reusable template.
Each person has discovered structure at a different level.
This suggests an important principle: representation determines visible redundancy.
Patterns invisible in one coordinate system or description may become obvious in another. Audio transformed into a frequency representation exposes relationships that are awkward to see in raw time samples. Images transformed into other bases can concentrate important structure differently. Text tokenised into recurring units reveals patterns not apparent if we insist on treating every byte independently.
Sometimes better compression begins before coding. It begins by looking at the object differently.
Human Language Is Full of Compressed Handles
Consider the sentence:
The teacher scaffolded the task.
To an education researcher, “scaffolded” can unfold into temporary support, task decomposition, prompts, modelling, guided practice and eventual withdrawal of assistance. One word points into a conceptual network.
Language constantly trades explicit detail for shared models.
This is why jargon can be wonderfully efficient inside a knowledgeable community and disastrously opaque outside it. The short description only works for a compatible decoder.
Learning Is Partly the Construction of Better Codebooks
A Primary student may experience “3 × 8” as repeated addition. Later, multiplication becomes a compact operation with properties, relationships and automatic associations. In algebra, “factorise” eventually activates a repertoire of patterns. In chemistry, a formula becomes more than letters and subscripts. In literature, “dramatic irony” compresses an entire relationship between audience knowledge, character knowledge and narrative effect.
Education builds representations that let learners hold larger structures with fewer conscious pieces.
But compression should follow understanding, not replace it. Memorising a label without constructing the structure behind it produces the appearance of compact knowledge without the decoder required to unfold it.
Primary School: Find the Rule
Give a learner:
2, 4, 6, 8, 10, 12, 14, 16
Then ask for a shorter way to describe the sequence. “Even numbers from 2 to 16” is not the sequence itself. It is a rule selecting the sequence.
Children can then compare patterned and unpatterned lists and discover that not every collection yields an equally neat rule.
Secondary School: Model, Then Encode the Difference
At Secondary level, show a nearly linear series:
100, 110, 120, 131, 140, 150
A simple model predicts increases of 10. Most values fit perfectly; one differs by 1. Instead of treating six values as unrelated facts, we can describe the rule and the exception.
That is compression, modelling and scientific thinking meeting in one small exercise.
JC and Beyond: Compression as Model Selection
At a more advanced level, compression connects naturally with statistical modelling. A model that captures genuine regularity can reduce the unexplained residual. A model that merely memorises every observation may describe the training data perfectly while providing little useful simplification or generalisation.
This is one reason description length has appeared as a powerful way to think about inference: useful explanations balance the complexity of the model against the information still needed to describe the data given that model.
Too simple a model leaves a large residue. Too elaborate a model becomes an expensive description in its own right.
When the Shortcut Stops Working
- The assumed pattern may be false.
- The dictionary may cost more than it saves.
- The data may be too small for setup costs to pay off.
- The receiver may lack the required codebook.
-
Explore the How Compression Works.