“Same as yesterday, except number 17 changed.”
That sentence might replace an enormous document.
But there is a catch so obvious that we often fail to see it: you need yesterday.
If yesterday’s document has vanished, our wonderfully compact message is almost useless. The missing information was never inside those six words. It lived somewhere else.
This gives us one of the most important ways to think about compression: sometimes the payload becomes small because the receiver, the environment or the past is carrying part of the description.
Quick Read
- A compressed representation may depend on information outside the payload.
- Shared dictionaries let sender and receiver avoid retransmitting familiar material.
- Delta representations describe changes relative to a known reference state.
- Caches turn earlier transmission into later savings.
- Standards and conventions act like shared codebooks.
- Human language is highly economical because receivers contribute prior knowledge.
- If the assumed context is absent or mismatched, compact communication can fail catastrophically.
- To measure compression honestly, ask what the entire reconstruction system must possess.
The One-Sentence Answer
Some representations become extremely small not because all the original information has vanished, but because part of what is needed for reconstruction has been relocated into shared context, prior state, dictionaries, standards or the receiver’s existing knowledge.
The Message Is Not Always the Whole System
Suppose two architects have worked together for twenty years. One sends the other:
Use the usual stair detail.
That message is tiny. Yet it may determine dimensions, materials, conventions and drawing details that would require pages to explain to an outsider.
The economy does not come from packing those pages microscopically into the sentence. It comes from shared state.
small message + shared context → large reconstruction
Remove the context and the equation breaks.
Shared Dictionaries
Imagine a website repeatedly sends the same frameworks, scripts, styles or textual fragments. If client and server both possess an agreed dictionary containing common material, later messages can refer to dictionary entries instead of retransmitting the underlying sequences in full.
This principle is not hypothetical. Modern web standards include mechanisms for compression dictionaries in which previously available resources can contribute dictionary material for later compression and decompression.
The transferred payload may become much smaller. But an honest accounting includes the dictionary. The bytes have not been defeated by philosophy; reusable information has been amortised across multiple exchanges.
Delta: Tell Me What Changed
Version 1 of a document contains 100,000 characters. Version 2 differs in three sentences.
One strategy is to send all 100,000 characters again.
Another is to say, in a suitably precise machine-readable form: keep the old version; remove this region; insert these new sequences.
Now the new state is represented relative to an old state.
new state = reference state + difference
This is extraordinarily powerful when change is small. It is also extraordinarily dependent on having the correct reference. Apply a patch to the wrong base and reconstruction can fail.
Caches Are Memory Turned Into Bandwidth
When your device caches something, it spends local storage so that a future request may avoid another full transfer. Yesterday’s bandwidth becomes today’s local memory; today’s memory becomes tomorrow’s saved bandwidth.
This is not compression in the narrowest algorithmic sense, but it reveals the same systems-level exchange. Information costs can move between locations and moments.
A system designer therefore asks not simply, “How many bytes crossed the network this time?” but also:
- What was stored earlier?
- Who paid to obtain it?
- How long must it remain valid?
- What happens when versions diverge?
- What is the recovery path after cache loss?
Standards Are Giant Invisible Dictionaries
Consider a JPEG file, a PDF, an HTML document or a Unicode text file. The file does not need to redefine the entire relevant standard every time it is opened. Decoder software already implements conventions governing how structures are interpreted.
This is one of civilisation’s great tricks: establish shared rules once, then communicate economically within them.
Traffic lights work similarly. A red circle of light does not contain a paragraph explaining legal obligations, road conventions, expected driver behaviour and the history of signalling. Society supplies the decoder.
Language Is Astonishingly Dependent on Borrowed Space
Imagine I say:
The MRT was packed after the concert.
A Singapore reader may instantly supply a network of unstated information: what the MRT is, what a station looks like, why crowds form after events, how trains constrain passenger flow, what “packed” means in context and perhaps even how the experience feels.
The sentence is short because the reader does work.
Human communication is therefore not a process in which speakers transmit complete miniature worlds. We transmit cues into partially shared worlds.
Expertise Is a Large Decoder
A doctor, engineer, mathematician, musician and lawyer can each read compact representations that are nearly meaningless to an outsider.
A chemical formula may be only a handful of symbols. A musical chord symbol may occupy three characters. A mathematical theorem may invoke an established result in one line. A legal citation can point toward pages of doctrine.
The expert is not necessarily receiving more characters. The expert possesses more machinery for expanding them.
This is why expertise can look like speed. The visible message is small because years of invisible preparation sit behind the decoder.
The Teacher’s Compression Error
Now we arrive at education.
A teacher says:
Just compare the themes.
The teacher may unconsciously unfold “compare” into identify a common dimension, retrieve evidence, establish similarities, distinguish differences, judge significance and organise the response.
The student may hear only two words.
This is a receiver mismatch. The instruction was compressed for a decoder the learner does not yet possess.
Good teaching does something counterintuitive: it deliberately makes expert communication longer. It decompresses hidden steps until the learner can build them internally. Only later should those steps become compact again.
Context Can Become a Single Point of Failure
Borrowed space creates efficiency, but also dependency.
- If the dictionary is missing, decoding fails.
- If sender and receiver use different versions, references may point to the wrong material.
- If a cache is corrupted, dependent objects may become unusable.
- If cultural assumptions differ, a compact human message can be misunderstood.
- If institutional knowledge leaves with experienced staff, shorthand that once worked may become opaque.
Compression therefore has a resilience question: how much external state are we willing to depend upon?
The Tiny URL and the Giant Database
A short identifier can appear to represent an enormous object. A database ID such as 58372 may retrieve a complete customer record, image, article or model.
But the identifier has not compressed that object in the ordinary lossless sense. It is an address. The large object remains somewhere else.
This distinction is essential whenever someone claims extraordinary information density. A pointer to information is not the same as containing the information.
“The Library of Congress” is a short phrase. It does not fit the library inside twenty characters.
Does a Seed Contain a Tree?
This tempting analogy needs care.
A seed contains genetic and cellular machinery that participates in development, but a tree is not reconstructed from the seed alone. Growth requires matter, energy, water, environmental conditions and immensely complex biological interactions. The seed is not a ZIP archive containing every atom of the future tree.
The analogy is useful precisely when we notice the external dependencies. Compact starting states can generate large outcomes because the environment supplies resources and lawful dynamics.
Primary School: What Must the Other Person Know?
Tell a child, “Draw the Singapore flag.” Then ask what knowledge was hidden inside that short instruction. The learner must know what Singapore is, what its flag looks like, which colours and shapes matter, and how to draw them.
Now tell someone unfamiliar with the flag the same sentence. Suddenly the instruction is insufficient.
The exercise reveals that message length and communicated meaning are not the same thing.
Secondary School: Reference States
Give students two nearly identical paragraphs and ask them to describe paragraph B using paragraph A plus changes. Then remove paragraph A and ask whether their compact description still works.
They will discover the cost of delta representation immediately.
JC and Beyond: Count the Whole Decoder
At higher levels, ask a harder systems question: when comparing two compression methods, what belongs inside the accounting boundary?
- compressed payload;
- dictionary;
- model parameters;
- decoder implementation;
- reference corpus;
- cached state;
- network lookup;
- computation;
- failure recovery.
Different applications legitimately draw that boundary differently. A decoder installed once and used a billion times can amortise its cost. But we should not pretend the decoder does not exist simply because we chose not to count it in each file.
The Receiver Determines Whether Compression Worked
A compressed object sitting alone is only potential. Successful compression is relational:
representation + compatible decoder + required context → usable reconstruction
That equation is useful for files, classrooms, organisations and cultures.
A perfectly efficient message for the wrong receiver can be functionally worthless.
The Hidden Economics of Shared Knowledge
Shared context is expensive to build but cheap to reuse.
A profession spends years creating standards, terminology, textbooks, training, software and conventions. Once established, experts communicate rapidly because they no longer need to restate the foundation during