A pattern can disappear when you stand too close to it.
Look at one tile in a mosaic and you see colour. Look at a hundred tiles and you see a flower. Look at ten thousand and the flower becomes part of a repeating border. The object has not changed. Your scale has.
Compression has the same problem. Regularity is not always visible at one resolution. Some patterns are local. Some are global. Some repeat inside larger patterns. Some only become obvious after we move from raw detail to a higher-level representation.
Quick Read
- Regularity can exist at many scales at once.
- Local models capture nearby dependencies; global models capture longer-range structure.
- Hierarchical representations let one rule describe repeated structure at several levels.
- Too narrow a view misses distant relationships; too broad a view can blur useful local detail.
- Images, music, language, genomes, cities and learning all exhibit multiscale structure.
- A good compressor often changes representation so that structure becomes concentrated at the scale where it is easiest to describe.
- Scale selection is therefore part of the modelling problem.
The One-Sentence Answer
Compression improves when we observe information at a scale that exposes its regularities, because the same data can appear noisy locally yet highly structured globally—or the reverse.
The Same Sequence, Three Views
Imagine a long sequence built from the motif ABCD repeated one thousand times.
At one-character scale, the stream contains four symbols in rotation. At four-character scale, it contains one repeating motif. At the level of the whole sequence, the description becomes:
Repeat ABCD one thousand times.
Each scale tells the truth. But one scale gives us a much shorter description.
Local Regularity
Local regularity is structure among nearby values.
In an image, neighbouring pixels often have similar colour. In audio, adjacent samples are related. In text, nearby letters and words constrain one another. In a time series, the next value may resemble the recent past.
Local models are attractive because they are fast and memory-efficient. They can often make good predictions with a small context.
But local regularity is not the whole story.
Global Regularity
A novel may repeat a theme hundreds of pages later. A piece of music may return to its opening melody after a long development. A genome may contain duplicated regions separated by great distances. A database may repeat the same schema across millions of records.
These are long-range relationships. A compressor with too small a window may never see them.
Expanding context can reveal them, but larger context costs memory, search and computation. The problem becomes architectural: how much of the past is worth remembering?
Hierarchy: Patterns Made of Patterns
Many real objects are hierarchical.
A book contains chapters, chapters contain sections, sections contain paragraphs, paragraphs contain sentences, sentences contain phrases and words. A city contains districts, roads, buildings, rooms and objects. A program contains modules, functions, statements and tokens.
Compression can exploit repetition at each level.
If every chapter follows a similar template, we can describe the template once. If paragraphs reuse sentence structures, we can exploit that too. If words repeat, another layer of compression appears.
The shortest useful description may therefore be hierarchical rather than flat.
Images: Zoom Changes the Statistics
Take a photograph of a brick wall.
At pixel scale, you see brightness and colour variation. At brick scale, rectangular patterns repeat. At wall scale, rows repeat. At architectural scale, the wall may be one component in a larger façade.
A transform can reorganise image information so that broad smooth structure and fine detail occupy different components. The representation then separates scales that were mixed together in raw pixels.
Even in strictly lossless settings, multiresolution representations can make correlations easier to model.
Music: Notes, Motifs, Phrases, Movements
A listener can hear individual notes. A musician hears motifs. A composer hears phrases, harmonic progressions and larger form.
The same sequence therefore supports descriptions at multiple scales.
“C, E, G” describes three notes. “C major triad” compresses them using a shared musical concept. “Return of the opening theme” can compress dozens of bars into a relationship with something heard earlier.
Expertise often means seeing larger units.
Language: Letters Are Not the Largest Pattern
Text compressors can exploit recurring characters, words and phrases. But language also contains document-level structure.
News articles often follow recurring rhetorical patterns. Scientific papers repeat section structures. Legal documents reuse clauses. School essays use recognisable introductions, evidence patterns and conclusions.
A compressor operating only at character scale misses much of this higher-order regularity.
A human reader does not. We continuously move between local syntax and global discourse.
The Danger of Looking Too Far Away
Bigger context is not automatically better.
A global model may waste resources tracking distant information that is irrelevant to the current prediction. It may confuse separate regimes. It may become slow, memory-hungry or brittle.
Good models allocate attention selectively. They preserve enough local detail while recognising useful long-range relationships.
Compression therefore needs both zoom and restraint.
Scale and Noise
A remarkable thing happens when we aggregate.
Individual measurements can be noisy. A larger-scale average may reveal a stable trend. Yet aggregation can also destroy meaningful variation.
For compression, this creates a choice: which variation is structure and which is residual?
In lossless compression, we cannot simply discard the residual. But we can model the broad trend separately and encode the fine deviations after it.
coarse structure + fine residual = exact reconstruction
Cities Are Multiscale Information Systems
A street map of Singapore contains different truths at different zoom levels.
At island scale, expressways and major regions matter. At town scale, arterial roads and MRT stations appear. At neighbourhood scale, blocks, paths and entrances matter. At room scale, none of those maps is sufficient.
Trying to put every detail at every scale onto one map creates clutter.
Maps therefore use multiscale representation: show the information appropriate to the current zoom. That is not lossless compression of the physical world, but it gives us a powerful mental model for hierarchical information design.
Genomes: Repetition Across Vast Distances
Genomic data contains local motifs and large repeated structures. Some repeats are short. Others span thousands or millions of bases.
A compressor that only sees a tiny local window may exploit base-level statistics while missing long duplicated regions. A larger-scale index can reveal distant similarity.
The challenge is to capture multiple scales without making search prohibitively expensive.
Scientific Models Change With Scale
Physics offers another analogy. We do not describe a falling ball by tracking every quark. We choose a scale where mass, velocity and forces are useful variables.
A fluid can be described as a continuum at one scale, molecules at another and quantum interactions at another.
Each representation compresses away irrelevant microscopic description for a particular question—though that is modelling rather than literal lossless compression.
The lesson transfers: the right variables often appear only at the right scale.
Education: Expertise Changes the Zoom Level
A beginner sees separate facts. An expert sees a system.
In algebra, the novice sees individual symbols. The expert sees factor patterns. In literature, the novice sees sentences. The expert sees narrative structure. In biology, the novice sees terms. The expert sees interacting systems.
Expert knowledge compresses many details into larger meaningful chunks without necessarily losing access to the details.
That ability—to zoom out without becoming vague, then zoom back in when evidence is needed—is one of the signatures of strong understanding.
Primary School: Zoom Out to Find the Rule
Show a child a long pattern of coloured shapes. Up close, ask them to name each shape. Then step back and ask whether a larger repeating unit appears.
The exercise teaches that patterns can live above the level of individual items.
Secondary School: Local Trend, Global Trend
Give students a noisy graph. Ask what happens between adjacent points, then ask what happens across the entire series.
They may discover local fluctuations sitting on top of a global trend.
This is the same conceptual move used in compression: separate broad structure from finer residual detail.
JC and Beyond: Multiresolution Thinking
At higher levels, wavelets, pyramids, hierarchical models and multiresolution analyses formalise the idea that information can be represented across scales.
These tools are useful because many natural signals have structure that is neither purely local nor purely global. A good representation lets us describe coarse behaviour and then refine it with progressively finer information.
Lossless systems can preserve every refinement while still exploiting the fact that coarse structure predicts part of the fine structure.
Scale Is Also About Time
A traffic sensor changes second by second, hour by hour, day by day and season by season.
A model at one-second scale sees noise and immediate flow. At daily scale, commuting rhythms appear. At annual scale, holidays and school calendars matter.
Compressing temporal data well may require recognising periodicities at several horizons.
The Best Scale Depends on the Question
There is no magical zoom level where all structure becomes visible.
- Too close: global repetition disappears.
- Too far: local exceptions disappear into averages.
- Only one scale: nested structure is missed.
- Every scale at once: the model may become too expensive.
Good compression therefore uses the scale that earns its cost.
A Useful Scale Checklist
- What patterns exist among neighbouring values?
- What patterns recur over long distances?
- Is there hierarchy?
- Can coarse structure predict fine structure?
- How much context can the system afford?
- Does changing scale make the residual simpler?
- Can the decoder reproduce the same hierarchy?
The Deeper Point
Compression is sometimes described as discovering repetition.
But repetition has a scale.
A forest contains repeated leaves, repeated branching structures, repeated trees and repeated ecological relationships. Which repetition matters depends on how far away we stand.
The same is true of information.
Sometimes the pattern was always there. We were simply looking at the wrong zoom.