Version 41 is 50 MB.
Version 42 is also 50 MB.
Only one paragraph changed.
Why pay another 50 MB to remember the edit?
Delta encoding stores a related object as a base plus instructions describing what changed.
This is the second pillar beneath How Compression Works | The Corpus. The Corpus owns family-level redundancy. Delta Encoding owns temporal and relational difference: how one member can borrow most of its description from another.
Quick Read
Delta encoding represents a target object relative to a base object. Instead of storing all target bytes independently, the encoder records reusable regions from the base plus inserted or changed data. This can make version histories dramatically smaller when successive objects are similar. The trade-off is dependency: reconstruction requires the right base, deep delta chains add CPU and latency, corruption can propagate through descendants, and choosing a bad base can make the delta larger than storing the object normally.
base object + delta instructions → target object
A Delta Is Not “The New File Minus the Old File” in Ordinary Arithmetic
Files are structured byte sequences.
A practical delta can contain instructions such as:
- copy this region from the base;
- insert these new bytes;
- skip this old region;
- copy another matching region.
The exact instruction language depends on the system.
The principle is stable: describe similarity by reference and spend new bytes mainly on difference.
The Base Is Side Information With an Address
A tiny delta is useless by itself if the decoder cannot obtain the base it refers to.
So the real compressed object is:
delta + base identity + base availability.
This is a concrete instance of Side Information: a shorter message is possible because the receiver already has—or can retrieve—something else.
The Best Base Is Not Always the Previous Version
Version 20 may resemble version 17 more than version 19.
A renamed source file may resemble an older object in another path.
A branch may share more structure with its common ancestor than with the newest object elsewhere.
Base selection is therefore a search problem inside the corpus.
Git Makes the Trade-Off Visible
Git packfiles can store an object as a delta against another base object. Current Git documentation explains that pack creation compares objects for delta-compression opportunities and that maximum delta depth matters because deeper chains require more work on the unpacking side.
That is the central systems trade-off in one sentence:
storage saved now can become reconstruction work later.
Delta Depth Is Borrowed Time
Object B is a delta from A.
C is a delta from B.
D is a delta from C.
To reconstruct D, a decoder may need to recover the chain back through several bases.
Borrowed Time owns the larger principle: saving space can push work into the future.
Deep Chains Can Turn Random Access Into Archaeology
You want one old object.
The system must fetch four ancestors, decompress them and apply several deltas.
The stored representation is compact.
The access path is not local anymore.
Locality owns the broader problem of making one small thing harder to reach while making the whole corpus smaller.
Snapshots Break Long Chains
A system can occasionally store a full object rather than another delta.
Now reconstruction can restart from a nearer full base.
This trades extra storage for bounded access cost.
A snapshot is therefore not wasted redundancy when it prevents pathological dependency depth.
Rebasing Changes the Dependency Graph
Suppose several deltas depend on a poor or fragile base.
The system can rebuild them against a better base.
Storage may shrink.
Access may improve.
But rewriting dependency relationships costs compute and creates a new version of the storage structure.
A Delta Chain Is a Graph, Not Just a File Format
Objects become nodes.
Delta dependencies become edges.
Now storage design can ask graph questions:
- Which bases have many descendants?
- Which edges are expensive?
- Which nodes deserve stronger protection?
- Where should full snapshots be inserted?
The corpus becomes structurally aware of ancestry.
Corruption Can Propagate Through Dependency
Lose one independent file and one file is damaged.
Lose a base required by fifty deltas and fifty logical objects may become unreconstructable.
Shared ancestry creates shared blast radius just as deduplication creates shared chunk dependence.
This is why base-object verification, checksums and durable storage matter.
A Wrong Base Can Make Compression Worse
If target and base are unrelated, the delta must describe almost everything as new data plus overhead.
At some point the system should stop being clever and store the object normally.
Good compression systems compare the cost of the relational representation with the cost of an independent one.
Similarity Search Has a Cost
To find a good base among millions of objects, a compressor cannot compare every object against every other object naively.
Practical systems use heuristics such as:
- same object type;
- similar size;
- related paths;
- recent versions;
- bounded search windows.
Git’s pack-object documentation exposes this explicitly through window and depth controls.
More Search Can Buy Better Compression
Search more candidate bases and the chance of finding a close match rises.
CPU and memory cost rise too.
The encoder therefore faces a rate–computation trade-off before the decoder ever sees the data.
Delta Encoding Is Not a Patch System by Definition
Software patches often use delta-like ideas because users already possess an earlier version.
But delta encoding is broader: backups, repositories, databases and content stores can all represent related objects through differences even when no human ever sees a “patch file.”
Network Transfer Can Exploit Receiver State
If the receiver already owns a base, the sender may need to transmit only the difference.
Git’s documentation describes “thin” packs that can omit common base objects during transfer and later be made self-contained by the receiver.
The network message becomes smaller because both sides share history.
But Shared History Must Be Exact
“I probably have something similar” is not enough.
Decoder and encoder must agree on the exact base identity required for reconstruction.
This is a version-control lesson with wider relevance: relational compression requires stable identity.
Deduplication and Delta Encoding Can Cooperate
First reuse chunks that are exactly identical.
Then delta-compress objects or chunks that are similar but not identical.
The first pillar, Deduplication, owns exact shared content.
This pillar owns residual difference against a base.
Canonicalisation Can Improve Delta Quality
If irrelevant formatting differences dominate, the delta spends bits describing noise.
Normalising a representation can make meaningful structural similarity easier to exploit—provided the application permits that canonicalisation.
Encrypted or Already-Compressed Data Often Delta Poorly
Encryption aims to make nearby plaintexts look unrelated in ciphertext.
Many compression formats can also transform local edits into broad byte changes.
Delta encoding works best when the representation preserves useful similarity.
Human Editing Is a Natural Delta Process
Teachers revise a worksheet.
Lawyers revise a contract.
Programmers revise source code.
Most revisions preserve far more than they change.
Version-aware compression matches the actual creation process better than pretending each final document appeared independently.
A Better Delta-Encoding Model
target → candidate bases → similarity search → choose base → copy/insert delta → compare with independent cost → store dependency → verify base → reconstruct target → periodically rebase/snapshot
A 28-Lens Delta Audit
- Target: what object must be represented?
- Family: which objects are plausible relatives?
- Base: which candidate gives the best useful delta?
- Identity: how is the base named?
- Similarity: how is closeness estimated?
- Search window: how many candidates are tested?
- Copy: what base regions are reused?
- Insert: what target bytes are new?
- Overhead: how large are instructions and references?
- Fallback: when is independent storage cheaper?
- Depth: how many deltas separate target from a full base?
- Access: how much work is needed to reconstruct?
- Locality: where do dependencies live?
- Snapshot: where should chains be broken?
- Rebase: can a better base be chosen later?
- Corruption: what descendants depend on a base?
- Verification: how is reconstruction checked?
- Transfer: does the receiver already own a base?
- Thin representation: can shared objects be omitted from transfer?
- Canonical form: are irrelevant differences hiding similarity?
- Encryption: has similarity been deliberately obscured?
- Compression order: was a similarity-destroying transform applied?
- Branching: does ancestry form a tree or graph?
- Garbage collection: can the base be removed safely?
- Compute: what encoder effort is justified?
- Memory: what candidate window is affordable?
- Regret: what if the chosen base was poor?
- Return: does base + delta reconstruct the exact target?
Laboratory 1: Paragraph Versions
Write a paragraph, revise two sentences, and record only the operations needed to transform the original into the revision.
Laboratory 2: Chain Depth
Create ten revisions where each depends on the previous one. Compare storage and reconstruction if version 10 depends through all nine deltas versus if version 6 is stored as a full snapshot.
Laboratory 3: Bad Base
Try to describe a poem as changes from an unrelated shopping list. The absurdity makes base selection intuitive.
For Primary Readers
If yesterday’s drawing is almost the same as today’s, keep yesterday’s drawing and write a small note saying what to change.
For Secondary Readers
Distinguish base, delta and reconstructed target. Then explain why a deep chain can save space but slow access.
For Advanced Readers
Model delta encoding as relational source coding over a dependency graph. Optimisation balances residual size, base-search cost, decoder locality, chain depth, failure propagation and future access patterns.
Common Misconceptions
- “A delta is simply the numerical subtraction of two files.” Practical deltas encode copy and insert relationships between structured byte sequences.
- “The previous version is always the best base.” Another related object may yield a smaller or shallower representation.
- “Smaller deltas are always better.” Deep dependency chains can hurt access and resilience.
- “Deduplication and delta encoding are the same.” Deduplication reuses exact identity; deltas exploit similarity.
- “A delta is self-contained.” Reconstruction requires the referenced base or equivalent shared state.
Research Corridor
- Git — Packfile Format — base objects and reference/offset delta representations.
- Git — pack-objects — delta search window, depth and decoder-performance trade-offs.
- Git User Manual — Efficient Object Storage.
- eduKateSG — Borrowed Time.
Frequently Asked Questions
What is delta encoding?
Delta encoding represents a target object relative to a base by storing instructions or data that describe how to reconstruct the target from that base.
Why do delta chains have depth limits?
Because reconstructing a deeply chained object requires applying several dependent deltas, increasing CPU, latency and failure exposure.
Why would a system store a full snapshot?
A full snapshot breaks dependency depth and can improve access speed and resilience at the cost of additional storage.
Final Thought: A Version Is Often Mostly Its Ancestor
The changed paragraph feels like a new file at the user interface.
At corpus level, it is largely inherited information.
Delta encoding becomes powerful when the system remembers that “new version” and “new information” are not the same quantity.
THE CORPUS · FOUR PILLAR LEGS
Return to The Corpus, or continue through Deduplication, Shared Dictionaries and Corpus Drift. Return to the Information & Representation Hub.