A document contains two thousand words.
One paragraph contains the answer.
The retrieval system turns the entire document into one vector.
One point in a high-dimensional space.
The question is not whether that point is useful.
It often is extraordinarily useful.
The harder question is:
What had to disappear so the whole document could become one point?
Quick Read
Single-vector embedding systems encode a query, sentence, passage or document into one dense vector. Retrieval then compares vectors using a similarity measure such as cosine similarity or dot product. The representation is compact enough to index millions or billions of items efficiently using approximate nearest-neighbour systems.
The compression is also severe. Word order, local evidence, multiple topics, rare details and token-level relationships must all influence one fixed representation. Recent 2026 retrieval research explicitly describes long-document single-vector encoding as a form of document-side early compression: a short but decisive span can be diluted before ranking even begins.
One-sentence answer: single-vector embeddings are lossy because they replace a rich structured object with one learned coordinate, preserving broad semantic geometry while forcing fine-grained evidence and competing meanings to share limited representational space.
Why Turn Meaning Into Coordinates?
Search needs a way to compare.
Traditional lexical search compares words, term frequencies and document statistics.
Dense retrieval instead learns a geometry.
A query such as “how do plants convert light into stored chemical energy?” can be placed near documents about photosynthesis even when the document does not repeat every query word exactly.
That is the great promise of embeddings: semantic resemblance can survive vocabulary change.
The Encoder Is a Compression Machine
A modern text encoder receives a sequence of tokens.
Internally, many layers produce contextual representations.
At the end, some pooling or designated representation compresses the sequence into one vector.
Thousands of token-level states become perhaps hundreds or a few thousand numerical dimensions.
The dimensionality may sound large.
But the source contains far richer structure than one fixed-length coordinate can preserve independently.
Pooling Decides What Survives
Mean pooling, special-token pooling, learned pooling and other strategies create different final representations.
Average too aggressively and a rare but decisive span can be diluted by hundreds of ordinary tokens.
Rely on one special token and the model must learn to funnel all useful document information into that location.
The pooling rule is not merely implementation detail.
It defines the final compression interface.
Evidence Dilution: The Important Sentence Can Drown
Consider a 5,000-word policy document.
One sentence near the end contains the exact exception the query asks about.
A single-vector encoder must produce one representation for the entire document.
The exception contributes to that representation, but so do thousands of other tokens.
Recent 2026 work on long-document retrieval calls this failure mode evidence dilution: document-level encoding can weaken the signal from the strongest relevant chunk before the retriever ever compares the document with the query.
The Retrieval Failure Can Happen Before Search Begins
This is one of the most important distinctions in the entire Lossy series.
Suppose the correct document is stored perfectly.
Suppose HNSW searches the vector index perfectly.
Suppose the reranker is excellent.
The system can still fail if the document embedding never placed the document near the query in the first place.
The search index cannot recover semantic evidence that the representation already compressed away.
Single-Vector Retrieval and Candidate Generation Must Stay Separate
Candidate Generation owns the pipeline question:
Which items enter the shortlist?
Single-Vector Embeddings own an earlier representation question:
What does each item look like to the retriever before candidate generation starts?
This boundary is critical for diagnosis. Candidate recall can be poor because the index searched badly or because the embedding geometry never made the relevant item look close.
Cosine Similarity Preserves Direction, Not Every Meaning
Dense retrieval often compares vectors by cosine similarity.
This asks whether two vectors point in similar directions.
The measure is compact and useful.
But the meaning of distance is learned from training data. If the training objective did not align with the receiver’s relevance notion, nearby vectors can be semantically plausible and operationally wrong.
Similarity Is Not Relevance
A passage can discuss the same topic and still fail to answer the question.
A product can be similar and still violate a hard requirement.
A legal case can use related language and still belong to the wrong jurisdiction.
A medical article can be semantically close while describing a different population.
Embedding similarity is therefore a useful retrieval signal, not a universal relevance oracle.
One Point Must Represent Several Topics at Once
Long documents are rarely about one thing.
A research paper contains problem, method, results, limitations and related work.
A school textbook chapter contains explanations, examples, exceptions and exercises.
A company policy contains general rules and rare special cases.
One vector must somehow locate all of that content in one place.
The representation can become an average semantic address that is close to the document’s main theme and far from one important minor theme.
The Drowning Problem Gets Worse as the Corpus Grows
Recent 2026 work examining the limits of single-vector embeddings highlights a corpus-scale problem sometimes described through the “drowning” intuition.
As more documents enter the database, more irrelevant documents can obtain similarity scores close to the relevant one.
Embedding similarity behaves partly like a noisy proxy for relevance.
In a small corpus, the relevant document may remain clearly visible.
In a huge corpus, the same representation can be drowned by many plausible competitors.
Domain Shift Changes the Geometry
An embedding model trained on general web text can work beautifully on ordinary search and poorly on specialist legal, scientific or educational corpora.
Recent analysis of single-vector retrieval finds that domain shift and misalignment between embedding similarity and task-specific relevance can be major sources of failure.
Fine-tuning can help.
But fine-tuning changes the geometry again and can reduce performance elsewhere.
Fine-Tuning Is a Reallocation of Representational Priority
Train the embedding model on legal relevance and legal distinctions gain priority.
Train it on product search and product attributes gain priority.
Train it on school question answering and curricular evidence may become more separable.
Fine-tuning does not make one vector universally richer.
It redistributes limited geometry toward the target task.
Chunking Is One Repair
Instead of embedding a whole long document as one vector, split it into chunks.
Each chunk receives its own coordinate.
A short relevant span now has a better chance to dominate its own local representation instead of being diluted by the rest of the document.
The price is index size, more candidate management and the loss of some cross-chunk context.
Chunk Size Is Another Lossy Dial
Large chunks preserve more context and risk evidence dilution.
Small chunks preserve local specificity and risk fragmenting meaning across boundaries.
A definition may sit in one chunk while its exception sits in the next.
There is no universal perfect chunk size because the unit of meaning depends on the task.
Multi-Vector Retrieval Keeps More Local Evidence
Late-interaction methods such as ColBERT-style retrieval represent a document with multiple token-level or sub-token-level vectors rather than one pooled vector.
At query time, fine-grained interactions can match query tokens against document-token vectors.
This preserves more local semantic structure.
It costs far more storage and retrieval complexity.
The system buys fidelity by refusing to compress the document all the way down to one point.
Late Interaction Shows Exactly What Single-Vector Search Gave Up
In a single-vector system, query and document are each compressed independently before they meet.
In late interaction, the query and document retain multiple contextual vectors and interact more finely at scoring time.
The added quality comes from delaying part of the compression until after more of the query-document relationship can be observed.
This is why “late interaction” is such an important phrase: interaction is postponed so evidence does not have to disappear as early.
Cross-Encoder Reranking Delays Compression Even Further
A cross-encoder can process a query and candidate document jointly.
Token-level interactions occur inside the model before a relevance score is produced.
This can be more precise than comparing independently produced single vectors.
But it is too expensive to apply to an entire billion-document corpus.
That is why modern pipelines often retrieve coarsely first and rerank a much smaller shortlist.
Fast Retrieval Is Built on Early Commitment
The single-vector approach works because every document can be encoded before the query arrives.
The expensive text encoder runs once.
The resulting vector can be indexed.
At search time, one query vector can be compared efficiently against the index.
The speed comes from committing to a query-independent document representation early.
That is also where the main loss enters.
Query-Independent Compression Cannot Know the Future Question
A document is embedded today.
Tomorrow, one user asks about its main subject.
Another asks about a rare footnote.
A third asks about a negated exception.
The same one vector must serve all three queries.
The encoder had to choose what mattered before it knew which future receiver would arrive.
This Is the Same Deep Problem as Preservation Masters
An archive keeps a rich master because future questions are unknown.
A single-vector embedding is closer to an access derivative.
It is compact, fast and task-shaped.
It should not replace the source document because future retrieval or reasoning may need distinctions the vector never preserved.
Embeddings Are Not Meaning Itself
A vector is produced by a model under a training objective.
It is not a universal coordinate assigned by nature.
Change the model and the same document moves.
Change the fine-tuning data and neighbourhoods change.
Change the similarity function and ranking changes.
The embedding space is a learned instrument.
Dimensionality Is Not the Whole Story
It is tempting to say single-vector retrieval fails because one vector simply has too few dimensions.
Recent 2026 theoretical and empirical work argues that dimensionality alone cannot explain the observed limitations.
Training alignment, corpus scale, domain shift and the representation’s inability to preserve all query-dependent distinctions can matter substantially.
A larger vector is not automatically a better relevance model.
The Representation Can Be Stable and Still Be Wrong for the Task
Suppose the same document always maps to the same vector.
That is reproducible.
But reproducibility does not prove task fitness.
A stable vector can consistently place the relevant document in the wrong neighbourhood.
Lossiness must therefore be evaluated by downstream retrieval behaviour, not merely vector consistency.
False Neighbours Are a Geometry Failure
Two documents can become neighbours because they share broad topic language.
Yet one answers the query and the other does not.
The embedding geometry preserved thematic resemblance and lost the distinction the receiver needed.
This is one reason rerankers help: they spend more representational precision on the shortlist.
False Separation Is the Mirror Failure
Two documents can express the same relevant idea using very different vocabulary, structure or domain conventions.
If the encoder does not align those forms, the vectors remain far apart.
The system has lost semantic equivalence rather than semantic distinction.
Hybrids Exist Because One Geometry Is Not Enough
Dense retrieval is good at semantic similarity.
Lexical retrieval is good at exact names, numbers, rare terms and literal constraints.
Hybrid systems preserve both routes into the candidate set.
This is not duplication for its own sake.
It is deliberate redundancy against representation-specific loss.
2026 Retrieval Systems Still Use Reranking Because Compression Is Useful, Not Perfect
Recent 2026 systems continue to combine hybrid or dense retrieval with cross-encoder reranking. SemEval-2026 multi-turn retrieval work, for example, used query rewriting, hybrid retrieval and cross-encoder reranking in a staged pipeline and reported gains over its baseline.
The architecture itself is evidence of the trade-off.
Single-vector and first-stage retrieval are efficient enough to search broadly.
Later interaction is accurate enough to inspect narrowly.
Education: A One-Number Student Profile Has the Same Structural Risk
A student can be reduced to one overall mark.
The mark may be useful for broad comparison.
It cannot preserve every misconception, strength, speed pattern, transfer ability and confidence state independently.
This is an analogy, not embedding mathematics.
The shared lesson is that one coordinate can be powerful for navigation and poor for diagnosis.
A CivDJ View: Rotate the Same Document Through Different Queries
Forward: does the document retrieve for its main topic?
Backward: which decisive spans became too weak inside the pooled vector?
Rotate: does the same document remain retrievable for exceptions, rare terms, negations, minority topics, long-tail questions and domain-specific meanings?
A document embedding should not be judged only by the easiest query that agrees with its main theme.
Wintour V1.0 Reader Test: What Is the Canonical Owner Here?
This article owns representation loss before retrieval.
- Candidate Generation owns the shortlist gate.
- HNSW owns graph navigation through a vector index.
- IVF owns coarse partition routing.
- LSH owns probabilistic locality buckets.
- Ranking owns ordering among surviving candidates.
- Projection owns high-dimensional-to-lower-dimensional mapping more generally.
Single-Vector Embeddings own the specific question of what happens when one rich object becomes one learned retrieval coordinate.
When Single-Vector Embeddings Are Exactly the Right Loss
Use them when the corpus is large, fast indexing matters, broad semantic retrieval is valuable, documents are short enough or coherent enough for pooling, and later stages can verify or rerank important candidates.
They are especially strong as a first-stage representation where speed, memory and scalable search matter more than preserving every token-level interaction.
When Single-Vector Embeddings Become Dangerous
- Long documents contain short decisive spans.
- Multiple unrelated topics compete inside one vector.
- Exact names, numbers, negations or rare terms matter.
- Domain shift changes what similarity means.
- The corpus grows until relevant items are drowned by many plausible neighbours.
- Embedding similarity is treated as final relevance rather than first-stage evidence.
- The original text is discarded and the vector becomes the only surviving representation.
A Practical Single-Vector Audit
- Unit: sentence, passage, chunk or whole document?
- Pooling: how does token-level structure become one vector?
- Metric: cosine, dot product or another similarity?
- Length: does retrieval degrade as documents become longer?
- Evidence dilution: do strong local spans become weak document-level matches?
- Domain: was the embedding model trained or adapted for this relevance notion?
- Hybrid route: can lexical retrieval protect names, numbers and rare terms?
- Reranking: does a more expressive second stage correct coarse similarity?
- Source: is the original text always available for verification?
Sources and Further Reading
- Lyu et al. (2026), Lost in a Single Vector: Improving Long-Document Retrieval with Chunk Evidence Aggregation — studies document-side early compression and evidence dilution in single-vector retrieval.
- On Strengths and Limitations of Single-Vector Embeddings (2026) — examines domain shift, retrieval misalignment, corpus drowning and gaps between single-vector and multi-vector representations.
- AAAI 2026: Incorporating Token Importance in Multi-Vector Retrieval — evaluates fine-grained late-interaction retrieval using token-level vectors.
- ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction — establishes efficient multi-vector late-interaction retrieval with aggressive compression.
- SemEval-2026 multi-turn retrieval system — demonstrates a contemporary hybrid-retrieval plus cross-encoder-reranking pipeline.
Continue Through eduKateSG
Continue with How Lossy Works | Candidate Generation, How Lossy Works | HNSW, How Lossy Works | Inverted File Indexing, and How Lossy Works | Coarse-to-Fine Retrieval.
Final Thought: One Point Is Fast Because It Is Not the Document
Single-vector embeddings transformed modern retrieval because one point is cheap to store, cheap to compare and easy to index.
The price is equally fundamental.
The point cannot remain every sentence, every exception, every local relationship and every possible future query at once.
It is a useful address.
It is not the house.