A search engine has one billion possible items.
The ranking model can deeply score one thousand.
Which thousand get through the door?
That is candidate generation.
Quick Read
Candidate generation is the first-stage retrieval process that reduces a very large corpus to a manageable shortlist for later ranking, reranking or verification. Candidate generators may use lexical indexes, approximate nearest-neighbour search, locality-sensitive hashing, graph indexes, filters, rules or learned retrieval models.
One-sentence answer: candidate generation is lossy because almost the entire search space is discarded before expensive ranking begins, preserving only the items the first-stage retriever believes are plausible enough to consider further.
Ranking Cannot Rescue What Retrieval Never Sends
A perfect reranker cannot rank a missing item first.
If the correct document, product, diagnosis candidate or recommendation never enters the shortlist, every later stage is blind to it.
This makes first-stage recall one of the most important hidden constraints in a multi-stage system.
The Candidate Set Is a Computational Budget
Retrieve ten candidates and downstream ranking is cheap.
Retrieve ten thousand and recall may improve, but ranking cost rises dramatically.
Candidate-set size is therefore a direct rate–distortion dial between efficiency and search coverage.
Lexical Retrieval Preserves Explicit Words
Classical inverted indexes retrieve documents containing query terms or related lexical signals.
This is fast and interpretable.
It can miss semantically relevant documents that use different vocabulary.
The candidate generator preserves lexical overlap and may lose paraphrastic similarity.
Vector Retrieval Preserves Learned Geometry
Embedding-based systems map queries and items into a vector space.
Nearby vectors become candidate matches.
This can retrieve paraphrases and conceptual neighbours.
But anything the embedding geometry fails to place nearby may remain invisible.
Hybrid Retrieval Tries to Preserve More Than One Route
A strong system can combine lexical and vector candidates.
One route catches exact terms, names and identifiers.
Another catches semantic resemblance.
The union costs more but reduces dependence on one representation’s blind spots.
Approximate Search Adds Another Layer of Loss
A vector retriever may itself use approximate nearest-neighbour indexing.
Now the system has two selection layers:
- the embedding decides what “near” means;
- the approximate index decides which near items are cheap enough to retrieve.
Errors can arise from either layer.
Filters Can Remove the Correct Answer Before Relevance Is Scored
Language, date, geography, permissions, inventory status and category filters can shrink the candidate pool before ranking.
These filters are often necessary.
They can also be wrong.
A mislabeled document can disappear from search even if its text is perfectly relevant.
Recall@k Measures the Door, Not the Final Order
If the relevant item appears somewhere in the top k candidates, the later ranker still has a chance.
If it does not, the pipeline has already failed.
Candidate recall therefore measures whether the correct neighbourhood entered the room before we care where it sits inside the room.
More Recall Is Not Always Better
A candidate generator that returns almost everything achieves excellent recall.
It also fails its efficiency job.
The objective is not maximum recall at any cost.
It is sufficient recall under a realistic latency, memory and reranking budget.
Candidate Generation and Ranking Must Stay Separate Owners
Candidate generation decides who is eligible for detailed comparison.
Ranking decides the order among those eligible items.
Confusing the two hides a critical source of error because a ranking metric cannot diagnose missing candidates.
Candidate Generation and Truncation Are Related but Not Identical
Truncation is a general observation problem in which some cases never enter the observed dataset.
Candidate generation is an intentional runtime search stage that creates a temporary shortlist from a larger known corpus.
The structural resemblance is useful, but the canonical jobs remain separate.
Recommendations Are Built on Candidate Gates
A recommender may have millions of products or videos.
The heavy ranking model cannot score all of them for every user interaction.
A lighter retrieval stage chooses a few hundred or thousand candidates.
Personalisation quality therefore depends on what the first stage considers plausible enough to expose.
Rare Items Are Vulnerable
Popular items have many interaction signals and often live in well-populated regions of a retrieval model.
Rare, new or minority items may have weaker representations and appear in fewer candidate sets.
The system can become self-reinforcing: what is not retrieved is not clicked, and what is not clicked receives less evidence for future retrieval.
Diversity Can Be Lost Before Ranking
A reranker may include diversity objectives.
But if the candidate generator returns one narrow topic or one dominant product family, the reranker cannot manufacture missing alternatives.
Diversity must survive into the candidate pool first.
Retrieval-Augmented Systems Live or Die Here
A generation model can reason only over evidence it receives.
If retrieval omits the authoritative passage, later reasoning may sound fluent while being poorly grounded.
Candidate-generation error can masquerade as reasoning error.
Education: Memory Retrieval Is a Candidate Problem
A learner can possess many facts but bring only a few into working memory when a question arrives.
The quality of thinking depends partly on whether the right ideas enter that shortlist.
This is an analogy, not database mechanics. The shared principle is that later reasoning cannot use a relevant item that was never retrieved.
A Systems View: Audit the Door Before the Room
Forward: does the final ranking look good?
Backward: was the true answer ever present in the candidate pool?
Rotate: do rare items, languages, new content, long-tail topics and alternative formulations enter at the same rates?
The first-stage gate should be treated as a knowledge bottleneck, not merely an optimisation detail.
When Candidate Generation Is Exactly the Right Loss
Use it when the corpus is too large for expensive full scoring, downstream ranking is powerful but costly, multiple retrieval routes can protect recall, and the candidate budget is validated against the real long-tail workload.
When Candidate Generation Becomes Dangerous
- Candidate recall is not measured separately from ranking quality.
- The shortlist is too small for rare queries.
- One retrieval representation dominates all others.
- Filters remove relevant items before ranking.
- New or minority items are systematically under-retrieved.
- The pipeline blames the ranker for evidence the retriever never supplied.
A Practical Candidate-Generation Audit
- Corpus: what full search space exists?
- Routes: lexical, vector, LSH, graph, rules or hybrid?
- Budget: how many candidates survive?
- Recall: how often is the relevant item present before ranking?
- Long tail: how do rare and new items perform?
- Filters: what is excluded before relevance scoring?
- Baseline: what does a more exhaustive search recover that the fast gate misses?
Research Anchors
Modern large-scale search and recommendation systems commonly separate retrieval from ranking because expensive models cannot score entire corpora. Approximate nearest-neighbour methods, including locality-sensitive hashing, remain important candidate-generation tools when high-dimensional similarity must be searched under tight latency budgets.
Continue Through eduKateSG
Continue with How Lossy Works | Locality-Sensitive Hashing, How Lossy Works | Ranking, and How Lossy Works | Truncation.
Final Thought: Every Search Result Begins With a Much Larger Set of Things You Chose Not to Search
Candidate generation makes impossible-scale ranking possible.
Its responsibility is to lose almost everything without losing what the future ranking stage will need.