How does superintelligence work when an AI has to find the right resource rather than merely the first resource? A search page may put a popular result at the top even when it is written for the wrong school level, requires an account or cannot be printed. A useful answer distinguishes items that meet the user’s complete requirements from items that are simply highly ranked.
The mechanism is filtering, relevance and ranking. Filtering selects items satisfying stated conditions. Ranking orders those items according to a chosen signal. Retrieval evaluation asks how many relevant items were found and how many displayed items were actually relevant. Confusing the three can make a polished recommendation list unreliable.
This guide follows a fictional school-resource search with nine indexed results, explicit requirements and twenty auditable claims. We calculate precision and recall at different positions, explain why a high display score is not a probability of teaching value, and show why source quality still needs checking. The catalogue is invented teaching data, not a measurement of Google or any AI product. “How Superintelligence Works” names an educational series, not evidence of artificial superintelligence.
1. Search begins with an information need
Imagine a pupil looking for a Primary 5 worksheet on plant parts that can be printed without creating an account. The words “plant worksheets” are only a rough shorthand. The full task includes school level, topic, format and access conditions.
A search assistant that remembers only “plants” may retrieve a beautiful article on plant cells for much older students. That result is broadly relevant to plants and still unsuitable for the actual request. The full information need must survive retrieval.
2. Filtering answers the eligibility question
A filter decides whether a candidate passes an explicit requirement. A year-level filter might require Primary 5. A format filter might require printable material. An access filter might exclude resources demanding sign-up. Under a strict request, eligible items must pass every mandatory check.
This is not yet a judgement of which eligible resource teaches best. It establishes the permitted candidate set before the assistant decides what to present first.
3. Ranking orders candidates
A ranking method decides which results appear above others. Different systems use different signals, such as topic relevance, quality, popularity and context. In our toy catalogue, an arbitrary display score creates an initial order from highest to lowest.
The display score has no probability meaning. A score of 97 is not a 97% chance that the material is correct, and 90 does not mean the worksheet will improve a pupil’s score by 90%. Such interpretations would be invented.
4. The first result can fail a mandatory condition
An attractive top result might require users to register, while the pupil explicitly wants no sign-up. The first item is then ineligible regardless of its popularity. The assistant should not silently relax the condition to make the top result appear useful.
Similar problems arise with dates, languages, licences, accessibility and intended audiences. A high rank can motivate inspection but cannot cancel a requirement the user actually stated.
5. Relevance depends on what was asked
Photosynthesis and plant parts are related subjects, but they are not identical tasks. A Primary 6 worksheet may be informative without satisfying a strict Primary 5 request. The level and content requirements matter even if a general search score favours another result.
If the user later broadens their request, the eligibility definition changes. The assistant should explain that change rather than pretend the earlier narrow request already permitted every related topic.
6. The fictional Cedar Learning Library
Imagine Cedar Learning Library indexing nine invented resources, labelled A to I. A pupil requests a Primary 5 resource on plant parts that is printable and requires no sign-up. All four attributes are hard conditions in this teaching exercise.
The catalogue metadata is assumed to have been checked for these four fields. Its initial order uses fictional popularity-style scores. No result represents a real webpage or any actual teaching materials.
7. Nine fictional search results
- A — display score 97; Primary 5 plant parts; printable YES; sign-up YES.
- B — score 95; Secondary 3 plant cells; printable YES; sign-up NO.
- C — score 93; Primary 5 plant parts; printable NO; sign-up NO.
- D — score 90; Primary 5 plant parts; printable YES; sign-up NO.
- E — score 88; Primary 5 photosynthesis; printable YES; sign-up NO.
- F — score 84; Primary 5 plant parts; printable YES; sign-up NO.
- G — score 80; Primary 5 plant parts; printable YES; sign-up NO.
- H — score 77; Primary 6 plant parts; printable YES; sign-up NO.
- I — score 74; Primary 5 plant parts; printable YES; sign-up NO.
8. Which four results are eligible?
D, F, G and I pass all four conditions. A fails because of sign-up. B is for Secondary 3 plant cells. C cannot be printed. E covers photosynthesis rather than the specified plant-parts topic. H targets Primary 6, not Primary 5.
The eligible set contains four out of nine results. That statement describes the invented metadata, not a quality review or a claim about any real resource’s factual reliability.
9. The first three can all fail
The first qualifying item, D, appears in fourth position. A, B and C each fail a requirement. Returning the top three unfiltered results would therefore give the pupil no resource that meets the complete request.
This is a deliberately constructed demonstration, not an empirical claim about search engines. It teaches why “first” and “relevant under the user’s rules” should be evaluated separately.
10. Precision answers one question
Precision is the proportion of shown items that meet the defined relevance condition. At the first five positions of our toy list, only D is eligible. Precision at five, commonly written P@5, is 1/5 = 20%.
This calculation tells us how many displayed items qualify. It does not tell us how many qualifying items remain further down. Precision alone cannot establish search coverage.
11. Recall answers another
Recall measures how many of all relevant items were found. There are four eligible items in the entire invented catalogue: D, F, G and I. The top five contain only D. Recall at five is therefore 1/4 = 25%.
Notice the denominators. Precision at five divides by five shown items. Recall at five divides by four eligible items. Mixing those denominators changes the question and produces misleading figures.
12. Precision and recall at three
The top three contain zero eligible items. Precision at three is 0/3 = 0%, and recall at three is 0/4 = 0%. The visually prominent part of the original list is completely unhelpful under the strict request.
That result does not imply any resource was inaccurate or that its broader topic was worthless. It simply records that none of the first three met all four specified conditions.
13. Precision and recall at eight
The top eight contain D, F and G: three of the four eligible resources. Precision at eight is 3/8 = 37.5%. Recall at eight is 3/4 = 75%. The final eligible resource, I, appears in ninth position.
Showing more results increases coverage, but the added list still includes ineligible materials. A longer list need not be a more efficient experience for the reader.
14. What happens with all nine?
Among all nine results, four are eligible. Precision becomes 4/9, approximately 44.4%. Recall reaches 4/4 = 100% because every eligible item is now shown.
Complete recall does not guarantee a convenient shortlist. A result list can include all suitable materials while burying them among many that miss the request. Good retrieval considers both coverage and what users see first.
15. The checked retrieval scorecard
- Total indexed results: nine.
- Eligible resources: D, F, G and I — four.
- Original top three: zero eligible; P@3 = 0%; recall@3 = 0%.
- Original top five: one eligible; P@5 = 1/5 = 20%; recall@5 = 1/4 = 25%.
- Original top eight: three eligible; P@8 = 3/8 = 37.5%; recall@8 = 3/4 = 75%.
- Original top nine: four eligible; P@9 = 4/9 ≈ 44.4%; recall@9 = 4/4 = 100%.
- Filtering by all four requirements leaves D, F, G and I.
- First three after filtering: D, F and G — P@3 = 3/3 = 100%.
- Recall at three after filtering: 3/4 = 75%.
16. Filtering changes the shortlist
Remove A, B, C, E and H because each fails one or more mandatory conditions. The remaining list, in original score order, is D, F, G and I. The first three filtered items now all meet the requirement, so precision at three is 100% within this toy set.
This change does not create new knowledge about plant science. It applies known attributes correctly. If the source catalogue is incomplete or the attributes are wrong, the filtered list can still disappoint.
17. A match is not a quality guarantee
Even a resource that fits the year, topic, format and access requirement could contain poor explanations or incorrect diagrams. Our invented catalogue says nothing about scientific accuracy, classroom suitability, copyright permission or pedagogical effectiveness.
A real tutor should inspect the material before using it. The assistant may help locate likely candidates, but it should not turn a metadata match into a guarantee of educational value.
18. Twenty inspectable claims
- 01. “A has the highest initial display score.” — SUPPORTED.
- 02. “A meets the no-sign-up requirement.” — CONTRADICTED.
- 03. “B is a Primary 5 resource.” — CONTRADICTED.
- 04. “C is printable.” — CONTRADICTED.
- 05. “D meets all four requirements.” — SUPPORTED.
- 06. “E matches the requested plant-parts topic.” — CONTRADICTED.
- 07. “F meets all four requirements.” — SUPPORTED.
- 08. “G meets all four requirements.” — SUPPORTED.
- 09. “H is labelled Primary 5.” — CONTRADICTED.
- 10. “I meets all four requirements.” — SUPPORTED.
- 11. “The original top three contain an eligible item.” — CONTRADICTED.
- 12. “The original top five contain exactly one eligible item.” — SUPPORTED.
- 13. “Precision at five is 20%.” — SUPPORTED.
- 14. “Recall at five is 25%.” — SUPPORTED.
- 15. “Precision at eight is 37.5%.” — SUPPORTED.
- 16. “Recall at eight is 75%.” — SUPPORTED.
- 17. “Filtering all four requirements leaves D, F, G and I.” — SUPPORTED.
- 18. “The highest display score guarantees the best teaching outcome.” — NOT ESTABLISHED.
- 19. “A score of 90 is a measured 90% chance of usefulness.” — CONTRADICTED by the stated score definition.
- 20. “The first three filtered items are all eligible.” — SUPPORTED.
19. Reconcile the labels
Supported cases are 01, 05, 07, 08, 10, 12, 13, 14, 15, 16, 17 and 20: twelve. Contradicted cases are 02, 03, 04, 06, 09, 11 and 19: seven. Case 18 is not established: one. Twelve plus seven plus one equals twenty.
Case 18 lacks learning-outcome evidence. Case 19 is different: treating an explicitly arbitrary ordering score as a calibrated probability conflicts with the toy catalogue’s definition. The distinction between unsupported and contradicted claims matters.
20. Scores require definitions
A search interface can display relevance values, ratings, popularity numbers or paid-placement labels. Those are not interchangeable. A score of 90 might be meaningful only within one application’s ranking formula, and the formula may not be available to the user.
In our example the scores only define order. They cannot legitimately be translated into probabilities of correctness, suitability or improved grades. A responsible AI response explains what the score measures, or says when that information is missing.
21. Must-have conditions versus preferences
Suppose a pupil says the resource must be printable but would preferably include colourful diagrams. The printability condition determines eligibility. Colourful diagrams could be used to rank eligible items if reliably known.
Treating a preference as mandatory may eliminate useful alternatives. Treating a mandatory condition as merely preferred may recommend something unusable. A search assistant should retain the strength of each user constraint.
22. A broader topic is not always the right topic
Photosynthesis and plant parts are related, but they are not synonymous. A result about plant cells or chemical processes may be valuable for a different lesson while still failing a specific request about structures of flowering plants.
An AI that collapses all plant topics into one category will report high topic similarity while missing the learner’s actual needs. Appropriate granularity is part of relevance.
23. A different school level may still be useful later
H is a Primary 6 resource, which fails the strict Primary 5 requirement. Yet a tutor might later decide that a selected Primary 6 explanation is appropriate enrichment for a particular pupil.
The correct procedure is to make that judgement explicitly and, if needed, change the filter. The possibility of enrichment does not make the original strict classification wrong.
24. What if an attribute is unknown?
Imagine D’s printability is not known. The other three requirements may be satisfied, but D’s final eligibility cannot be established until that feature is checked. Treating UNKNOWN as YES creates a false positive; treating it as definitely NO can hide a useful result.
A good interface may maintain two lists: confirmed eligible results and resources requiring review. This lets users benefit from known information without manufacturing certainty.
25. The collection can hide missing resources
Even the best ranking method cannot retrieve an item that was never indexed. A catalogue might omit free worksheets from small publishers, local-language resources or materials distributed by schools through restricted channels.
An assessment of retrieval quality should therefore consider source coverage as well as ordering. A perfect top-three precision score over an incomplete collection does not establish that the best available resource was found.
26. Where source provenance enters
A title, link or brief preview may no longer match the actual file. A website can change its access terms, remove a PDF or update its learning content. The assistant should distinguish an indexed description from the live source.
For a real recommendation, retain the source link, inspection date and relevant evidence. Open the candidate when appropriate and check that it still meets the request rather than inventing an inspection.
27. Popularity and educational suitability are different
A popular document may attract many clicks because it has a familiar title, is heavily shared or serves a broad audience. That does not automatically mean its questions are correct, accessible, suitable for a specific learner or permitted for reuse.
Popularity can be a useful discovery signal. It is not a substitute for subject expertise and a content review. A reliable assistant preserves this difference in its explanation.
28. Learning needs can change the final ordering
Among D, F, G and I, a pupil might want fewer questions, more diagrams, worked solutions or additional challenge. The toy catalogue does not record any of those properties, so it cannot prove which of the four is best for learning.
The responsible next step is to inspect their content or ask which preference matters. A model should not pretend an original generic display score proves a pedagogical ranking.
29. Fictional baseline behaviour
Imagine a made-up assistant that simply returns A, B and C because they occupy the top three positions. It has followed the initial ordering while ignoring the no-sign-up, school-level and printability constraints.
This is a failure to preserve requirements through retrieval, not evidence that the assistant cannot recognise the word plants. A focused repair identifies hard constraints before picking the visible winners.
30. The repaired workflow
First interpret the complete request. Second retrieve candidate materials. Third verify the properties that determine eligibility. Fourth mark each candidate eligible, ineligible or needing confirmation. Fifth rank the eligible items using a stated preference. Sixth inspect quality before recommending material for real pupils.
A fictional implementation that returns D, F and G first after filtering has solved the toy eligibility problem. That is not a benchmark result for any real model, nor does it establish that those invented resources are scientifically or pedagogically correct.
31. A transfer exercise at rank four
Among the original first four results, only D qualifies. Precision at four is therefore 1/4 = 25%. Recall at four is 1/4 = 25%, because D is one of the four eligible items in the complete set.
The equal percentages are a coincidence of these denominators. They arise from different questions and need not match in other situations.
32. A transfer exercise when rules change
Suppose sign-up is now allowed but all other requirements remain. A becomes eligible, because it is Primary 5, about plant parts and printable. The eligible population increases from four to five.
Any subsequent recall calculation must use the new total of five. The search assistant should not reuse old precision and recall numbers after changing the relevance definition.
33. Guidance for teachers and parents
Before accepting an AI shortlist, check what the lesson needs, the child’s level, the accessibility and format requirements, whether sign-up is required, and whether the content has been inspected. A link to a resource is only the start of the evaluation.
Ask the AI to explain why an item was included, which required attributes are verified and what remains unknown. A shorter, well-supported shortlist can be more helpful than a long catalogue with unexplained choices.
34. Guidance for AI builders
Record the query, hard requirements, relevance labels, source identifiers and rank positions. Report precision and recall at useful cutoffs, and keep relevance evaluation separate from claims about factual quality or learner outcomes.
Test changes in requirements, stale metadata and items with unknown properties. Do not score a system only on whether its final prose sounds useful. Inspect the actual objects it retrieved and how well they satisfy the stated task.
Frequently asked questions
Is the first result always the best? No. Position depends on the ranking method and may not satisfy every requirement.
What is precision at K? The proportion of the first K returned items that meet the defined relevance condition.
What is recall at K? The proportion of all relevant items found within the first K results.
Can precision be high but recall low? Yes. A short list can include only eligible items while missing many others.
Does a score of 90 mean 90% accurate? Only if the system defines and validates that interpretation; our arbitrary display score does not.
Does eligibility mean educational quality? No. Format, subject and level matching are distinct from correctness and learning effectiveness.
What if an attribute is missing? Preserve the uncertainty and inspect the source rather than guessing.
The central lesson
A good AI search assistant keeps the user’s actual need intact from query to shortlist. Ranking is about order; filtering is about eligibility; verification is about whether the retrieved resource supports the intended use. None can automatically replace the others.
Before trusting an answer, ask which constraints were applied, what the scores mean, how many relevant items may be absent, and whether content quality was inspected. Those questions turn a search result into a better-supported learning decision.
Connected reading and reliable references
- How Super Intelligence Works — main learning curriculum.
- Categories and Borderline Cases — classification before ranking.
- Goals, Constraints and Trade-Offs — preserving mandatory requirements.
- Comparisons and Baselines — measures and denominators.
- Stanford Introduction to Information Retrieval: Rank-Based Measures — precision and recall.
- Manning, Raghavan and Schütze: Evaluation of Ranked Retrieval Results — information retrieval methods.
- NIST TREC — research into information retrieval evaluation.