How does superintelligence work when an AI is researching a question and needs to decide when to stop searching? A system that stops after the first attractive result may miss important evidence. A system that searches forever may waste time and still never produce an answer. A useful research assistant needs a stopping rule tied to the user’s actual goal, the quality of the evidence and the consequences of missing something.
The mechanism is evidence-based stopping. Before searching, define what would count as a usable answer, what must be checked, how much effort is available and whether the task requires a shortlist or a genuinely comprehensive review. Then inspect results, distinguish confirmed evidence from guesses, and stop only when the specified conditions are met—or explain why they cannot be met.
This accessible guide follows an invented school-resource search that inspects six candidates to find three qualified printable worksheets. It contrasts a valid shortlist with an exhaustive search, calculates simple coverage measures and checks twenty statements. Every result and score is fictional; nothing here claims that real resources were inspected or that a particular deployed AI behaves this way. “How Superintelligence Works” is an educational series title rather than a claim of achieved artificial superintelligence.
1. Why search is not automatically better when it is longer
Imagine asking for three suitable worksheets. Once three have been verified against the requirements, the core request may be satisfied. Searching another hundred pages could uncover alternatives but may not materially improve the immediate task.
However, the amount of search required changes if the user asks for every available worksheet, the best possible worksheet or a formal systematic review. Those goals have different standards. Stopping rules must reflect the task rather than use one fixed number of searches everywhere.
2. The information need comes first
Before searching, write down what counts as relevant. A Primary 5 pupil wants plant-parts worksheets that can be printed without signing in. A source about photosynthesis may be broadly related but fail the specified topic; a useful page behind an account requirement may fail the access condition.
An AI that forgets one requirement can believe it has enough evidence when it has only enough vaguely related material. Quality must be evaluated against the full request.
3. A shortlist is not an exhaustive catalogue
A shortlist answers “Give me a few suitable choices.” A comprehensive catalogue answers “Find all relevant choices within a specified search universe.” The first may stop after a target count. The second requires a defined search scope and a defensible completeness procedure.
Neither approach is universally better. A pupil seeking something to practise tonight may value a short inspected list. A researcher making a systematic claim about an entire field needs much stronger coverage.
4. Define the stopping rule before looking
Suppose the fictional desk sets a target of three independently checked eligible resources and a search budget of eight inspections. It stops when the first of these occurs: the target is met, the eight-inspection budget is exhausted or a material blocker prevents further work.
These conditions tell us what “enough” means in the classroom scenario. They also make it easier to audit a result. An assistant should not quietly redefine success halfway through a search just because it likes the first links it found.
5. What counts as an eligible result?
For this task, each candidate must cover Primary 5 plant parts, be printable and not require sign-up. The desk must have inspected or reliably verified those properties, not merely inferred them from a title.
A page that might fit but has not been checked is a candidate, not a confirmed success. A model can show it separately for review but should not add it to the target count.
6. The invented Cedar Resource Search
Imagine a small fictional index containing twelve candidate resources A through L in a fixed order. The desk initially inspects candidates from the top downward. The index was designed for this teaching exercise, so every property listed below is a stipulated fact or explicitly unknown.
No real website, teacher or document is represented. The purpose is to test the decision process and counting, not to recommend actual learning materials.
7. The twelve reference candidates
- A — Primary 5 plant parts, printable, no sign-up: CONFIRMED ELIGIBLE.
- B — Primary 5 plant parts, not printable: CONFIRMED INELIGIBLE.
- C — Primary 5 plant parts, printable, no sign-up: CONFIRMED ELIGIBLE.
- D — Primary 5 plant parts, printability UNKNOWN: NEEDS CHECK.
- E — Primary 6 plant parts: CONFIRMED INELIGIBLE under the strict year-level rule.
- F — Primary 5 plant parts, printable, no sign-up: CONFIRMED ELIGIBLE.
- G — Primary 5 plant parts, printable, no sign-up: CONFIRMED ELIGIBLE.
- H — Primary 5 photosynthesis, rather than plant parts: CONFIRMED INELIGIBLE.
- I — Primary 5 plant parts, printable, no sign-up: CONFIRMED ELIGIBLE.
- J — Primary 5 plant parts, printable but sign-up required: CONFIRMED INELIGIBLE.
- K — Primary 5 plant parts, printable, no sign-up: CONFIRMED ELIGIBLE.
- L — Primary 5 plant parts, access conditions UNKNOWN: NEEDS CHECK.
8. Inspect the first two
A is checked and eligible, so the verified count is one. B is checked but cannot be printed, so it does not increase the count. Two inspections have produced only one result that meets every requirement.
The distinction matters. A system that counts every inspected page toward the goal would claim two suitable resources when only one is confirmed. The effort count and the success count are different.
9. Inspect the third candidate
C is checked and eligible. There are now two verified choices after three inspections. The user’s target of three has not yet been met.
An assistant could report partial progress at this point, but should not call the request complete. The stopping rule is explicit, and one more confirmed item is needed.
10. An unknown is not a success
D looks promising but its printability has not been established. The desk classifies it as NEEDS CHECK rather than assuming it qualifies. After inspecting the fourth candidate’s available metadata, the verified-success count remains two.
An alternative workflow might spend extra effort resolving D immediately. Our fictional process instead keeps its known status and proceeds down the list. Different strategies can be compared, but neither may fabricate the missing fact.
11. A relevant-looking item can fail a hard condition
E covers plant parts and might be educationally useful, but it is labelled for Primary 6. Under this invented strict Primary 5 filter, it is ineligible. After five inspected candidates, the verified count is still two.
That does not establish that Primary 6 material would be harmful or unsuitable for every pupil. It simply applies the user’s precise request rather than silently broadening the scope.
12. The sixth inspection meets the target
F is confirmed eligible. The three verified resources are now A, C and F. Exactly six candidates have been inspected to meet the target of three under the given order. Six is below the eight-inspection budget.
The desk can stop at this point for the stated shortlist task. It should provide those three results and disclose that the remainder of the index was not inspected during this run. A defensible stopping point need not pretend the entire search universe has been covered.
13. The first-six inspection ledger
- Target: three confirmed eligible resources.
- Maximum budget: eight inspected candidates.
- After A: one confirmed eligible.
- After B: still one; B fails printability.
- After C: two confirmed eligible.
- After D: still two; D has unknown printability.
- After E: still two; E fails year level.
- After F: three confirmed eligible.
- Stopping point: six inspections, target achieved.
- Result shortlist: A, C and F.
- Uninspected during the shortcut: G through L.
14. The shortcut does not prove these are the best three
The first three confirmed choices meet the mandatory conditions. The toy record does not provide quality ratings, the length of the worksheets, explanation clarity or lesson outcomes. It therefore cannot prove A, C and F are the three best possible resources.
“Here are three checked, eligible choices” is supported. “These are the optimal choices among every available resource” would be an unsupported upgrade. Relevance and ranking are not automatically the same.
15. What would an exhaustive review find?
The complete fictional index contains six confirmed eligible resources: A, C, F, G, I and K. B, E, H and J are confirmed ineligible. D and L remain unresolved because their relevant attributes are unknown.
Even after inspecting all twelve metadata records, the source does not conclusively identify every eligible item if the unknown fields could change eligibility. A systematic review would need to resolve D and L or report that completeness remains limited.
16. Coverage in the invented index
The six-inspection shortlist captures three of the six confirmed eligible records in the full reference list. Its confirmed-eligible coverage is therefore 3/6 = 50%, measured against those six known positives.
That 50% is a retrospective observation using the invented full reference list. The desk at the sixth inspection would not know that denominator merely from the first six results. An AI must not imply it measured completeness without seeing the necessary reference set.
17. Why a search budget is useful
An application may limit the number of sources it inspects because each page takes time and computational effort. A clear budget can make the process predictable and avoid endless browsing through repetitive results.
The budget should be appropriate to the task. A casual shortlist can have a smaller budget than a safety-critical literature review. When the budget expires before enough evidence is found, the assistant should report the incomplete outcome rather than invent additional confirmations.
18. Stopping due to exhaustion is different from success
Suppose only two qualifying resources had appeared before all eight allowed inspections were completed. The budget would have been exhausted without meeting the target. The correct status would be partial completion: two verified resources and a stated limit.
A system that describes such a result as “three excellent choices found” would fail both counting and source integrity. Knowing when to stop includes knowing whether the reason was success, exhaustion or a blocker.
19. Why source trust matters more than count alone
Three copied articles repeating the same unsupported claim may not constitute three independent pieces of evidence. For a factual research question, the goal should account for source quality and independence as well as raw link count.
Our toy worksheet task counts distinct inspected resources, not scientific confirmation of a contested claim. An assistant should choose a stopping criterion appropriate to the question rather than transfer a simplistic three-source rule everywhere.
20. Twenty claims about the search
The fictional desk asks the AI to classify claims as SUPPORTED, CONTRADICTED or NOT ESTABLISHED. Some concern the frozen policy and first-six path; others concern the full invented index.
The correct distinction is between what was actually verified under the task and what remains unobserved or unsupported. A short successful search does not establish universal optimality.
- 01. “The target is three eligible resources.” — SUPPORTED.
- 02. “The budget is eight inspections.” — SUPPORTED.
- 03. “A is confirmed eligible.” — SUPPORTED.
- 04. “B is confirmed eligible.” — CONTRADICTED.
- 05. “C is confirmed eligible.” — SUPPORTED.
- 06. “D is confirmed ineligible.” — NOT ESTABLISHED.
- 07. “E fails the strict year-level requirement.” — SUPPORTED.
- 08. “F is confirmed eligible.” — SUPPORTED.
- 09. “The third confirmed result arrives at F.” — SUPPORTED.
- 10. “Six candidates have been inspected when the target is met.” — SUPPORTED.
- 11. “All twelve were inspected before the shortcut stopped.” — CONTRADICTED.
- 12. “Stopping at F proves no better worksheet exists.” — NOT ESTABLISHED.
- 13. “G is confirmed eligible in the full reference index.” — SUPPORTED.
- 14. “H matches the specified plant-parts topic.” — CONTRADICTED.
- 15. “I is confirmed eligible in the full index.” — SUPPORTED.
- 16. “J requires sign-up.” — SUPPORTED.
- 17. “K is confirmed eligible in the full index.” — SUPPORTED.
- 18. “L definitely qualifies.” — NOT ESTABLISHED.
- 19. “The full reference index lists six confirmed eligible items.” — SUPPORTED.
- 20. “The full reference index lists six confirmed ineligible items.” — CONTRADICTED; four are confirmed ineligible, two unresolved.
21. Reconcile the classifications
SUPPORTED: 01, 02, 03, 05, 07, 08, 09, 10, 13, 15, 16, 17 and 19 — thirteen. CONTRADICTED: 04, 11, 14 and 20 — four. NOT ESTABLISHED: 06, 12 and 18 — three. Thirteen plus four plus three equals twenty.
These checks expose the temptation to turn an unknown resource into an eligible or ineligible one, or to mistake a sufficient shortlist for a complete search. The target and the evidence boundary must remain visible.
22. Different questions need different stopping conditions
“Find three printable worksheets” is an existence-and-suitability task. “Find every applicable government rule” requires a much stronger completeness method. “What is the latest verified schedule?” may depend on a current authoritative source rather than accumulating many old pages.
An assistant should not reuse the same stopping rule for all tasks. It must determine whether the outcome requires a fixed number of choices, an authoritative record, broad coverage or an estimate with stated uncertainty.
23. Independent verification can terminate a factual search
Some questions have a narrow decisive source, such as an official standard stating a particular measurement definition. Once the correct authoritative passage has been read and the exact statement checked, further broad searching may add little value.
Other questions, such as why educational outcomes differ between groups, require competing explanations, research designs and a broader evidence base. Stopping after the first plausible answer can create confirmation bias.
24. Repeated evidence may add little information
Suppose ten search results copy the same press release without new inspection or analysis. Reading all ten may not provide ten independent confirmations. The sources’ common origin matters.
A good research assistant can track provenance and identify when another search hit simply restates existing information. This helps allocate attention to genuinely new evidence rather than counting duplicates as progress.
25. Diminishing returns does not settle every task
In some searches, the first few sources answer the main question and later ones repeat the same information. That can justify stopping when the user needs a simple explanation and the answer has been checked.
However, an apparently repetitive search can still miss a rare counterexample, an important correction or an updated official notice. The expected value of further searching depends on the consequences of missing evidence, not just on how boring the last few results seemed.
26. A search can stop because the question changed
Imagine the pupil now allows Primary 6 resources. E could become eligible under the revised target, and the denominator of eligible items would change. The original shortlist remains a valid answer to the earlier strict request.
A reliable assistant should update only the named constraint and reassess affected items. It should not pretend that the changed rule was part of the original search evaluation.
27. Search failure should be reported honestly
Sometimes a source is inaccessible, a link is broken or an institution has not published the necessary record. If an exact fact is required and the evidence cannot be obtained, the assistant may have to stop with a limitation.
An incomplete search is not a licence to invent a date, source quotation or booking availability. An explicit “not verified” often serves the user better than a confident but unsupported answer.
28. A stopping rule is a decision policy
At each step the assistant can choose to inspect another candidate, ask a clarifying question or return the current findings. The best next action depends on the evidence already gathered, the cost of further work and the user’s goal.
This resembles decision-making under uncertainty, but it is not necessarily a universal mathematical optimum. Our toy rule is a transparent practical policy: find three and stop within eight inspections.
29. A systematic review needs a different standard
Systematic reviews aim to identify and synthesise relevant evidence using predefined methods, search strategies and eligibility criteria. They generally should not stop simply because three convenient studies support a preferred conclusion.
The Cochrane Handbook describes rigorous literature searching and documentation. It does not imply that every everyday question requires such extensive work, but it demonstrates why comprehensive evidence claims have more demanding procedures than a casual shortlist.
30. How to log the stopping decision
Record the original question, mandatory criteria, search sources, inspected candidate count, verified successes, unresolved candidates, stopping reason and material limitations. This makes the process reproducible and easier to correct if new information arrives.
“Stopped at six inspections because the third eligible resource was verified” is much clearer than “research finished”. The explanation tells the reader what was achieved and what was not attempted.
31. A fictional baseline assistant
Imagine a made-up assistant that stops after candidate C because it has seen two attractive worksheets and an unclear third result. It calls D qualified without checking printability and declares the task complete. That is a failure of the target count and the evidence requirement.
A repaired process retains UNKNOWN for D, keeps looking and stops at F only after three distinct confirmed eligible resources have been obtained. This is an illustrated method, not measured performance of an actual deployed model.
32. A transfer exercise with historical sources
A student asks for two reliable sources documenting the date of a historic event. If a primary archive and a scholarly reference independently support the date, the student may have enough for a simple classroom explanation. A full historical research project might require broader inspection and disagreements.
The relevant stopping criterion is tied to the task and the authority of the evidence, not to an arbitrary universal rule that two sources always settle every question.
33. A transfer exercise with an unanswered question
Suppose an official notice gives a school event’s date but omits the start time. Searching several copies of the same notice will not create a start time. The next useful step may be to ask the organiser or locate an official timetable.
The assistant should stop repeating ineffective searches when the source gap is clear. It can explain which single missing fact blocks a complete answer instead of manufacturing it.
34. What a helpful AI answer sounds like
“I checked six fictional candidates and found three that met the Primary 5, plant-parts, printable and no-sign-up requirements: A, C and F. Candidate D still needs a printability check. I stopped because the requested shortlist target was reached; the remaining candidates were not examined in this pass.”
That report is transparent about the evidence and stopping reason. It does not pretend the shortlist is globally optimal, that all sources have been searched or that an unknown has been resolved.
35. How to evaluate stopping behaviour
Test tasks requiring a fixed shortlist, an authoritative single source, comprehensive coverage and a response under a time or inspection budget. Include duplicates, inaccessible sources, ambiguous candidates and changing requirements.
Score not only whether the final links are suitable, but whether the assistant stopped for the correct reason and represented partial coverage accurately. An answer can be accurate yet inefficient, or efficient yet incompletely justified.
Frequently asked questions
When should AI stop searching? When the task’s evidence requirement is met, its justified budget is exhausted or a blocker makes further search unproductive, with the outcome reported accurately.
Does three sources always prove a claim? No. Their quality, independence and applicability matter.
Can a shortlist be complete without being exhaustive? It can completely satisfy a request for a limited number of choices while not covering every available alternative.
Does a promising unverified result count? Not toward a target requiring verified eligibility.
Does further searching always improve accuracy? No. It may add repetition, introduce low-quality information or be unnecessary for the task.
What if the inspection budget runs out? Report the verified partial results and disclose the remaining limitation.
How should AI document stopping? Name the criteria, number inspected, findings, unresolved gaps and reason for stopping.
The central lesson
Good research is neither stopping at the first attractive sentence nor searching forever. It is knowing what evidence the question requires, keeping track of what has been verified, and stating exactly why the answer is sufficiently supported—or why it remains incomplete.
The practical habit is to decide in advance what “enough” means. Then hold the assistant to that standard and preserve the difference between a useful shortlist, a verified fact and a comprehensive body of evidence.
Connected learning and authoritative references
- How Super Intelligence Works — complete educational curriculum.
- Ranking and Filtering — Why the First Search Result Is Not Always Best.
- Clarifying Questions — When AI Should Ask Instead of Guess.
- Counterexamples — Why One Real Exception Matters.
- Manning, Raghavan and Schütze: Introduction to Information Retrieval — ranked retrieval and relevance evaluation.
- Cochrane Handbook, Chapter 4: Searching for and Selecting Studies — standards for systematic evidence search.
- NIST AI Risk Management Framework — evaluation and proportional evidence requirements.