VIEW THIS AS

Auto mode follows the Route Engine until you choose a viewpoint.

YOU ARE HERE

ROUTE CHECK

CONNECTED TO

WHAT NEXT

Use the canonical route for this room, or HELP if you are unsure.

Super Intelligence | Can AI Make Genuine Discoveries? | Novelty, Replication and Scientific Credit

Three students work together at a classroom table, reviewing open textbooks and notes.

Super Intelligence (SI) cannot be assessed from isolated impressive answers alone. This article examines AI discovery: distinguishing plausible generation from novel, verified and reproducible contributions to knowledge. It preserves the locked Clementi-depth floor with worked cases, repeated-trial logic, verification, diagnostics, recovery and RFE closure.

Search Intent and Direct Answer

The central question in AI discovery is distinguishing plausible generation from novel, verified and reproducible contributions to knowledge. The analysis separates novelty, prior art, evidence, replication, causal contribution, attribution and credit because each can look strong while another remains weak. Super Intelligence (SI) evidence becomes more credible when performance survives this decomposition rather than relying on a single attractive output.

First principles begin with the success condition. Define what “done” means before the system starts. For a question, completion may be one correct answer. For a project, completion includes intermediate state, changing requirements, verification and recovery. A system can be excellent at local steps and still fail the end-to-end objective.

Short tasks hide dependencies. The system can receive all necessary context at once, produce one response and stop. Long tasks require maintaining goals, deciding what to do next, storing state and noticing when earlier assumptions are no longer valid. This is why task horizon is a meaningful axis of capability rather than merely a larger token count.

Definition and Boundary

Short tasks hide dependencies. The system can receive all necessary context at once, produce one response and stop. Long tasks require maintaining goals, deciding what to do next, storing state and noticing when earlier assumptions are no longer valid. This is why task horizon is a meaningful axis of capability rather than merely a larger token count.

Introduce one error midway through a workflow and observe what happens. Does the system notice a contradiction, identify the faulty step, roll back and replan? Or does it confidently build on the error? Recovery tests reveal capabilities that clean benchmarks cannot show. Real environments contain partial failures, so robust SI claims need evidence of repair.

Reliability should be reported as a distribution across repeated attempts. Best-case samples answer what a system can sometimes do. Operational deployment needs to know what it usually does, how bad the tail failures are and whether those failures can be detected. The acceptable threshold depends on consequence and reversibility.

First Principles

Reliability should be reported as a distribution across repeated attempts. Best-case samples answer what a system can sometimes do. Operational deployment needs to know what it usually does, how bad the tail failures are and whether those failures can be detected. The acceptable threshold depends on consequence and reversibility.

Uncertainty is not the same as error. A system can be uncertain and correct, confident and wrong, or appropriately uncertain because the evidence is incomplete. Calibration asks whether confidence tracks actual correctness across many cases. A well-calibrated system makes uncertainty useful for deciding when to verify or escalate.

Abstention can be a capability. When evidence is missing or the problem exceeds the system’s reliable range, saying “I do not know” or requesting additional information may be better than producing a plausible answer. Measure whether abstention occurs on genuinely difficult cases without becoming so frequent that the system is unusable.

Why the Distinction Matters

Abstention can be a capability. When evidence is missing or the problem exceeds the system’s reliable range, saying “I do not know” or requesting additional information may be better than producing a plausible answer. Measure whether abstention occurs on genuinely difficult cases without becoming so frequent that the system is unusable.

Verification should be matched to the claim. Facts can be checked against authoritative sources; calculations recomputed; code tested; proofs formally inspected; scientific claims experimentally evaluated. The stronger the consequence, the more independent the verification should be. Fluency is not verification.

Human baselines must be named. Average users, trained professionals, frontier experts and expert teams answer different questions. The comparison should also state whether humans and AI can use normal tools, how much time each receives, how retries are handled and whether hidden assistance is present.

The Core Measurement Problem

Human baselines must be named. Average users, trained professionals, frontier experts and expert teams answer different questions. The comparison should also state whether humans and AI can use normal tools, how much time each receives, how retries are handled and whether hidden assistance is present.

Cost matters because a system can be slower or more expensive yet achieve a higher maximum score, or slightly weaker but cheap enough to transform routine work. Report quality, time and cost separately. Superiority on one axis should not silently become superiority on all three.

Novelty requires comparison with prior knowledge. A result that is new to the model user may already be established in the literature. Discovery claims should search prior art and define what is new: a hypothesis, proof, method, empirical result or useful connection. Novelty is necessary for discovery but not sufficient.

What a Short Task Hides

Novelty requires comparison with prior knowledge. A result that is new to the model user may already be established in the literature. Discovery claims should search prior art and define what is new: a hypothesis, proof, method, empirical result or useful connection. Novelty is necessary for discovery but not sufficient.

Replication converts an interesting result into stronger knowledge. Another evaluator should be able to reproduce the method or independently confirm the result. In empirical science, replication may require new experiments. In mathematics, formal verification can play a related role. SI discovery claims should survive external checking.

Current evidence shows AI systems contributing to coding, mathematics, scientific modelling, literature analysis and hypothesis generation. These are meaningful research capabilities. They do not imply that every generated idea is novel, correct or independently discovered. Credit should follow the verified contribution.

Worked Example: A Single Answer

Current evidence shows AI systems contributing to coding, mathematics, scientific modelling, literature analysis and hypothesis generation. These are meaningful research capabilities. They do not imply that every generated idea is novel, correct or independently discovered. Credit should follow the verified contribution.

Safety depends on accurate confidence and recovery. A system that knows when to escalate can be safer than one with a slightly higher average score but unpredictable overconfidence. Long-horizon autonomy increases the need for checkpoints because undetected errors have more time to propagate.

Education should train students to verify rather than merely consume AI output. Ask them to identify a claim, locate evidence, estimate confidence, find a counterexample and explain what would change their mind. This turns AI use into evidence practice rather than answer outsourcing.

Worked Example: A Multi-Step Project

Education should train students to verify rather than merely consume AI output. Ask them to identify a claim, locate evidence, estimate confidence, find a counterexample and explain what would change their mind. This turns AI use into evidence practice rather than answer outsourcing.

Organisations should define checkpoints before deployment. Specify when a human must review, what evidence is required, which actions are reversible and what error rate triggers rollback. Record system version and workflow changes because reliability can shift when tools, prompts or models change.

Progress has four stages: answer correctly, remain correct under variation, detect and repair errors, and complete long tasks reliably. This ladder is more informative than a single benchmark percentage. Broad SI would require strength across the ladder and across domains.

Worked Example: An Error Midway

Progress has four stages: answer correctly, remain correct under variation, detect and repair errors, and complete long tasks reliably. This ladder is more informative than a single benchmark percentage. Broad SI would require strength across the ladder and across domains.

RFE closes the loop. Receiver: who needs the result? Function: what complete job must be done? Evidence: what proves the job was completed correctly? Exit: when should the system abstain, escalate or be removed from the workflow? Applied to AI discovery, RFE connects intelligence to dependable outcomes.

The central question in AI discovery is distinguishing plausible generation from novel, verified and reproducible contributions to knowledge. The analysis separates novelty, prior art, evidence, replication, causal contribution, attribution and credit because each can look strong while another remains weak. Super Intelligence (SI) evidence becomes more credible when performance survives this decomposition rather than relying on a single attractive output.

Worked Example: Expert Review

The central question in AI discovery is distinguishing plausible generation from novel, verified and reproducible contributions to knowledge. The analysis separates novelty, prior art, evidence, replication, causal contribution, attribution and credit because each can look strong while another remains weak. Super Intelligence (SI) evidence becomes more credible when performance survives this decomposition rather than relying on a single attractive output.

First principles begin with the success condition. Define what “done” means before the system starts. For a question, completion may be one correct answer. For a project, completion includes intermediate state, changing requirements, verification and recovery. A system can be excellent at local steps and still fail the end-to-end objective.

Short tasks hide dependencies. The system can receive all necessary context at once, produce one response and stop. Long tasks require maintaining goals, deciding what to do next, storing state and noticing when earlier assumptions are no longer valid. This is why task horizon is a meaningful axis of capability rather than merely a larger token count.

Worked Example: Scientific Research

Short tasks hide dependencies. The system can receive all necessary context at once, produce one response and stop. Long tasks require maintaining goals, deciding what to do next, storing state and noticing when earlier assumptions are no longer valid. This is why task horizon is a meaningful axis of capability rather than merely a larger token count.

Introduce one error midway through a workflow and observe what happens. Does the system notice a contradiction, identify the faulty step, roll back and replan? Or does it confidently build on the error? Recovery tests reveal capabilities that clean benchmarks cannot show. Real environments contain partial failures, so robust SI claims need evidence of repair.

Reliability should be reported as a distribution across repeated attempts. Best-case samples answer what a system can sometimes do. Operational deployment needs to know what it usually does, how bad the tail failures are and whether those failures can be detected. The acceptable threshold depends on consequence and reversibility.

Repeated Trials and Reliability

Reliability should be reported as a distribution across repeated attempts. Best-case samples answer what a system can sometimes do. Operational deployment needs to know what it usually does, how bad the tail failures are and whether those failures can be detected. The acceptable threshold depends on consequence and reversibility.

Uncertainty is not the same as error. A system can be uncertain and correct, confident and wrong, or appropriately uncertain because the evidence is incomplete. Calibration asks whether confidence tracks actual correctness across many cases. A well-calibrated system makes uncertainty useful for deciding when to verify or escalate.

Abstention can be a capability. When evidence is missing or the problem exceeds the system’s reliable range, saying “I do not know” or requesting additional information may be better than producing a plausible answer. Measure whether abstention occurs on genuinely difficult cases without becoming so frequent that the system is unusable.

Error Accumulation

Abstention can be a capability. When evidence is missing or the problem exceeds the system’s reliable range, saying “I do not know” or requesting additional information may be better than producing a plausible answer. Measure whether abstention occurs on genuinely difficult cases without becoming so frequent that the system is unusable.

Verification should be matched to the claim. Facts can be checked against authoritative sources; calculations recomputed; code tested; proofs formally inspected; scientific claims experimentally evaluated. The stronger the consequence, the more independent the verification should be. Fluency is not verification.

Human baselines must be named. Average users, trained professionals, frontier experts and expert teams answer different questions. The comparison should also state whether humans and AI can use normal tools, how much time each receives, how retries are handled and whether hidden assistance is present.

State and Memory

Human baselines must be named. Average users, trained professionals, frontier experts and expert teams answer different questions. The comparison should also state whether humans and AI can use normal tools, how much time each receives, how retries are handled and whether hidden assistance is present.

Cost matters because a system can be slower or more expensive yet achieve a higher maximum score, or slightly weaker but cheap enough to transform routine work. Report quality, time and cost separately. Superiority on one axis should not silently become superiority on all three.

Novelty requires comparison with prior knowledge. A result that is new to the model user may already be established in the literature. Discovery claims should search prior art and define what is new: a hypothesis, proof, method, empirical result or useful connection. Novelty is necessary for discovery but not sufficient.

Uncertainty and Confidence

Novelty requires comparison with prior knowledge. A result that is new to the model user may already be established in the literature. Discovery claims should search prior art and define what is new: a hypothesis, proof, method, empirical result or useful connection. Novelty is necessary for discovery but not sufficient.

Replication converts an interesting result into stronger knowledge. Another evaluator should be able to reproduce the method or independently confirm the result. In empirical science, replication may require new experiments. In mathematics, formal verification can play a related role. SI discovery claims should survive external checking.

Current evidence shows AI systems contributing to coding, mathematics, scientific modelling, literature analysis and hypothesis generation. These are meaningful research capabilities. They do not imply that every generated idea is novel, correct or independently discovered. Credit should follow the verified contribution.

Abstention and Escalation

Current evidence shows AI systems contributing to coding, mathematics, scientific modelling, literature analysis and hypothesis generation. These are meaningful research capabilities. They do not imply that every generated idea is novel, correct or independently discovered. Credit should follow the verified contribution.

Safety depends on accurate confidence and recovery. A system that knows when to escalate can be safer than one with a slightly higher average score but unpredictable overconfidence. Long-horizon autonomy increases the need for checkpoints because undetected errors have more time to propagate.

Education should train students to verify rather than merely consume AI output. Ask them to identify a claim, locate evidence, estimate confidence, find a counterexample and explain what would change their mind. This turns AI use into evidence practice rather than answer outsourcing.

Verification and Independent Checks

Education should train students to verify rather than merely consume AI output. Ask them to identify a claim, locate evidence, estimate confidence, find a counterexample and explain what would change their mind. This turns AI use into evidence practice rather than answer outsourcing.

Organisations should define checkpoints before deployment. Specify when a human must review, what evidence is required, which actions are reversible and what error rate triggers rollback. Record system version and workflow changes because reliability can shift when tools, prompts or models change.

Progress has four stages: answer correctly, remain correct under variation, detect and repair errors, and complete long tasks reliably. This ladder is more informative than a single benchmark percentage. Broad SI would require strength across the ladder and across domains.

Human Baselines

Progress has four stages: answer correctly, remain correct under variation, detect and repair errors, and complete long tasks reliably. This ladder is more informative than a single benchmark percentage. Broad SI would require strength across the ladder and across domains.

RFE closes the loop. Receiver: who needs the result? Function: what complete job must be done? Evidence: what proves the job was completed correctly? Exit: when should the system abstain, escalate or be removed from the workflow? Applied to AI discovery, RFE connects intelligence to dependable outcomes.

The central question in AI discovery is distinguishing plausible generation from novel, verified and reproducible contributions to knowledge. The analysis separates novelty, prior art, evidence, replication, causal contribution, attribution and credit because each can look strong while another remains weak. Super Intelligence (SI) evidence becomes more credible when performance survives this decomposition rather than relying on a single attractive output.

Tools, Time and Cost

The central question in AI discovery is distinguishing plausible generation from novel, verified and reproducible contributions to knowledge. The analysis separates novelty, prior art, evidence, replication, causal contribution, attribution and credit because each can look strong while another remains weak. Super Intelligence (SI) evidence becomes more credible when performance survives this decomposition rather than relying on a single attractive output.

First principles begin with the success condition. Define what “done” means before the system starts. For a question, completion may be one correct answer. For a project, completion includes intermediate state, changing requirements, verification and recovery. A system can be excellent at local steps and still fail the end-to-end objective.

Short tasks hide dependencies. The system can receive all necessary context at once, produce one response and stop. Long tasks require maintaining goals, deciding what to do next, storing state and noticing when earlier assumptions are no longer valid. This is why task horizon is a meaningful axis of capability rather than merely a larger token count.

Retries and Selection Effects

Short tasks hide dependencies. The system can receive all necessary context at once, produce one response and stop. Long tasks require maintaining goals, deciding what to do next, storing state and noticing when earlier assumptions are no longer valid. This is why task horizon is a meaningful axis of capability rather than merely a larger token count.

Introduce one error midway through a workflow and observe what happens. Does the system notice a contradiction, identify the faulty step, roll back and replan? Or does it confidently build on the error? Recovery tests reveal capabilities that clean benchmarks cannot show. Real environments contain partial failures, so robust SI claims need evidence of repair.

Reliability should be reported as a distribution across repeated attempts. Best-case samples answer what a system can sometimes do. Operational deployment needs to know what it usually does, how bad the tail failures are and whether those failures can be detected. The acceptable threshold depends on consequence and reversibility.

Novelty and Prior Art

Reliability should be reported as a distribution across repeated attempts. Best-case samples answer what a system can sometimes do. Operational deployment needs to know what it usually does, how bad the tail failures are and whether those failures can be detected. The acceptable threshold depends on consequence and reversibility.

Uncertainty is not the same as error. A system can be uncertain and correct, confident and wrong, or appropriately uncertain because the evidence is incomplete. Calibration asks whether confidence tracks actual correctness across many cases. A well-calibrated system makes uncertainty useful for deciding when to verify or escalate.

Abstention can be a capability. When evidence is missing or the problem exceeds the system’s reliable range, saying “I do not know” or requesting additional information may be better than producing a plausible answer. Measure whether abstention occurs on genuinely difficult cases without becoming so frequent that the system is unusable.

Replication and Reproducibility

Abstention can be a capability. When evidence is missing or the problem exceeds the system’s reliable range, saying “I do not know” or requesting additional information may be better than producing a plausible answer. Measure whether abstention occurs on genuinely difficult cases without becoming so frequent that the system is unusable.

Verification should be matched to the claim. Facts can be checked against authoritative sources; calculations recomputed; code tested; proofs formally inspected; scientific claims experimentally evaluated. The stronger the consequence, the more independent the verification should be. Fluency is not verification.

Human baselines must be named. Average users, trained professionals, frontier experts and expert teams answer different questions. The comparison should also state whether humans and AI can use normal tools, how much time each receives, how retries are handled and whether hidden assistance is present.

What Current Evidence Supports

Human baselines must be named. Average users, trained professionals, frontier experts and expert teams answer different questions. The comparison should also state whether humans and AI can use normal tools, how much time each receives, how retries are handled and whether hidden assistance is present.

Cost matters because a system can be slower or more expensive yet achieve a higher maximum score, or slightly weaker but cheap enough to transform routine work. Report quality, time and cost separately. Superiority on one axis should not silently become superiority on all three.

Novelty requires comparison with prior knowledge. A result that is new to the model user may already be established in the literature. Discovery claims should search prior art and define what is new: a hypothesis, proof, method, empirical result or useful connection. Novelty is necessary for discovery but not sufficient.

What Current Evidence Does Not Establish

Novelty requires comparison with prior knowledge. A result that is new to the model user may already be established in the literature. Discovery claims should search prior art and define what is new: a hypothesis, proof, method, empirical result or useful connection. Novelty is necessary for discovery but not sufficient.

Replication converts an interesting result into stronger knowledge. Another evaluator should be able to reproduce the method or independently confirm the result. In empirical science, replication may require new experiments. In mathematics, formal verification can play a related role. SI discovery claims should survive external checking.

Current evidence shows AI systems contributing to coding, mathematics, scientific modelling, literature analysis and hypothesis generation. These are meaningful research capabilities. They do not imply that every generated idea is novel, correct or independently discovered. Credit should follow the verified contribution.

Connection to Super Intelligence (SI)

Current evidence shows AI systems contributing to coding, mathematics, scientific modelling, literature analysis and hypothesis generation. These are meaningful research capabilities. They do not imply that every generated idea is novel, correct or independently discovered. Credit should follow the verified contribution.

Safety depends on accurate confidence and recovery. A system that knows when to escalate can be safer than one with a slightly higher average score but unpredictable overconfidence. Long-horizon autonomy increases the need for checkpoints because undetected errors have more time to propagate.

Education should train students to verify rather than merely consume AI output. Ask them to identify a claim, locate evidence, estimate confidence, find a counterexample and explain what would change their mind. This turns AI use into evidence practice rather than answer outsourcing.

Safety Implications

Education should train students to verify rather than merely consume AI output. Ask them to identify a claim, locate evidence, estimate confidence, find a counterexample and explain what would change their mind. This turns AI use into evidence practice rather than answer outsourcing.

Organisations should define checkpoints before deployment. Specify when a human must review, what evidence is required, which actions are reversible and what error rate triggers rollback. Record system version and workflow changes because reliability can shift when tools, prompts or models change.

Progress has four stages: answer correctly, remain correct under variation, detect and repair errors, and complete long tasks reliably. This ladder is more informative than a single benchmark percentage. Broad SI would require strength across the ladder and across domains.

Education and Student Use

Progress has four stages: answer correctly, remain correct under variation, detect and repair errors, and complete long tasks reliably. This ladder is more informative than a single benchmark percentage. Broad SI would require strength across the ladder and across domains.

RFE closes the loop. Receiver: who needs the result? Function: what complete job must be done? Evidence: what proves the job was completed correctly? Exit: when should the system abstain, escalate or be removed from the workflow? Applied to AI discovery, RFE connects intelligence to dependable outcomes.

The central question in AI discovery is distinguishing plausible generation from novel, verified and reproducible contributions to knowledge. The analysis separates novelty, prior art, evidence, replication, causal contribution, attribution and credit because each can look strong while another remains weak. Super Intelligence (SI) evidence becomes more credible when performance survives this decomposition rather than relying on a single attractive output.

Organisation Diagnostic Checklist

The central question in AI discovery is distinguishing plausible generation from novel, verified and reproducible contributions to knowledge. The analysis separates novelty, prior art, evidence, replication, causal contribution, attribution and credit because each can look strong while another remains weak. Super Intelligence (SI) evidence becomes more credible when performance survives this decomposition rather than relying on a single attractive output.

First principles begin with the success condition. Define what “done” means before the system starts. For a question, completion may be one correct answer. For a project, completion includes intermediate state, changing requirements, verification and recovery. A system can be excellent at local steps and still fail the end-to-end objective.

Short tasks hide dependencies. The system can receive all necessary context at once, produce one response and stop. Long tasks require maintaining goals, deciding what to do next, storing state and noticing when earlier assumptions are no longer valid. This is why task horizon is a meaningful axis of capability rather than merely a larger token count.

Progress Ladder

Short tasks hide dependencies. The system can receive all necessary context at once, produce one response and stop. Long tasks require maintaining goals, deciding what to do next, storing state and noticing when earlier assumptions are no longer valid. This is why task horizon is a meaningful axis of capability rather than merely a larger token count.

Introduce one error midway through a workflow and observe what happens. Does the system notice a contradiction, identify the faulty step, roll back and replan? Or does it confidently build on the error? Recovery tests reveal capabilities that clean benchmarks cannot show. Real environments contain partial failures, so robust SI claims need evidence of repair.

Reliability should be reported as a distribution across repeated attempts. Best-case samples answer what a system can sometimes do. Operational deployment needs to know what it usually does, how bad the tail failures are and whether those failures can be detected. The acceptable threshold depends on consequence and reversibility.

Adversarial Test

Reliability should be reported as a distribution across repeated attempts. Best-case samples answer what a system can sometimes do. Operational deployment needs to know what it usually does, how bad the tail failures are and whether those failures can be detected. The acceptable threshold depends on consequence and reversibility.

Uncertainty is not the same as error. A system can be uncertain and correct, confident and wrong, or appropriately uncertain because the evidence is incomplete. Calibration asks whether confidence tracks actual correctness across many cases. A well-calibrated system makes uncertainty useful for deciding when to verify or escalate.

Abstention can be a capability. When evidence is missing or the problem exceeds the system’s reliable range, saying “I do not know” or requesting additional information may be better than producing a plausible answer. Measure whether abstention occurs on genuinely difficult cases without becoming so frequent that the system is unusable.

Recovery Test

Abstention can be a capability. When evidence is missing or the problem exceeds the system’s reliable range, saying “I do not know” or requesting additional information may be better than producing a plausible answer. Measure whether abstention occurs on genuinely difficult cases without becoming so frequent that the system is unusable.

Verification should be matched to the claim. Facts can be checked against authoritative sources; calculations recomputed; code tested; proofs formally inspected; scientific claims experimentally evaluated. The stronger the consequence, the more independent the verification should be. Fluency is not verification.

Human baselines must be named. Average users, trained professionals, frontier experts and expert teams answer different questions. The comparison should also state whether humans and AI can use normal tools, how much time each receives, how retries are handled and whether hidden assistance is present.

RFE Closure

Human baselines must be named. Average users, trained professionals, frontier experts and expert teams answer different questions. The comparison should also state whether humans and AI can use normal tools, how much time each receives, how retries are handled and whether hidden assistance is present.

Cost matters because a system can be slower or more expensive yet achieve a higher maximum score, or slightly weaker but cheap enough to transform routine work. Report quality, time and cost separately. Superiority on one axis should not silently become superiority on all three.

Novelty requires comparison with prior knowledge. A result that is new to the model user may already be established in the literature. Discovery claims should search prior art and define what is new: a hypothesis, proof, method, empirical result or useful connection. Novelty is necessary for discovery but not sufficient.

Frequently Asked Questions

Novelty requires comparison with prior knowledge. A result that is new to the model user may already be established in the literature. Discovery claims should search prior art and define what is new: a hypothesis, proof, method, empirical result or useful connection. Novelty is necessary for discovery but not sufficient.

Replication converts an interesting result into stronger knowledge. Another evaluator should be able to reproduce the method or independently confirm the result. In empirical science, replication may require new experiments. In mathematics, formal verification can play a related role. SI discovery claims should survive external checking.

Current evidence shows AI systems contributing to coding, mathematics, scientific modelling, literature analysis and hypothesis generation. These are meaningful research capabilities. They do not imply that every generated idea is novel, correct or independently discovered. Credit should follow the verified contribution.

Continue the Super Intelligence (SI) Series

Current evidence shows AI systems contributing to coding, mathematics, scientific modelling, literature analysis and hypothesis generation. These are meaningful research capabilities. They do not imply that every generated idea is novel, correct or independently discovered. Credit should follow the verified contribution.

Safety depends on accurate confidence and recovery. A system that knows when to escalate can be safer than one with a slightly higher average score but unpredictable overconfidence. Long-horizon autonomy increases the need for checkpoints because undetected errors have more time to propagate.

Education should train students to verify rather than merely consume AI output. Ask them to identify a claim, locate evidence, estimate confidence, find a counterexample and explain what would change their mind. This turns AI use into evidence practice rather than answer outsourcing.

End-to-End Reliability Matrix

Build an end-to-end matrix with project stages down the rows and success, detection, recovery and verification across the columns. A local step can score highly while the overall project remains fragile because one undetected error propagates. This matrix exposes where reliability is lost and where an additional checkpoint creates the most value. Super Intelligence (SI) claims should increasingly be tested at the complete-work level.

A calibration matrix groups outputs by stated confidence and compares that confidence with observed correctness. If answers labelled highly confident are not more accurate than medium-confidence answers, the confidence signal is weak. If low-confidence cases are genuinely harder, escalation becomes useful. Calibration should be measured on representative tasks rather than inferred from confident language.

Freeze comparison conditions before testing. Define the task, human baseline, tools, time, retry policy, scoring rule and assistance available to each side. Then run enough cases to estimate variation. Changing the rules after seeing who wins produces a story, not a fair comparison. Transparent conditions allow readers to interpret both human and AI strengths without pretending one number captures everything.

A discovery protocol has four gates. Gate one: novelty against prior art. Gate two: validity through proof, experiment or appropriate evidence. Gate three: replication or independent confirmation. Gate four: causal contribution—what did the AI actually add relative to humans, tools and existing knowledge? Passing all four supports a stronger discovery claim than merely generating an interesting idea.

Calibration Matrix: Confidence Versus Correctness

A calibration matrix groups outputs by stated confidence and compares that confidence with observed correctness. If answers labelled highly confident are not more accurate than medium-confidence answers, the confidence signal is weak. If low-confidence cases are genuinely harder, escalation becomes useful. Calibration should be measured on representative tasks rather than inferred from confident language.

Freeze comparison conditions before testing. Define the task, human baseline, tools, time, retry policy, scoring rule and assistance available to each side. Then run enough cases to estimate variation. Changing the rules after seeing who wins produces a story, not a fair comparison. Transparent conditions allow readers to interpret both human and AI strengths without pretending one number captures everything.

A discovery protocol has four gates. Gate one: novelty against prior art. Gate two: validity through proof, experiment or appropriate evidence. Gate three: replication or independent confirmation. Gate four: causal contribution—what did the AI actually add relative to humans, tools and existing knowledge? Passing all four supports a stronger discovery claim than merely generating an interesting idea.

Failure injection deliberately introduces a wrong intermediate result, missing source, broken tool or changed requirement. Observe whether the system detects the problem, localises it and repairs the workflow. This is more realistic than testing only clean success paths. Long-horizon systems will encounter failure; the relevant question is whether failure becomes drift or recovery.

Fair Comparison Protocol: Freeze Conditions First

Freeze comparison conditions before testing. Define the task, human baseline, tools, time, retry policy, scoring rule and assistance available to each side. Then run enough cases to estimate variation. Changing the rules after seeing who wins produces a story, not a fair comparison. Transparent conditions allow readers to interpret both human and AI strengths without pretending one number captures everything.

A discovery protocol has four gates. Gate one: novelty against prior art. Gate two: validity through proof, experiment or appropriate evidence. Gate three: replication or independent confirmation. Gate four: causal contribution—what did the AI actually add relative to humans, tools and existing knowledge? Passing all four supports a stronger discovery claim than merely generating an interesting idea.

Failure injection deliberately introduces a wrong intermediate result, missing source, broken tool or changed requirement. Observe whether the system detects the problem, localises it and repairs the workflow. This is more realistic than testing only clean success paths. Long-horizon systems will encounter failure; the relevant question is whether failure becomes drift or recovery.

The workbook asks the reader to take one claim and document seven fields: task, comparator, conditions, result, uncertainty, verification and receiver outcome. Then add one adversarial case and one recovery case. Finally, state what evidence would justify a stronger claim. This creates an auditable chain from demonstration to conclusion.

Discovery Protocol: Novelty to Replication

A discovery protocol has four gates. Gate one: novelty against prior art. Gate two: validity through proof, experiment or appropriate evidence. Gate three: replication or independent confirmation. Gate four: causal contribution—what did the AI actually add relative to humans, tools and existing knowledge? Passing all four supports a stronger discovery claim than merely generating an interesting idea.

Failure injection deliberately introduces a wrong intermediate result, missing source, broken tool or changed requirement. Observe whether the system detects the problem, localises it and repairs the workflow. This is more realistic than testing only clean success paths. Long-horizon systems will encounter failure; the relevant question is whether failure becomes drift or recovery.

The workbook asks the reader to take one claim and document seven fields: task, comparator, conditions, result, uncertainty, verification and receiver outcome. Then add one adversarial case and one recovery case. Finally, state what evidence would justify a stronger claim. This creates an auditable chain from demonstration to conclusion.

Dependability is itself a capability. A system that produces brilliant outputs but cannot sustain state, express useful uncertainty or recover from mistakes is different from one that is slightly less spectacular locally but consistently completes the real job. Broad SI evaluation should reward the latter properties because real intelligence operates through time, uncertainty and consequence.

Failure Injection: Test Recovery, Not Just Success

Failure injection deliberately introduces a wrong intermediate result, missing source, broken tool or changed requirement. Observe whether the system detects the problem, localises it and repairs the workflow. This is more realistic than testing only clean success paths. Long-horizon systems will encounter failure; the relevant question is whether failure becomes drift or recovery.

The workbook asks the reader to take one claim and document seven fields: task, comparator, conditions, result, uncertainty, verification and receiver outcome. Then add one adversarial case and one recovery case. Finally, state what evidence would justify a stronger claim. This creates an auditable chain from demonstration to conclusion.

Dependability is itself a capability. A system that produces brilliant outputs but cannot sustain state, express useful uncertainty or recover from mistakes is different from one that is slightly less spectacular locally but consistently completes the real job. Broad SI evaluation should reward the latter properties because real intelligence operates through time, uncertainty and consequence.

Build an end-to-end matrix with project stages down the rows and success, detection, recovery and verification across the columns. A local step can score highly while the overall project remains fragile because one undetected error propagates. This matrix exposes where reliability is lost and where an additional checkpoint creates the most value. Super Intelligence (SI) claims should increasingly be tested at the complete-work level.

Practical Workbook: Audit One Complete Claim

The workbook asks the reader to take one claim and document seven fields: task, comparator, conditions, result, uncertainty, verification and receiver outcome. Then add one adversarial case and one recovery case. Finally, state what evidence would justify a stronger claim. This creates an auditable chain from demonstration to conclusion.

Dependability is itself a capability. A system that produces brilliant outputs but cannot sustain state, express useful uncertainty or recover from mistakes is different from one that is slightly less spectacular locally but consistently completes the real job. Broad SI evaluation should reward the latter properties because real intelligence operates through time, uncertainty and consequence.

Build an end-to-end matrix with project stages down the rows and success, detection, recovery and verification across the columns. A local step can score highly while the overall project remains fragile because one undetected error propagates. This matrix exposes where reliability is lost and where an additional checkpoint creates the most value. Super Intelligence (SI) claims should increasingly be tested at the complete-work level.

A calibration matrix groups outputs by stated confidence and compares that confidence with observed correctness. If answers labelled highly confident are not more accurate than medium-confidence answers, the confidence signal is weak. If low-confidence cases are genuinely harder, escalation becomes useful. Calibration should be measured on representative tasks rather than inferred from confident language.

Final Synthesis: Dependability Is a Capability

Dependability is itself a capability. A system that produces brilliant outputs but cannot sustain state, express useful uncertainty or recover from mistakes is different from one that is slightly less spectacular locally but consistently completes the real job. Broad SI evaluation should reward the latter properties because real intelligence operates through time, uncertainty and consequence.

Build an end-to-end matrix with project stages down the rows and success, detection, recovery and verification across the columns. A local step can score highly while the overall project remains fragile because one undetected error propagates. This matrix exposes where reliability is lost and where an additional checkpoint creates the most value. Super Intelligence (SI) claims should increasingly be tested at the complete-work level.

A calibration matrix groups outputs by stated confidence and compares that confidence with observed correctness. If answers labelled highly confident are not more accurate than medium-confidence answers, the confidence signal is weak. If low-confidence cases are genuinely harder, escalation becomes useful. Calibration should be measured on representative tasks rather than inferred from confident language.

Freeze comparison conditions before testing. Define the task, human baseline, tools, time, retry policy, scoring rule and assistance available to each side. Then run enough cases to estimate variation. Changing the rules after seeing who wins produces a story, not a fair comparison. Transparent conditions allow readers to interpret both human and AI strengths without pretending one number captures everything.


Discovery Requires More Than Producing Something New

A generated idea can be novel and still be wrong, trivial or scientifically irrelevant. Scientific discovery requires a stronger chain: the system identifies or helps identify a previously unknown or unrecognised relationship, produces a testable claim, and that claim survives appropriate verification. In many sciences, experimental replication remains the decisive step.

For Super Intelligence (SI), this means “AI generated a hypothesis” and “AI made a discovery” should not be used interchangeably.

Novelty Must Be Checked Against Prior Knowledge

An idea may appear new to the model user while already existing in an obscure paper. Strong discovery systems therefore need literature search, provenance and expert review to establish novelty. This is especially difficult in fields with millions of publications.

Co-Scientist addresses this partly by grounding hypotheses in literature and comparing them against prior work, but its own Nature paper calls for broader validation across disciplines and stronger evaluation methods.

Experimental Validation Changes the Status of an AI Hypothesis

Google DeepMind’s Co-Scientist provides a useful 2026 example. The Nature paper reports AI-assisted hypotheses in biomedical problems, including drug-repurposing and combination-therapy candidates, with some findings validated through laboratory experiments. That is stronger evidence than a generated scientific narrative because the hypothesis encountered an external empirical test.

The system is still described as collaborative and expert-in-the-loop. The evidence supports AI-assisted discovery, not the claim that human science has become unnecessary.

AlphaEvolve Shows a Different Kind of Discovery: Verifiable Algorithms

DeepMind’s AlphaEvolve combines language-model proposal generation with automated evaluators and evolutionary search. In mathematics and computer science, many candidate algorithms can be executed or formally checked, making discovery unusually amenable to automation. DeepMind reports new or improved solutions to open mathematical problems and algorithms deployed inside Google’s computing infrastructure.

This is an important pattern for SI: discovery accelerates when novelty generation is paired with a fast, objective evaluator.

Deployment Is an Extra Layer of Evidence

An algorithm that survives production use has passed a different test from one that only improves a benchmark. DeepMind reports AlphaEvolve-derived optimisations deployed in data-centre scheduling, storage systems and TPU-related design work. Production impact does not prove general scientific intelligence, but it strengthens the claim that the system found useful, nontrivial improvements.

For discovery evaluation, “used successfully by others” is a valuable evidence layer after novelty and verification.

The AI Scientist Demonstrates End-to-End Research Automation—but Peer Review Is Not Replication

The AI Scientist work published in Nature in 2026 automates idea generation, coding, experiments, analysis and manuscript writing within machine learning. Its associated project also demonstrated an AI-generated paper passing a workshop peer-review process. That is historically significant for research automation.

But peer review assesses plausibility and quality; it does not make every conclusion true. Nature’s accompanying commentary emphasised that AI-generated research still raises questions about quality, institutional adaptation and the difference between generating papers and generating important knowledge.

Negative Results Can Also Be Genuine Scientific Output

A discovery system should not be rewarded only for positive findings. Demonstrating that a plausible technique fails under controlled conditions can be scientifically useful, especially if the result eliminates a branch of the search space. The AI Scientist Nature paper itself centres on automating a research lifecycle rather than presenting a spectacular positive scientific breakthrough.

SI science should value information gain, not sensational results.

Replication Separates a Finding From an Anecdote

A result becomes stronger when independent researchers can reproduce it using the described method. AI can create a new challenge here: automated systems may generate papers faster than the scientific community can replicate them. If generation throughput outruns verification throughput, the literature can fill with plausible but weakly checked findings.

Discovery rate should therefore be measured as verified knowledge produced per unit time, not papers produced per unit time.

Scientific Credit Has Several Layers

Who should receive credit when an AI system proposes an idea, a human defines the question, another team runs the experiment and an institution supplies the tools? Authorship, inventorship, intellectual contribution and legal rights are related but not identical. Journal policies and law continue to evolve by jurisdiction and institution.

A factual account should therefore describe contributions explicitly rather than assuming that one label—“the AI discovered it” or “the human discovered it”—captures the whole research process.

Worked Example: A Drug Repurposing Hypothesis

An AI finds a connection between a known drug and a disease mechanism. Stage one is novelty checking: is the connection absent from prior literature? Stage two is mechanistic plausibility. Stage three is experimental testing in cells or models. Stage four may involve clinical research. The claim becomes stronger at each stage.

Calling stage one a completed medical discovery would overstate the evidence; refusing to call an experimentally validated novel hypothesis a meaningful AI-assisted discovery would understate it.

Worked Example: A New Matrix Multiplication Algorithm

An AI system proposes code implementing a lower-operation-count algorithm. The program can be executed and mathematically checked. Other researchers can reproduce the count. Here the verification chain can be much shorter than in biomedicine because correctness is more formal.

This explains why automated discovery may advance unevenly across disciplines: the speed of the evaluator changes the speed of trustworthy discovery.

What Would Count as Strong Evidence of Autonomous Scientific Discovery?

Stronger evidence would show a system independently selecting important problems, producing genuinely novel hypotheses, designing decisive experiments, interpreting unexpected results, surviving replication across laboratories and generating findings that experts subsequently build on. It should do this repeatedly across more than one narrow field.

That standard is substantially higher than producing a paper or passing peer review once.

RFE Closure: Discovery Ends With New Verified Knowledge

The problem is conflating novelty, publication and truth. The function of AI scientific discovery is to expand reliable knowledge faster or more effectively. The receiver is the scientific community and, eventually, the people who benefit from the resulting understanding or technology. Evidence includes novelty checks, experimental validation, replication, independent use and durable scientific impact.

The exit condition is to withdraw or downgrade a claimed discovery when replication fails, prior art is found or the result depends on an evaluation artefact. SI science should make correction easier, not make claims harder to unwind.

Continue the Super Intelligence (SI) Evidence Series

Next: interpretability—what we can and cannot learn about the internal mechanisms that produce advanced AI behaviour.