AGI and ASI are proposed levels of artificial intelligence, not labels automatically earned by impressive benchmark scores. Artificial general intelligence concerns broad, flexible capability under a chosen definition; artificial superintelligence generally describes a further hypothetical level exceeding human capability across many domains. Performance, generality and autonomy need separate evidence.
This chapter tests claims about recursive self-improvement using a fictional four-version research laboratory. The standard benchmark climbs from 70% to 86%, but transfer performance declines after an initial improvement. Students calculate percentage points, assess a bounded mathematical feedback model and learn why model-assisted development does not prove inevitable self-improvement.
It fills topic 099 in How Super Intelligence Works, which uses Super Intelligence as an editorial umbrella rather than a claim that existing tools possess artificial superintelligence.
Contents and evidence routes
Definitions · Recursive improvement · Lantern Lab · Work the scores · Bounded feedback example · Independent transfer · Sources
Chapters 1–9: 1. Artificial intelligence, AGI and ASI are different claims · 2. Definitions need dimensions, not one impressive answer · 3. Autonomy is about permitted action, not intelligence alone · 4. The jagged capability problem · 5. Recursive improvement is a proposed feedback process · 6. The fictional Lantern Research Iteration Lab · 7. Calculate what improved on the standard test · 8. Now calculate the independent transfer results · 9. A high benchmark score cannot define AGI by itself
Chapters 10–18: 10. Development can be assisted without being autonomous · 11. Not every proposed revision survives validation · 12. The test set should not become the training target · 13. Feedback loops do not necessarily accelerate · 14. A mathematical growth example is not a forecast · 15. A separate timing model is also only a model · 16. Synthetic data can help and can also mislead · 17. Reward objectives can be incomplete proxies · 18. Capabilities and real-world utility are different
Chapters 19–28: 19. Autonomous agents can make safety consequences larger · 20. Recursive self-improvement is a contested future scenario · 21. A proposed intelligence explosion is not an observed event · 22. The human decision remains ethically important · 23. Independent transfer: a fourth version appears · 24. Explain the transfer answer without invented causation · 25. What would count as stronger evidence of recursive progress? · 26. A learning activity in evidence literacy · 27. Frequently asked questions · 28. Authoritative sources and the final curriculum route
1. Artificial intelligence, AGI and ASI are different claims
Artificial intelligence is a broad label for computational systems designed to perform tasks associated with perception, language, learning, search or decision support. A system can be excellent at one task while having little competence in another. General-purpose models work across many task families, but that breadth alone does not settle the contested definition of artificial general intelligence.
Artificial general intelligence, or AGI, is commonly used for hypothetical or proposed systems with broad, flexible capabilities across many cognitive domains. Artificial superintelligence, or ASI, generally refers to a further hypothetical degree of capability exceeding human performance across a very wide range of relevant domains. No single globally binding metric establishes when those labels apply.
The word ‘Super Intelligence’ in eduKateSG’s series title is editorial. It is not a declaration that current models have achieved AGI or ASI.
2. Definitions need dimensions, not one impressive answer
One model may outperform specialists on certain mathematics or coding tests yet fail unfamiliar physical tasks, source checking or sustained workflows. Another may act independently through tools without matching a human’s reasoning quality. Performance, generality and autonomy are related but different dimensions.
Google DeepMind’s Levels of AGI paper, published in ICML 2024, proposes assessing depth of performance, breadth of abilities and related autonomy considerations. It is a research framework, not a universally accepted official certification of AGI.
When evaluating a headline, ask which abilities were tested, on what tasks, under what conditions and how much human support was involved. A score cannot stand for capabilities that were never measured.
3. Autonomy is about permitted action, not intelligence alone
A system may solve difficult tasks while having no access to files or external tools. Another may execute several simple tasks autonomously because an application has granted it broad permissions. Neither observation, by itself, proves the system possesses general human-level reasoning.
Autonomy also creates safety and accountability questions. A competent assistant should not be allowed to publish, transfer funds or access private data merely because it scores well in an unrelated benchmark. Authorised tool scope belongs to the organisation’s controls.
The Capability versus Autonomy article explains why what a model can do is distinct from what it should be allowed to do.
4. The jagged capability problem
A leading system can perform strongly on demanding standardised evaluations while making basic mistakes on different tasks. This uneven profile is often described as jagged capability. Generality cannot be inferred from a small selection of impressive examples chosen after seeing the results.
The International AI Safety Report 2026 describes major progress and persistent failures in reliability, long-horizon work and unfamiliar contexts. The report also stresses uncertainties about future advances and the effectiveness of safeguards.
An honest article keeps both observations: AI capability is substantial in some areas, and current evidence does not establish reliable performance on every new situation.
5. Recursive improvement is a proposed feedback process
An AI-assisted development loop can propose a code change, run authorised tests, evaluate outcomes and retain a verified improvement. A later version might help propose further changes. When improved systems contribute to the process of designing their successors, people sometimes discuss recursive self-improvement.
The label does not mean every iteration will work or that no humans are needed. Progress requires useful experiments, data, compute, source control, evaluation and permission to deploy. An error in any stage can halt the loop or produce a regression. A change can improve a narrow benchmark while harming a more important task.
Treat recursion as a process to examine, not a guaranteed mathematical route to unlimited capability.
6. The fictional Lantern Research Iteration Lab
Our teaching fixture is an imaginary research assistant used to improve an equally fictional question-answering tool. A protected test contains one hundred standard tasks and a separate set of twenty unfamiliar transfer tasks. Version V1 correctly completes seventy of the hundred standard cases and fourteen of the twenty transfer cases.
After an approved development iteration, V2 completes seventy-eight standard cases and fifteen transfer cases. A further iteration produces V3, with eighty-two standard successes but only thirteen transfer successes. All versions, changes and tasks are invented; no actual model was trained or evaluated.
The learner’s goal is to inspect both task distributions and identify whether successive improvements are consistent across domains.
7. Calculate what improved on the standard test
On the one-hundred-item test, V1 scores 70%, V2 scores 78% and V3 scores 82%. The first update adds eight percentage points; the second adds four. Saying that the score improved by eight percent in the first update would confuse percentage points with relative percentage change.
The relative improvement from 70 to 78 is (78−70)/70 ≈ 11.4%. The relative improvement from 78 to 82 is 4/78 ≈ 5.1%. Both changes are positive on this particular benchmark, but the later marginal gain is smaller.
These are paper calculations. They do not establish a universal law of AI progress or the existence of self-improvement beyond the invented procedure.
8. Now calculate the independent transfer results
Across twenty unfamiliar cases, V1 scores 14/20 = 70%, V2 scores 15/20 = 75%, and V3 scores 13/20 = 65%. The second iteration produces a regression of ten percentage points relative to V2 on these transfer cases.
Version V3 is best on the original standard set while worst on the independent transfer set. That is the core teaching contrast. A developer who reports only the 82% figure might imply broad progress without mentioning a meaningful weakness in the more demanding or different domain.
An appropriate release decision depends on the actual use, the importance of the transfer cases and an acceptance criterion established before testing.
9. A high benchmark score cannot define AGI by itself
The standard test may cover one familiar task type, while the transfer set covers another. Even perfect performance on both would not establish competence across the very wide range of tasks a general intelligence definition might require.
A claim about AGI or ASI needs a specified capability definition and evidence that bears on breadth, depth, independence and reliability. Without that bridge, a single benchmark improvement remains a benchmark improvement.
The Benchmarks chapter explains why rankings reverse when task mixtures change.
10. Development can be assisted without being autonomous
Imagine a human researcher uses V1 to generate possible optimisation ideas, checks code changes, runs a controlled experiment and approves V2. V2 then proposes ideas for V3, which the researcher evaluates. A model has contributed to successive improvements, but the humans and testing infrastructure still own the selection and deployment decisions.
It would be inaccurate to call that a fully autonomous, unchecked process if a person controlled all consequential steps. It would also be inaccurate to dismiss the assistance merely because it was supervised. Describe exactly which parts the AI performed and which parts remain human responsibilities.
Claims of independence require evidence of actual permissions and observed actions.
11. Not every proposed revision survives validation
A generated optimisation idea can introduce a software bug, make training less stable, reduce performance on rare tasks or use more resources than it saves. A loop that accepts every proposed modification without independent tests can degrade quickly.
An effective improvement process records candidate changes, expected benefits, test results and reasons for acceptance or rejection. Keeping an earlier known-good version allows recovery when an apparent improvement produces unexpected failures.
Version control is an accountability mechanism; it is not proof that the latest model is universally better.
12. The test set should not become the training target
If developers repeatedly inspect a fixed public benchmark and revise a system to raise its score, the benchmark becomes part of the development process. Apparent improvement may reflect familiarity with that particular distribution rather than greater general capability.
Maintain appropriate held-out and changed-context tasks, document overlap with development data and look for whether a proposed capability transfers beyond the examples used to tune it. No fixed set can prove all possible abilities.
Our invented V3 illustrates the principle: its original benchmark improves while a separate test deteriorates.
13. Feedback loops do not necessarily accelerate
A feedback process can yield diminishing gains, stagnate or regress. The first improvement may fix obvious errors, while later issues require better data, architecture, reasoning or access to the world. Improvements can become harder and more expensive, even as the model helps researchers.
Limits can arise from training-data quality, compute, experiment cost, evaluator reliability, memory, software, physical measurements and the ability to identify genuinely useful changes. None of these constraints disappears simply because an agent can generate its own proposals.
A story of endless accelerating improvement is a hypothesis requiring much stronger evidence than one or two successful development cycles.
14. A mathematical growth example is not a forecast
Consider an entirely invented recurrence p_next = p + 0.2(1−p) for an abstract score p between zero and one. Starting from p=0.60, one update gives 0.68. Another gives 0.744, and a third gives 0.7952. The gap to one shrinks at each step.
This equation was chosen to demonstrate bounded arithmetic. It does not model any actual AI development programme or predict that capability will approach perfect performance. In the real world, scores may change non-monotonically as tasks become harder, costs increase or evaluation conditions change.
Do not turn a useful algebra exercise into a timeline for AGI or ASI.
15. A separate timing model is also only a model
Suppose a fictional engineer requires eight hours to evaluate each candidate revision. Three candidates need twenty-four evaluator-hours under the simplified rule, regardless of how quickly a model generated their code. Reducing idea-generation time does not automatically remove the review bottleneck.
If more candidates are proposed than staff or computing capacity can test, the queue grows. Selecting better experiments can help, but the benefit requires appropriate evidence and cannot be inferred from raw numbers of generated proposals.
This connects recursive improvement to institutional review and infrastructure constraints rather than treating it as disembodied intelligence.
16. Synthetic data can help and can also mislead
A model may generate practice data or candidate examples to improve another version. Synthetic examples can be useful when their labels, diversity and relevance are checked. But repeating unverified outputs as training targets may propagate errors and reduce coverage of unfamiliar cases.
The earlier Synthetic Data and Learning Bottlenecks article explores this concern. The general lesson is that new text volume does not equal new verified evidence.
Recursive improvement requires feedback from reliable external outcomes, not only agreement between versions of the same model.
17. Reward objectives can be incomplete proxies
Developers may choose a score because it is convenient to measure. A system could learn to optimise that number while losing performance on the actual user’s goal. That is not proof of strategic intention; it can arise from ordinary optimisation against an incomplete objective.
The fictional V3 improves the visible standard-test score while doing worse on the transfer set. A responsible development programme should predeclare important evaluation dimensions and avoid moving the goalposts after inspecting the new model.
Alignment and oversight remain relevant even when a capability metric improves.
18. Capabilities and real-world utility are different
A system could solve specialised problems under controlled conditions yet fail in a workplace that uses unfamiliar interfaces, contradictory records or delayed external tools. General performance claims require evaluation on the actual task and environment rather than extrapolation from abstract puzzle scores.
The International AI Safety Report 2026 discusses this evaluation gap and persistent issues with agent reliability. It does not provide a simple rule for converting current benchmark performance into a guaranteed future level of general intelligence.
The practical question for a school or organisation remains whether a specific version can reliably and appropriately perform the specific authorised job.
19. Autonomous agents can make safety consequences larger
An agent that can read, write and call tools across several steps can create real consequences when it misinterprets a source or acts on a fabricated premise. More capable models can make such workflows useful, but tool authority must still be bounded and observed.
Permissions, sandboxes, approval and independent monitoring can reduce certain risks. None is a universal guarantee, and controls should be tested against the actual model and application. The Alignment and Human Control chapter develops those responsibilities.
Progress in autonomy should not be confused with an automatic moral permission to act.
20. Recursive self-improvement is a contested future scenario
Some researchers expect increasingly capable AI to accelerate science and engineering, possibly including the design of future AI systems. Others emphasise limits from experimentation, data, compute and organisational deployment. Forecasts can differ dramatically because the underlying development pathways remain uncertain.
The International AI Safety Report 2026 describes serious uncertainty about the trajectory through 2030 and distinguishes present capability evidence from scenarios of future risk. It does not establish that a particular takeoff speed or recursive explosion is inevitable.
Report competing perspectives as uncertainty in evidence and assumptions, not as a reason to present speculation as confirmed fact.
21. A proposed intelligence explosion is not an observed event
An intelligence explosion is a hypothesis that improvements in a system’s design capabilities might lead to successively more powerful improvements. It is not established simply because one model writes code that helps improve another system. The process would have to repeatedly produce real, validated gains and overcome the relevant constraints.
Do not equate every new model release, larger training run or higher benchmark number with a self-directed recursive chain. The origin of improvements may include human ideas, hardware, new data and changed evaluation procedures.
A careful article describes the observed development process before choosing dramatic labels.
22. The human decision remains ethically important
Even if future systems become much more capable, questions about which goals should be pursued, what risks are acceptable and who has authority to deploy them are not answered by a capability score. Legitimate institutions and affected people have roles that cannot be replaced by the claim that a model is intelligent.
Accountability requires traceable decisions, appropriate oversight, ability to correct harms and clear limits on private information and external actions. In high-stakes settings, a system’s apparent competence may increase rather than remove the need for institutional safeguards.
Responsible policy must distinguish technical possibility from social permission.
23. Independent transfer: a fourth version appears
Extend the fictional Lantern Lab. Suppose V4 completes eighty-six of the hundred familiar tasks but only twelve of the twenty unfamiliar tasks. Its familiar-task score is 86%, four percentage points above V3. Its transfer score is 12/20 = 60%, five percentage points below V3.
If the deployment requires at least 70% transfer success, V4 fails that stipulated gate even though its headline benchmark improves. A developer who promotes it solely because 86 exceeds 82 has ignored the predeclared task requirement.
This changed-context exercise tests whether the student can apply a rule to new numbers rather than repeat the earlier conclusion.
24. Explain the transfer answer without invented causation
The visible scores are V1 70/70, V2 78/75, V3 82/65 and V4 86/60, where the first number is familiar-task percentage and the second is transfer-task percentage. Familiar-task performance climbs; transfer performance rises once and then falls.
The packet does not identify what code or training changes caused these patterns. We therefore cannot conclude that model self-training caused overfitting or that an algorithm strategically manipulated the test. Those would be new causal claims requiring a proper development trace.
The right conclusion is limited to the observed-within-fixture comparison and the resulting failure to meet the stated release condition.
25. What would count as stronger evidence of recursive progress?
Researchers would need an auditable sequence showing which AI system proposed each change, what tools and data it used, how changes were validated on independent tasks, what human role remained and how resource and safety costs changed. A claimed improvement should survive fresh evaluations beyond its development set.
Repeatedly improving one benchmark through human-guided prompt edits can be useful engineering, but it is not a demonstration of unrestricted recursive self-improvement. Evidence for generality and autonomy must be presented separately.
A high-quality report makes its level of certainty explicit and tells readers which next test could falsify the proposed explanation.
26. A learning activity in evidence literacy
Have students create a three-column ledger labelled demonstrated, calculated and hypothetical. Place the Lantern V1–V4 scores under calculated from fictional inputs. Place the notion of inevitable unlimited improvement under hypothetical. Put the dated source statements about current AI capabilities under demonstrated by the cited study within its stated evaluation limits.
Then ask learners to propose a new independent test, identify what outcome would reverse their conclusion and explain which human or technical approvals remain necessary. The exercise teaches scientific reasoning and responsible engineering rather than excitement about a label.
The strongest learning transfer comes when the student can preserve these distinctions in an unfamiliar technology claim.
27. Frequently asked questions
Has AI achieved AGI? Definitions and thresholds remain contested; performance on selected tasks is not a universally accepted certificate. Avoid claims of a definitive threshold without a specified measurement standard.
Does AGI automatically mean ASI? No. The terms refer to different proposed levels of capability and cannot be used interchangeably.
Does AI writing better code prove recursive self-improvement? No. A validated system development loop involves source control, tests, independent evaluation, deployment and often human participation.
Will intelligence inevitably grow exponentially? No such general law follows from the mechanisms discussed. Future trajectories are uncertain.
Applied improvement audit: independent evidence, bottlenecks and decision gates
A change must have a traceable causal owner
When a later version performs better, the team should record what changed: training data, prompt, code, model weights, tools, hardware or evaluation conditions. Several factors can move simultaneously, making it difficult to attribute the gain to a particular improvement proposed by AI.
A recursive-improvement claim needs more than a chronological sequence of higher numbers. It must identify the agent’s actual contribution and distinguish it from human research, new resources and a revised test.
Data contamination can imitate general progress
If a benchmark has been used repeatedly to select changes, developers may learn what its questions reward. A model can gain points through familiarity with the test rather than through flexible reasoning. A fresh held-out collection is therefore important when assessing how broadly a change transfers.
The Lantern Lab’s smaller transfer set illustrates an evidence gap, not a definitive diagnosis of overfitting. Without the actual training and selection trace, one cannot identify the cause of V3’s regression.
Compute capacity constrains the improvement loop
Generating possible experiments may become cheap while training and validating each candidate remain expensive. A model that suggests one hundred variations does not create the compute and human capacity to test them. A useful research programme prioritises experiments with clear hypotheses and meaningful expected information.
Even when generation of proposals accelerates, the next bottleneck may be reliable datasets, independent evaluation, hardware, scientific experiments or responsible deployment approval. A faster proposal stage does not establish an accelerating overall process.
The evaluator should not be the candidate’s sole witness
Suppose the improving system also writes the rubric and grades its own new version. It might inadvertently reward an output style familiar to itself or overlook errors that a legitimate user would notice. Independent evaluation processes and appropriate human or external checks help reduce this circularity.
An AI-based grader can be useful if validated against reliable reference examples. It should not be described as an objective authority merely because it is another model.
A release gate is stronger than a progress narrative
Before developing V2 or V3, imagine the fictional team requires at least 70% on the twenty-item transfer evaluation and no unauthorised external actions. V2 scores 75% and passes the numerical part. V3 scores 65% and fails it, despite its 82% familiar-test result.
That decision follows a rule stated before looking at the performance numbers. Changing the threshold after the result to call V3 successful would undermine the test’s purpose. Whether this fictional gate is sufficient for a real use case remains another question.
More autonomous research can increase governance demands
A tool-using agent may obtain source code, suggest changes and launch tests. Those operations cross boundaries that a text-only brainstorming assistant never crosses. Appropriate permissions, quotas, sandboxing, safe data and accountable approval become more important, not less, as the system’s reach expands.
An operation that changes production software or handles sensitive research data requires separate authority from the ability to draft a proposed experiment. Technical capability does not invent that approval.
A sequence of improvements can slow down
The familiar-task increments in Lantern are +8, +4 and +4 percentage points from V1 through V4. Even within this invented series, the rate of gain is not steadily accelerating. The transfer results move +5, then −10, then −5 points. These two trends cannot be honestly collapsed into a simple upward exponential curve.
A model predicting runaway improvement from this sample would be extrapolating far beyond the data. The only defensible calculation is what happened in the stipulated versions and what was not measured.
A bounded recurrence is an educational toy
The recurrence p_next=p+0.2(1−p) has a fixed point at one by construction. Starting with p=0.6, it gives 0.68, 0.744 and 0.7952. The remaining gap shrinks because each iteration removes one fifth of the remaining difference.
This is an algebraic exercise, not an empirical description of AI capability. Nothing in the mathematical formula guarantees that an actual engineering process follows it or even that a broad intelligence level can be represented by one bounded scalar.
Forecast disagreement is a reason for explicit assumptions
Claims about an imminent, slow or sudden transition to AGI frequently depend on definitions, compute trajectories, algorithmic progress, adoption constraints and evaluation choices. A serious discussion should expose those assumptions rather than converting a scenario into a dated prediction presented as fact.
Where evidence is incomplete, a responsible article compares the models and limits of the proposals. The International AI Safety Report 2026 itself describes substantial uncertainty, including disagreement among experts about timing and emerging risks.
Independent practice: a new acceptance rule
Suppose a fictional team changes the deployment requirement to at least 80% familiar-task accuracy and 70% transfer accuracy. V1 fails the first condition, V2 scores 78% and fails it, V3 meets the familiar criterion but fails transfer at 65%, and V4 also fails transfer at 60%. None passes both conditions.
The exercise shows how a decision can be evidence-based without declaring one version categorically superior. An accepted candidate must meet the legitimate intended task, not merely outrank earlier models on a selected average.
Continue the connected Super Intelligence learning route: Business and Institutions · Defensive Cybersecurity · Government and Public Systems. These separate chapters trace institutional workflow, defensive system boundaries, public accountability and the limits of broader capability claims. See the full numbered curriculum.
28. Authoritative sources and the final curriculum route
Primary reading includes Google DeepMind’s Levels of AGI (ICML 2024) and the International AI Safety Report 2026, which distinguish current evidence, conceptual frameworks, future scenarios and persistent uncertainty. Neither validates our invented Lantern model versions or guarantees a timetable for AGI or ASI.
Continue with eduKate’s Fast Takeoff versus Slow Takeoff, Synthetic Data and Learning Bottlenecks, Benchmarks and Alignment and Oversight. The full map is How Super Intelligence Works, How X Works and The Gold Standard of Education.
All Lantern Lab versions, scores, iteration times and recurrence parameters are fictional mathematical teaching fixtures. No actual AI system was recursively trained, improved or evaluated. Source review: 22 October 2026.