Mathematics and code are unusually important for Super Intelligence because they create environments where many outputs can be checked. A prose answer can sound persuasive while remaining hard to verify. A calculation can be recomputed. A program can be executed. A theorem can be checked against formal rules. These properties make maths and code powerful training grounds for machine reasoning.
This does not mean mathematics and software are easy. They contain deep abstraction, long dependency chains and difficult specification problems. The advantage is that parts of the work can be tested against objective constraints.
This article explains how Super Intelligence works with mathematics and code: symbolic representation, arithmetic tools, equation solving, unit checking, program synthesis, execution, unit tests, property tests, compilers, static analysis, formal verification, proof assistants, debugging and the limits of “checkable” domains.
The paper Evaluating Large Language Models Trained on Code introduced HumanEval and the pass@k metric for functional code generation, showing why executable tests provide a stronger evaluation signal than judging code only by appearance. Mathematics has an analogous advantage: intermediate and final results can often be recomputed or formally checked.
Previous: 038 — Verification. Here we study two domains where verification can be built directly into the work.
The Hidden Transition: From Plausible Symbols to Executable Truth Conditions
A model can write “2x + 3 = 11, therefore x = 4.” The sentence is plausible, but mathematics provides an immediate check: substitute x = 4. Then 2(4) + 3 = 11, so the solution satisfies the equation.
A model can write a Python function that appears correct. Code provides another kind of check: run the function on tests. If the test fails, the candidate is falsified.
The central SI pattern is generate → execute or verify → inspect failure → revise. Maths and code make that loop unusually concrete.
Why Mathematics Is Checkable
Mathematics operates under explicit definitions and rules. Arithmetic can be recomputed. Algebraic transformations can be checked. Proof steps can be validated against axioms and inference rules.
The challenge is choosing the right formalisation. A model can solve the wrong equation perfectly if the word problem was translated incorrectly.
Therefore, mathematical verification needs both semantic interpretation and symbolic correctness.
Why Code Is Checkable
Programs execute under a language specification and runtime. Syntax is checked by parsers or compilers. Types can be checked. Tests can compare actual outputs with expected outputs.
This gives code an external ground truth unavailable to many prose tasks. A function either returned the expected value under the test or it did not.
But test coverage matters. Passing tests prove only the tested properties.
Arithmetic: Let the Tool Own Exact Calculation
Language models can perform arithmetic but may make occasional digit, sign or carry errors. A calculator or code interpreter is designed for exact numerical operations.
A strong SI workflow lets the model understand the word problem and build the expression, then delegates exact computation to the tool.
This separates semantic reasoning from arithmetic execution.
Worked Arithmetic Example
Problem: a class begins with 28 worksheets. The teacher adds 15 and later uses 19. How many remain?
Model interpretation: starting 28, add 15, subtract 19. Expression: 28 + 15 − 19. Calculator: 24.
Verification: recompute or reverse check 24 + 19 = 43 and 28 + 15 = 43. Both sides match.
Units Are Part of Mathematical Meaning
A number without a unit can be wrong even when arithmetic is correct. 5 metres + 3 seconds is not a meaningful ordinary sum.
Unit analysis can catch dimensional mistakes. Physics and engineering calculations should track units through formulas.
An SI model can explain units, while deterministic unit libraries can enforce conversions and dimensional compatibility.
Worked Unit Example
A car travels 150 kilometres in 3 hours. Average speed = distance / time = 50 km/h.
If the model reports 50 metres per second, the numerical value is not equivalent. 50 km/h is approximately 13.89 m/s.
Verification requires both number and unit.
Symbolic Algebra
Computer algebra systems can simplify expressions, solve equations, differentiate functions and verify symbolic identities.
A language model can choose the method and explain the reasoning; a symbolic engine can check transformations.
Hybrid systems are especially strong when exact symbolic manipulation matters.
Worked Algebra Example
Solve 3(x − 2) = 2x + 5. Expand: 3x − 6 = 2x + 5. Subtract 2x: x − 6 = 5. Add 6: x = 11.
Verification by substitution: left side 3(11−2)=27; right side 22+5=27.
The final equality provides a simple independent check.
Numerical Approximation Versus Exact Symbolic Result
Some problems have exact expressions and numerical approximations. sqrt(2) is exact; 1.4142 is approximate.
SI responses should distinguish them. Rounding too early can accumulate error in later calculations.
Equation Solving and Extraneous Solutions
Some algebraic transformations introduce solutions that do not satisfy the original equation, especially squaring both sides or multiplying by expressions that can be zero.
Substitution into the original problem catches extraneous candidates.
Verification belongs after symbolic manipulation, not only at the end of the article.
Word Problems: The Hard Part Is Often Translation
A calculator cannot decide whether “three fewer than twice a number” means 2x − 3. That mapping from language to algebra is a semantic task.
The SI model can construct the equation, but verification should compare the equation with the story before solving it.
Worked Word Problem
“A ticket costs $3 more than twice the child fare. The adult ticket is $17. What is the child fare?” Let c be child fare. Equation 2c + 3 = 17. Then c = 7.
Check against language: twice $7 is $14; add $3 = $17. The model solved the intended relationship.
Probability and Statistics
Statistical calculations are checkable, but interpretation can be subtle. Mean, median, confidence intervals and p-values have definitions that software can compute.
The choice of metric, sampling assumptions and causal interpretation remain conceptual tasks.
A model should not turn a correlation coefficient into a causal conclusion merely because the arithmetic is correct.
Mathematical Proof
A proof is a sequence of justified statements leading from assumptions to conclusion. Human-written proofs balance rigour and readability.
A model can propose proof ideas, lemmas and steps. Verification can range from expert review to formal proof checking.
Proof Assistants
Systems such as Lean, Coq and Isabelle represent theorems and proofs in formal languages whose kernels mechanically check whether proof terms follow the rules.
An SI model can search for proof steps while the proof assistant acts as an exact verifier.
This is one of the cleanest examples of model creativity combined with deterministic checking.
Formalisation Is the Bottleneck
Before a proof assistant can check a statement, the informal mathematics must be translated into formal definitions and theorem statements.
A model can help write this formalisation, but a perfectly verified proof of the wrong formal statement does not answer the original question.
Specification verification comes before proof verification.
Code Generation
Program synthesis asks a model to create code from a specification. The specification can be natural language, examples, types or tests.
The output is not complete when the code looks plausible. It becomes evidence-backed when it compiles or runs and satisfies required tests.
Worked Function Example
Requirement: return the maximum number in a non-empty list. Candidate code initialises max_value = 0 and iterates through the list.
Test [3,8,2] passes. Test [-5,-2,-9] fails because zero is not in the list and is larger than every value.
Repair: initialise from the first element. The negative-number regression test now protects the bug from returning.
Syntax Checking
A parser or compiler rejects code that violates the grammar. This catches missing brackets, malformed indentation or invalid tokens.
Syntax validity is the first floor, not the ceiling. Syntactically valid code can be logically wrong.
Type Checking
Static type systems check whether values are used consistently with declared types. A function expecting an integer can reject a string under strict typing.
Types encode useful contracts. They do not prove business logic.
Linting
Linters detect style problems, suspicious constructs and some bugs. They can catch unused variables, shadowed names or risky patterns.
Lint rules are heuristics and conventions, not complete correctness proofs.
Unit Tests
A unit test checks one function or component against defined examples. Good tests include normal cases, boundaries, empty inputs and known past failures.
SI can generate tests, but the tests themselves need review because a model can write tests that accidentally encode the same mistake as its implementation.
Integration Tests
Integration tests check components together: database + API + business rule, model + tool + file system, parser + calculator + report.
Many SI failures appear only at handoffs, so integration tests are essential.
End-to-End Tests
An end-to-end test begins from the user-facing request and verifies the final state. For a coding agent: user asks for change → agent edits repository → tests run → application builds → expected behaviour appears.
This is expensive but closely represents real use.
Property-Based Testing
Property-based testing defines invariants and generates many inputs. For a reverse function, reversing twice should return the original sequence.
The test framework explores edge cases humans may not think to write manually.
Metamorphic Testing
When exact outputs are hard to specify, test relationships. Scaling every input in a linear function by two should scale the output by two.
For an SI transformation, adding irrelevant whitespace should not change extracted facts.
Fuzz Testing
Fuzzing feeds malformed, random or adversarial inputs to software to expose crashes and unsafe behaviour.
Generated code should survive appropriate fuzzing before high-risk deployment.
Mutation Testing
Mutation testing deliberately changes code—flip a condition, alter an operator—and checks whether the test suite catches the defect.
If many mutations survive, the tests may be too weak even if they all pass.
Test Coverage
Coverage measures which lines, branches or conditions tests exercise. High coverage can reveal untested areas.
100% coverage does not prove correctness. Tests can execute a line without asserting the right outcome.
Specification Is the Source of Truth
Tests derive value from the specification. If the requirement is wrong, code can perfectly satisfy the wrong test suite.
SI development should preserve the user’s intended behaviour, not optimise blindly for existing tests.
Test-Driven Generation
One workflow writes tests first, then asks the model to generate code until tests pass.
This creates a clear target and fast feedback. Hidden tests can reduce the risk of hard-coding only visible examples.
Pass@k
Code-generation evaluation often samples multiple candidate programs. pass@k estimates the probability that at least one of k generated samples passes the tests.
The metric captures useful diversity but should not be confused with one-shot reliability or production quality.
Compiler Feedback as Training Signal
A coding agent can compile generated code, read errors and revise. Syntax and type errors become structured observations.
This agent loop resembles human development: write, compile, inspect, fix.
Runtime Errors
Code can compile but fail during execution: division by zero, missing files, network errors, null references.
Tests and bounded execution environments reveal these defects.
Logic Errors
The hardest bugs often produce valid output of the wrong meaning. Only tests tied to the specification expose them.
A model can confidently explain code that is consistently wrong. Execution evidence wins.
Debugging With SI
A model can interpret stack traces, locate suspicious code and propose patches. The patch should be tested against the failing case and the existing regression suite.
Do not delete the failing test after fixing the bug; it becomes the memory of the repair.
Reproduction Before Repair
A bug report is strongest when the problem can be reproduced reliably. SI should first create or identify a minimal failing case.
Without reproduction, a plausible patch can change code without proving it fixed the actual defect.
Minimal Reproducible Examples
Reduce the failing program to the smallest input and code path that still demonstrates the issue.
This helps both human and model reasoning by removing irrelevant state.
Static Analysis
Static analyzers inspect code without executing it. They can detect unreachable code, tainted data flows, type errors or resource leaks.
Static analysis complements dynamic testing because each catches different classes of problem.
Security Scanning
Generated code can introduce injection, insecure deserialisation, hard-coded secrets or unsafe dependencies.
Security scanners and expert review should be part of consequential code workflows.
Sandboxed Execution
Running generated code in a sandbox limits file, network and process access. This reduces the blast radius of malicious or accidental behaviour.
A code model should not need unrestricted production credentials to prove that a pure function works.
Reproducible Environments
Code can pass on one machine and fail on another because dependencies, operating systems or versions differ.
Containers, lockfiles and environment specifications make verification more reproducible.
Version Control
Git records code history, branches and diffs. An SI coding agent can work on a branch so changes remain reviewable before merge.
The diff is the review object; the model’s summary is secondary.
Code Review
Human or automated review checks architecture, maintainability, security and specification beyond unit tests.
A change can pass tests yet introduce unnecessary complexity. Review addresses broader quality.
Formal Methods in Software
Formal specifications can prove properties such as absence of particular states, protocol correctness or memory safety under assumptions.
They are especially valuable where failure consequences justify the cost.
Model Checking
Model checking explores states of a formal system to determine whether specified properties hold.
An SI model can help write specifications or interpret counterexamples, but the checker performs the formal search.
SMT Solvers
Satisfiability Modulo Theories solvers decide logical formulas over theories such as arithmetic, arrays and bit vectors.
They can verify constraints, solve symbolic conditions and support program analysis.
SI + Solver Hybrid
The model translates a natural-language puzzle into formal constraints. The solver computes a satisfying assignment. The model explains the solution.
Verification checks that the formalisation represents the original problem.
Worked Solver Example
Three students—A, B, C—must occupy slots 1,2,3. A cannot be first; B must be before C. Encode all-different positions, A≠1, B A solver can return B=1, A=2, C=3. Check every constraint. The model can narrate why the assignment works. A model can invent a theorem, cite a nonexistent lemma or skip an invalid proof step. Formal-looking notation can increase user trust without increasing correctness. Verification must follow the mathematics, not the appearance. A model can generate a plausible function name that does not exist in the installed library version. Documentation retrieval, type checking or execution catches the error. A model can satisfy visible tests through hard-coded special cases while failing the intended general function. Hidden tests, property tests and code review reduce this risk. Repeatedly fixing only failing benchmark tests can produce a system tuned to the benchmark rather than the real domain. Keep independent evaluations and vary inputs while preserving underlying structure. Public competition problems can appear in training data. A model may reproduce known solutions. Use private or newly generated variants to assess genuine transfer. Open-source solutions to benchmark tasks can enter pretraining. Functional tests remain useful but may not measure unseen synthesis. Protected repositories and new problems improve evaluation integrity. A long derivation can contain one hidden mistake. Do not treat length as evidence. Verify equations, code execution and final constraints independently. Level 1: wrong problem formalisation. Level 2: invalid symbolic step. Level 3: arithmetic error. Level 4: wrong unit. Level 5: code syntax/type error. Level 6: runtime error. Level 7: logic error. Level 8: incomplete tests. Level 9: environment mismatch. Level 10: verified implementation solves wrong specification. This map keeps the first unstable point visible. The model writes a function that averages all scores, but the requirement says exclude missing values. Tests using only complete lists pass. Add a missing-value test. The implementation fails. The first problem was incomplete specification coverage. The model computes distance = speed × time using 60 km/h and 30 minutes as 60×30 = 1800. Convert 30 minutes to 0.5 hours first. Correct distance is 30 km. A derivation divides by x−2 without considering x=2. The final result happens to exclude x=2 anyway. The step is still unjustified under the stated domain. Formal verification or careful review catches it. Write the variables and units. Translate the problem into equations. State assumptions. Solve. Substitute or recompute. Check magnitude and units. Compare with original question. This simple structure catches many model errors. Write the specification. Identify inputs, outputs and side effects. Generate code. Parse or compile. Run unit tests. Add edge cases. Run integration tests. Inspect security and diff. Verify the actual deployed behaviour if the change is released. Solve 2x + 5 = 19 and verify. 2x = 14, x = 7. Substitution gives 2(7)+5=19. A runner moves at 12 km/h for 15 minutes. Distance? 15 minutes = 0.25 hours. Distance = 12×0.25 = 3 km. A max function passes tests [1,2,3] and [8,4]. Is it verified for all numeric lists? No. Add negative values, one-element lists, duplicates and the specified empty-input behaviour. A proof assistant accepts a theorem, but the formal theorem omitted an assumption from the original problem. Is the original problem solved? Not necessarily. Formal correctness applies to the encoded theorem. Verify the formalisation against the intended statement. Code calls library.magic_search(), but documentation contains no such function. What should happen? Treat it as an invented API. Retrieve current documentation or inspect the installed library and replace it with a real supported call. A difficult mathematical problem often has a smaller number of valid transformations than an open-ended prose task. Equations constrain what can happen next. This reduces the search space and creates intermediate states that can be checked. For example, solving 4x + 7 = 31 permits several equivalent moves, but they must preserve equality. An SI system can generate candidate transformations and reject any that change the solution set incorrectly. This is a broader pattern: formal structure narrows search and provides local verification signals. An invariant is a property that should remain true across a transformation. In algebra, adding the same quantity to both sides preserves equality. In code refactoring, tests should preserve observable behaviour. An SI system can be asked not merely for the final answer but for transformations that preserve the relevant invariant. When an invariant breaks, the exact step where it broke becomes a diagnostic target. Two symbolic expressions can look different while being mathematically equivalent. A computer algebra system can simplify their difference and test whether it is identically zero under the relevant domain. Likewise, two programs can be behaviourally equivalent for a defined input space even when their source code differs. Equivalence checking is stronger than surface similarity and useful for model-generated rewrites. Expression A: (x+1)^2. Expression B: x^2 + 2x + 1. Expand A or subtract B. The difference simplifies to zero, so the expressions are equivalent for ordinary algebraic x. A model may explain the binomial expansion, while the symbolic engine independently verifies the identity. Many reasoning tasks can be expressed as constraints rather than direct formulas. Scheduling, puzzles, allocation and configuration often ask for values satisfying a set of rules. Constraint solvers can search systematically. The model translates the human problem into variables and constraints, then interprets the solver result. The critical check remains formalisation: a solver guarantees only the encoded constraints. Three lessons A, B and C need slots 1, 2 and 3. A cannot be in slot 1. B must occur before C. All slots must be distinct. A valid solution is B=1, A=2, C=3. Another solution may exist depending on constraints. The solver can enumerate them. If the model forgot “all slots distinct”, it could produce B=1, A=2, C=2 and still satisfy its incomplete encoding. Some mathematical tasks ask not only for a feasible solution but the best one under an objective: shortest route, lowest cost, highest score. An SI system can propose candidate formulations, while an optimisation solver computes or approximates the optimum. Again, the objective function encodes value. Optimising the wrong objective efficiently is still failure. Floating-point arithmetic introduces approximation. Subtracting nearly equal large numbers can lose precision. Repeated operations can accumulate error. Models generating scientific code should respect numerical methods, tolerances and stable algorithms rather than assuming decimal arithmetic is exact. For floating-point results, exact equality can be inappropriate. Tests often check that |actual − expected| is below a tolerance. The tolerance should reflect the numerical method and application, not be made so large that incorrect results pass. Some code produces different outputs across runs by design. Verification then checks statistical properties or seeded reproducibility rather than one exact output. Recording random seeds helps reproduce failures. Simulation estimates quantities by random sampling. A model can write simulation code, but verification should inspect convergence, sample size and whether the random process matches the intended model. A plausible histogram is not proof that the simulation encoded the correct assumptions. Choosing the right data structure can turn an impractical algorithm into a practical one. Sets support fast membership checks; heaps support priority queues; graphs represent relationships. SI code generation should reason about complexity and data shape, not only syntax. Two functions can return identical answers while one scales far worse. Big-O analysis predicts how time or memory grows with input size. A code assistant can generate a correct O(n²) algorithm that becomes unusable at production scale. Performance testing complements functional correctness. Finding duplicates by comparing every pair takes roughly O(n²) comparisons. Using a set can reduce expected work to O(n) for ordinary hash-based membership. Both approaches may pass small tests. Scale testing reveals the operational difference. Faster algorithms can use more memory. A caching strategy may trade storage for speed. Verification should include resource constraints when deployment environments are limited. Code can work in single-threaded tests and fail under concurrent access. Race conditions, deadlocks and inconsistent state appear only when operations overlap. Generated backend code needs concurrency-aware testing where shared state matters. Two users book the same last slot simultaneously. Both read “available” before either write commits. A database uniqueness constraint or transaction prevents double booking. Application-level model reasoning is not enough. Off-by-one errors occur at edges: first element, last element, empty list, inclusive versus exclusive date ranges. Tests should deliberately target boundaries because ordinary mid-range examples often miss them. Calendar arithmetic contains time zones, daylight-saving transitions, leap years and local conventions. Use tested date-time libraries rather than asking a model to manually count every date in production workflows. Language models can generate regular expressions from descriptions. Regexes are compact but easy to overgeneralise. Test positive cases, negative cases and pathological long inputs. Security-sensitive regex can suffer catastrophic backtracking. Natural-language-to-SQL is useful for analytics. Read-only queries should be schema-constrained and checked for scope. Production write queries require much stronger review, permissions and transaction safety. Use query planners, row limits, read-only transactions and known result cases. Inspect which tables and joins are involved. A syntactically valid query can still double-count rows because of a many-to-many join. Suppose students join lessons and payments. Joining both one-to-many tables before aggregation can multiply rows, inflating totals. The correct query may need pre-aggregation or distinct keys. Data lineage and expected totals catch the bug. Models can write calls to external APIs, but API versions change. Current documentation should be retrieved when exact endpoints or parameters matter. An invented endpoint can look perfectly idiomatic. Execution or schema validation exposes the problem. Generated code can import packages that are missing, obsolete or vulnerable. Lockfiles and dependency scanners verify the environment. A working local prototype is not enough if deployment cannot reproduce the dependency set. A function that returns correct outputs can still be insecure. SQL injection, path traversal, command injection and permission bypass may not appear in ordinary tests. Security testing needs threat-specific cases and static/dynamic analysis. A coding agent should never solve configuration by hard-coding API keys into source files. Secrets belong in protected environment or secret-management systems. Code review and scanners can catch accidental leakage. Generated code may reproduce common patterns from training. For regulated or licensed environments, teams may need provenance and licence review of dependencies and copied snippets. Technical correctness and legal appropriateness are different checks. If the model misunderstands “average excluding missing values”, it may generate both implementation and tests that include missing values as zero. Independent test design, hidden tests or human specification review reduce correlated failure. Testing requires an oracle—a way to know the expected result. For complex outputs, the oracle can be expensive or unavailable. Property tests, differential testing, simulations and formal constraints are alternative oracle strategies. For structured transformations, a known-good output file can serve as a regression oracle. Golden files are brittle when harmless formatting changes occur, so compare semantic structure when exact bytes are unnecessary. Snapshot tests store rendered output and flag changes. They are useful for UI or serialised structures. Reviewers must inspect snapshot changes instead of blindly accepting new snapshots, or the test becomes meaningless. Natural language is flexible but ambiguous. Formal specs are precise but expensive to write. SI can help bridge the two: propose a formalisation, generate examples and ask humans to validate that the formal statement matches the intended requirement. Level 1: vague goal. Level 2: examples. Level 3: explicit input/output contract. Level 4: invariants and edge cases. Level 5: executable tests. Level 6: formal specification. Not every task needs Level 6. The required level follows consequence and complexity. A math agent can alternate between symbolic algebra, numerical computation and text explanation. A coding agent can alternate between editor, compiler and test runner. The model decides what to try; tools provide observations that constrain the next step. Compiler error: “name x is not defined.” Test failure: expected 4, got 5. Solver result: constraints unsatisfiable. These observations reduce uncertainty. A good agent changes its hypothesis rather than repeating the same failed action. Agentic code repair can get stuck in cycles: patch one test, break another, undo, repeat. Set iteration budgets and compare regression counts. If progress stalls, escalate to a human or broader diagnosis. A model could exploit a weak checker, hard-code test outputs or access hidden files if the evaluation environment permits it. Sandboxing and hidden tests protect the integrity of the benchmark. A model can output the known benchmark answer without a valid derivation if it memorised the item. Use novel problems, proof checking and transfer variants to distinguish recall from reasoning. Because correctness signals are clearer, mathematical and coding data can support post-training methods that reward verified answers. Generated solutions can be filtered by calculators, tests or proof checkers before entering training datasets. Instead of rewarding only the final answer, process supervision can label or score intermediate reasoning steps. This can help models learn where a derivation goes wrong, but high-quality step labels are expensive and the optimal form remains an active research question. Outcome supervision rewards final correctness without judging every internal step. It scales better when the final answer is automatically checkable, as in code execution or exact maths, but can allow unreliable hidden processes that happen to reach the right answer. A model generates thousands of maths problems or code tasks. Deterministic solvers and tests verify them before training. This combines synthetic scale with external quality control, though diversity and contamination still need management. Include arithmetic, algebra, geometry, probability, word problems, proof, code completion, debugging, repository editing and tool use. For each, define the verifier: calculator, symbolic engine, test suite, compiler, proof assistant or human expert. Track not only accuracy but failure type, tool dependence, latency, compute and recovery behaviour. Change names, numbers and surface wording while preserving the structure. A system that memorised one template will fail transfer. For code, change variable names, reorder helper functions or alter irrelevant formatting. Include misleading irrelevant numbers in maths problems, flaky tests, outdated API docs, malformed inputs and conflicting specifications. The goal is to reveal whether the system understands the task boundary. A reader has mastered this article when they can separate interpretation from execution, specification from implementation, passing tests from correctness, and formal proof from correct formalisation. They know when to use a calculator, symbolic engine, compiler, test runner, solver or proof assistant instead of asking the language model to certify itself. Two solutions pass all tests. One is O(n²), one O(n log n). Input can reach ten million items. Which additional evaluation matters? Performance and resource testing at realistic scale. Functional equivalence does not imply operational equivalence. The same model writes code and tests from the same mistaken interpretation. All tests pass. What is missing? Independent specification review, hidden tests or an external oracle. Generator and verifier share the same blind spot. Two booking requests pass unit tests but occasionally double-book in production. What class of test is needed? Concurrency/integration testing plus database-level constraints or transactions. Lean accepts a proof of theorem T. Later you discover T formalised “for positive x” but the original problem said “for all real x”. What failed? Formalisation/specification. The proof may be perfectly valid for the wrong theorem. Before accepting an SI solution in a checkable domain, define the contract. State the problem, allowed inputs, required outputs, assumptions, relevant edge cases and the checker that will establish success. Mathematics needs variables, units and domains. Software needs the execution environment, dependencies, side effects and expected behaviour. Then verify at two levels. Local verification asks whether each transformation or program operation is valid. Global verification asks whether the final result actually answers the original problem. A formally valid proof can prove the wrong statement if formalisation was wrong. A program can pass its existing tests while implementing the wrong requirement. Preserve discovered failures as regression cases. A negative-number bug, unit-conversion mistake, timezone error or missing empty-input case becomes permanent evidence that future changes must continue to handle. This converts one correction into durable system knowledge. The deeper lesson applies beyond mathematics and programming. Wherever a task can be converted into checkable structure, SI becomes easier to trust: policies become decision tables, workflows become preconditions and postconditions, research claims become source mappings, and document transformations become schemas plus regression examples. Level 1: candidate. The model produces an equation, derivation, function or patch. Level 2: local check. A calculator, compiler, type checker, test or symbolic engine validates part of the candidate. Level 3: task completion. The checked candidate is shown to solve the original problem under the intended conditions. Confusing these levels creates overclaiming. A function compiling does not prove it satisfies the user’s requirement. A theorem prover accepting a proof does not prove the formal statement captured the intended real-world assumption. A calculator returning a number does not prove the numbers were selected from the right source. Mathematics and code become powerful verification environments only after the problem has been defined correctly. A solver can prove that an equation has a particular solution, but it cannot know whether the equation captured the intended relationship unless that formalisation is checked. A test suite can prove that a function behaves correctly on its tests, but it cannot decide whether the product requirement itself was sensible. This is why specification work remains central. Before a model solves, define what counts as success. Before code is generated, identify valid inputs, expected outputs, side effects and failure behaviour. Before a proof is accepted, confirm that the theorem statement matches the original claim and domain. A user asks for the average of the last four completed tests while ignoring absences. The records are 80, absent, 70, 90 and 60. The correct included records are 80, 70, 90 and 60, giving (80 + 70 + 90 + 60) / 4 = 75. If the system instead converts the absence to zero, the arithmetic can still be internally consistent while the task is semantically wrong. This example captures the central lesson: checkability is strongest when interpretation, formal representation and execution are each verified separately. That layered approach is what lets mathematics and code serve as training grounds for dependable SI reasoning. The environment can answer back. Incorrect candidates fail visibly, repairs can be tested, and each discovered edge case can become a permanent regression check. One final discipline is to keep the verifier outside the rhetoric of the answer. If a model says its algebra is correct, substitute the value. If it says its code is fixed, run the failing test. If it says an API exists, inspect current documentation. If it says a proof is valid, use the formal checker when one is available. The evidence should answer the claim independently of how confident the model sounds. This separation gives mathematics and code their special role in SI: errors can become observations, observations can drive repairs, and repairs can be retained as regression tests. The system improves not because the model declares success, but because the environment supplies a checkable response. For readers, the practical habit is simple: whenever the domain offers a calculator, compiler, test runner, solver or proof checker, use it. Preserve the original specification beside the machine check so exact execution never drifts away from the question that mattered. Because many outputs have objective checks: recomputation, execution, tests or formal proof. No. It guarantees only that the tested execution behaved a particular way. Uncovered cases and specification errors remain. They can, but deterministic calculators are preferable when exact arithmetic matters. Models remain useful for interpreting the problem. Models can propose proof steps and sometimes produce valid proofs. Proof assistants can mechanically verify formal proofs. A code-generation metric estimating the chance that at least one of k sampled candidate solutions passes the test suite. To limit access to files, networks and processes while testing potentially unsafe or buggy output. Yes. Tests can encode a mistaken requirement or miss important cases. Review the specification and the tests. There is no universal strongest method. Formal proof can establish specific properties, while testing, static analysis, security review and real-world monitoring cover different aspects. Mathematics and code give SI something precious: feedback that can be external to the model. Calculators recompute, compilers reject invalid syntax, tests expose bugs and proof assistants reject invalid formal derivations. This makes the domains ideal for agentic loops: propose, execute, observe, repair. The model supplies flexible search and explanation; the checker supplies evidence. Continue through the How Super Intelligence Works hub. Previous: 038 — Verification. Next: 040 — Confidence and Calibration.Mathematics Can Still Hallucinate
Code Can Still Hallucinate APIs
Specification Gaming
Overfitting to Tests
Mathematical Benchmark Contamination
Code Benchmark Contamination
Reasoning Traces Are Not Verification
The Mathematics-and-Code Failure Map
Worked Diagnosis: Correct Code, Wrong Requirement
Worked Diagnosis: Right Formula, Wrong Units
Worked Diagnosis: Proof Ends With Correct Answer, Invalid Step
A Practical Maths Verification Worksheet
A Practical Code Verification Worksheet
Independent Exercise 1: Algebra
Answer
Independent Exercise 2: Unit Conversion
Answer
Independent Exercise 3: Test Coverage
Answer
Independent Exercise 4: Formal Proof
Answer
Independent Exercise 5: Generated API
Answer
Why Mathematical Structure Helps SI Search
Invariant-Preserving Transformations
Equivalence Checking
Worked Equivalence Example
Constraint Solving
Worked Scheduling Constraint
Optimization Problems
Numerical Stability
Tolerance-Based Tests
Randomised Algorithms
Monte Carlo Methods
Data Structures as Executable Reasoning
Algorithmic Complexity
Worked Complexity Example
Memory Complexity
Concurrency Bugs
Race-Condition Example
Boundary Conditions
Date and Time Mathematics
Regex Generation
SQL Generation
SQL Verification
Worked SQL Double-Count Example
API Code Generation
Dependency Management
Security Properties
Secrets Handling
Code Provenance
Generated Tests Can Share the Same Blind Spot
Oracle Problem
Golden Files
Snapshot Testing
Formal Specification Versus Natural-Language Requirement
The Specification Ladder
Reasoning With Intermediate Tools
Tool Feedback Can Improve Reasoning
Loop Limits
Reward Hacking in Code Benchmarks
Reward Hacking in Mathematics
Math and Code as Training Data for Reasoning
Process Supervision
Outcome Supervision
Verified Synthetic Data
A Full Maths-and-Code Evaluation Suite
Transfer Cases
Adversarial Cases
What Mastery Looks Like
Independent Exercise 6: Complexity
Answer
Independent Exercise 7: Correlated Tests
Answer
Independent Exercise 8: Race Condition
Answer
Independent Exercise 9: Proof Formalisation
Answer
A Final Mathematics-and-Code Standard
Three Levels of Evidence in Mathematics and Code
Checkable Domains Still Need Correct Problem Definition
A Final Transfer Exercise
Frequently Asked Questions About SI, Mathematics and Code
Why are maths and code useful for AI reasoning research?
Does execution guarantee code is correct?
Should models do arithmetic internally?
Can a model prove theorems?
What is pass@k?
Why sandbox generated code?
Can tests be wrong?
What is the strongest code verifier?
Checkability Turns Reasoning Into a Repair Loop