Machine reasoning is the organised use of model inference to solve a problem that cannot be answered reliably by one immediate continuation. A reasoning system may decompose a task, track intermediate state, compare alternatives, use tools, verify calculations, search possible paths and revise a candidate answer before presenting the result.
This article uses “reasoning” as an operational description of problem-solving behaviour. It does not claim that a model has human consciousness, private subjective thought or a mind identical to a person’s. The measurable question is whether the system can transform inputs into correct, generalisable and verifiable solutions.
This distinction matters for the How Super Intelligence Works series because machine reasoning belongs to engineered machine-intelligence mechanisms, while the separate How Intelligence Works estate covers intelligence more broadly. Here we stay with runtime machine problem solving.
Research such as Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, Self-Consistency Improves Chain of Thought Reasoning in Language Models, ReAct and Tree of Thoughts explores several ways to organise model inference for harder tasks. These methods differ, but all move beyond treating one sampled completion as the only possible route.
Previous: 032 — Next-Token Prediction. Here we ask how token generation becomes deliberate problem solving.
The Hidden Transition: From Continuation to Problem State
A model can complete “2 + 2 =” with “4” through a direct learned pattern. Harder tasks require more structure. If a problem contains several conditions, the system needs to preserve which facts are given, what must be found and which operations remain valid.
This can be represented as a problem state: known facts, unknown quantities, constraints, candidate steps and a completion test.
Reasoning begins when runtime computation is organised around changing that state rather than merely producing fluent continuation.
A Simple Worked Arithmetic Problem
Problem: “A store begins with 80 notebooks, receives 35, sells 12, transfers 8 and removes 5 damaged books. How many remain?”
A direct model can output 90. A reasoning-oriented route makes the structure visible: opening balance = 80; inflow = +35; outflows = −12, −8 and −5; expression = 80 + 35 − 12 − 8 − 5; result = 90.
The arithmetic can then be verified with a calculator. The model’s reasoning role is selecting the correct quantities and signs; the tool’s role is exact computation.
Reasoning Is Not the Same as Explanation
A model can generate a persuasive explanation after guessing an answer. Conversely, it can arrive at a correct answer through internal computation without producing a long visible derivation.
Therefore, a natural-language explanation is evidence that can be inspected, not guaranteed access to the model’s exact internal computation.
Reasoning evaluation should check outcomes and intermediate constraints where possible rather than trusting narrative confidence.
Problem Decomposition
Decomposition splits a hard problem into smaller subproblems whose results can be recombined.
For a research task, subproblems might be: identify the current policy, find the previous policy, extract the changed clause, locate the stated reason and compare dates.
A good decomposition preserves dependencies. A bad decomposition separates facts that must be considered together.
Worked Decomposition Example
Task: “Compare two school schedules and identify conflicts for one tutor.”
Subtask 1: extract Tutor A’s lessons from Schedule 1. Subtask 2: extract Tutor A’s lessons from Schedule 2. Subtask 3: normalise times. Subtask 4: detect overlaps. Subtask 5: produce a conflict table with source references.
Each subtask has an observable output. The final result can be checked against the original schedules.
Intermediate Representations
Reasoning becomes easier when information is converted into a representation suited to the task: equations for arithmetic, tables for schedules, graphs for dependencies, structured objects for code or evidence packets for research.
The model does not need to keep every relationship in prose. External scratchpads and tools can preserve state more reliably.
Reasoning With Constraints
A constraint says what a valid solution must satisfy. In scheduling: no overlapping lessons, room must be free, tutor must be available. In mathematics: the equation must balance. In code: tests must pass.
Constraint checking narrows the search space and prevents plausible but invalid answers.
A Constraint-Satisfaction Example
Three lessons A, B and C need rooms R1 and R2. A and B occur at the same time; C occurs later. A requires R1. A valid assignment must place B in R2 while C can use either room later.
A model can propose assignments, while deterministic code verifies conflicts. The reasoning system combines flexible proposal with exact constraint checking.
Forward Reasoning
Forward reasoning starts from known facts and applies valid operations until a conclusion appears.
If A implies B and B implies C, then observing A can support C through two steps.
In machine systems, those implications may be learned patterns, explicit rules or tool results.
Backward Reasoning
Backward reasoning starts from the goal and asks what must be true for that goal to hold.
To prove a student qualifies for a programme, the system may work backward through required age, score and prerequisite conditions.
Backward reasoning is useful when the goal has clear prerequisites.
Deductive Reasoning
Deduction applies rules so that conclusions follow from premises. Formal logic and program verification can check this precisely.
Language models can generate deductive chains, but a formal solver can provide stronger proof when the task can be encoded symbolically.
Inductive Reasoning
Induction infers general patterns from examples. Machine learning itself is largely inductive: a model generalises from training data to new cases.
At inference time, a model can also infer a pattern from demonstrations in context.
Abductive Reasoning
Abduction proposes the best explanation for observations. If a device will not connect, possible explanations include network outage, wrong credentials or hardware failure.
An SI system can rank hypotheses and request tests that distinguish them. The final diagnosis should remain provisional until evidence confirms it.
Analogy
Analogical reasoning maps structure from one problem to another. A model may recognise that balancing a chemical equation and balancing an accounting ledger both involve preserving constraints, even though the domains differ.
Useful analogy transfers relational structure, not superficial vocabulary.
Chain-of-Thought Prompting
Chain-of-thought prompting encourages generation of intermediate steps before the final answer. Research showed that this can improve performance on some arithmetic, commonsense and symbolic tasks in sufficiently capable models.
The technique does not guarantee correct reasoning. A detailed chain can still contain an early mistake that propagates.
Few-Shot Chain-of-Thought
Examples can demonstrate the desired intermediate structure: identify facts, perform operations, verify the result, then answer.
The model learns the pattern from context without a weight update.
Zero-Shot Reasoning Prompts
Simple instructions such as asking the model to work step by step can change the generated reasoning pattern.
Effectiveness varies by model and task. Prompt phrases are not a substitute for evaluation.
Self-Consistency
Self-consistency samples several reasoning paths and chooses the answer supported by the most consistent outcomes.
This can improve tasks where one sampled chain is noisy. It spends more inference compute and can still fail when all paths share the same misconception.
A Self-Consistency Example
A word problem is solved ten times with varied sampling. Seven paths produce 42, two produce 41 and one produces 44.
Selecting 42 is stronger evidence than one isolated sample, but verification with the underlying equation remains better than majority vote alone.
ReAct: Reasoning and Acting
ReAct interleaves model-generated reasoning or task state with actions such as search or tool use and observations returned from the environment.
The important architectural idea is feedback. The model does not need to solve everything from static memory; it can obtain evidence and update the next step.
Worked ReAct-Style Example
Question: “Is the current device-loan period shorter than last year’s?”
Step 1: retrieve current policy. Observation: three school days. Step 2: retrieve archived policy. Observation: five school days. Step 3: compare. Conclusion: yes, the current normal period is two school days shorter.
The final answer is grounded because reasoning is interleaved with source retrieval.
Tree of Thoughts
Tree-of-thought approaches explore multiple candidate intermediate states rather than one linear chain.
A model proposes branches; a scoring process evaluates them; promising branches are expanded.
This can help puzzles and planning tasks where early choices strongly affect later success.
Search Over Reasoning Paths
Breadth-first, depth-first, best-first or heuristic search can operate over model-generated states.
The model becomes a proposal mechanism inside a larger algorithm. Reasoning quality can improve without changing model weights.
Planning as Reasoning
Planning chooses a sequence of actions that moves from the current state to a goal under constraints.
The model must distinguish a plausible written plan from an executable plan whose preconditions and resources are real.
Worked Planning Example
Goal: publish four reviewed articles without changing the homepage. State: drafts exist; featured images available; URLs must remain stable. Constraints: publish only reviewed items; no menu changes; verify each live URL.
A good plan creates or updates the four article objects, attaches images, checks SEO metadata, publishes them, verifies links and updates the hub surgically.
A plan that says “rebuild the whole site” violates the constraint even if it could eventually produce published articles.
Reasoning With Tools
Calculators, code runners, search engines and databases can externalise operations that models perform unreliably in text.
A reasoning system should choose tools based on task structure: calculator for arithmetic, database for state, search for current public evidence, code for executable checks.
Tool Selection Is Itself a Reasoning Problem
The system must decide whether a tool is needed, which tool fits, what arguments to supply and how to interpret the result.
A correct tool call can fail if the wrong resource or version is selected.
Reasoning With Code Execution
For algorithmic problems, a model can write a small program, execute it and inspect the result.
This creates strong evidence for calculations but does not prove the program encoded the right problem. Validate the specification first.
Reasoning With Formal Solvers
SAT solvers, theorem provers and symbolic algebra systems can verify statements under formal rules.
A language model can translate a natural-language problem into formal representation, while the solver handles exact proof or search.
Reasoning With Databases
A model can infer which query answers a question, but the database supplies authoritative rows.
Reasoning then combines retrieved state with policy rules rather than inventing current facts from model memory.
Reasoning With Search
Search expands the evidence base. The system can formulate queries, inspect sources, identify conflicts and search again.
Research reasoning should stop when the needed claims are supported, not when the system has accumulated the largest number of links.
Verification as a Reasoning Stage
Verification asks whether the candidate solution satisfies the task. Different domains need different checks.
Arithmetic can be recomputed. Code can be tested. Citations can be opened. Database writes can be read back. Formal proofs can be checked by a prover.
Generate Then Verify
One common architecture separates proposal from checking. The model generates a candidate; another process verifies it.
This reduces pressure on one generation to both invent and judge itself in the same pass.
Critique and Revision
A second inference pass can critique a draft for missing assumptions or contradictions, then a third pass revises it.
Critique is useful only when it targets measurable criteria. Generic “make it better” loops can add verbosity without correctness.
Verifier Models
A model can be trained or prompted to score candidate reasoning. This can improve search efficiency.
Verifier errors matter. A persuasive wrong solution can receive a high score unless the verifier is independently evaluated.
Process Verification
Process verification checks intermediate steps, not only the final answer.
For a long equation, catching the first incorrect transformation prevents the rest of the chain from being rewarded.
Outcome Verification
Outcome verification checks only whether the final result is correct.
It is cheaper but provides less information about where the reasoning failed.
Process Versus Outcome Supervision
Training can use labels on intermediate steps or only final outcomes. Process labels can teach local correctness, while outcome labels allow multiple valid paths.
The appropriate balance depends on domain and annotation cost.
Reasoning and Working Memory
Complex tasks need temporary state: variable values, subgoals, source identities and unresolved questions.
The model’s context can hold some of this state, but external scratchpads or structured memory reduce the risk of losing it in long generations.
External Scratchpads
A scratchpad can store equations, evidence tables, code results or plan state outside the final user response.
The model reads and updates selected fields through tools rather than repeatedly regenerating the entire history.
Reasoning and Context Windows
A long reasoning chain consumes context. Earlier intermediate states can eventually be truncated or diluted.
Summarisation, structured state and retrieval can preserve the essential information without carrying every token.
Reasoning and Hallucination
Longer reasoning does not automatically reduce hallucination. A model can elaborate unsupported assumptions across many steps.
Grounding and verification must accompany deeper reasoning when the task depends on external facts.
Reasoning and Uncertainty
A reasoning system should distinguish known facts, assumptions, candidate hypotheses and verified conclusions.
Uncertainty can trigger more evidence gathering instead of being hidden by a confident final answer.
Reasoning and Calibration
More test-time reasoning can improve accuracy while changing confidence calibration.
Measure both correctness and whether confidence tracks outcomes.
Reasoning and Search Cost
Exploring ten paths costs more than one. Tool calls and verifier passes add latency and money.
The system needs a stopping rule based on expected value, not unlimited “thinking”.
Reasoning and Diminishing Returns
The first extra verification pass may produce a large benefit; the tenth may add little.
Test-time compute should be allocated where difficulty and consequence justify it.
Reasoning and Task Difficulty
Easy tasks can be harmed by unnecessary complexity. A direct lookup does not need a five-step search tree.
Runtime routing can send simple tasks to fast paths and hard tasks to deeper reasoning.
Reasoning Failure Map
Level 1: wrong problem representation. Level 2: missing constraint. Level 3: poor decomposition. Level 4: invalid intermediate step. Level 5: wrong tool or source. Level 6: early commitment. Level 7: verification failure. Level 8: search stops too early or too late. Level 9: final answer overstates evidence.
This eduKateSG map identifies where the reasoning process first became unstable.
Worked Diagnosis: Correct Arithmetic, Wrong Problem
The calculator returns 90, but the model included a pending inventory movement that should have been excluded.
The arithmetic is correct. The reasoning failure occurred in record selection before calculation.
Worked Diagnosis: Good Plan, Impossible Action
The plan says to book Friday 7 PM, but the schedule database shows the slot is occupied.
The plan is not executable. Reasoning needs a state check and alternative search.
Worked Diagnosis: Five Paths Agree, Source Is Wrong
Self-consistency cannot repair a stale policy supplied in every context. The shared evidence error dominates all branches.
Worked Diagnosis: Verifier Rewards Style
A verifier model prefers longer polished chains even when shorter correct chains are better.
The verification objective is misaligned. Evaluate the verifier on correctness rather than rhetorical quality.
A Practical Reasoning Benchmark
Choose tasks with known solutions and distinct failure modes: arithmetic, logic, code, source comparison and planning.
Measure direct-answer baseline, decomposition, tool-assisted reasoning, multi-sample self-consistency and verification.
Compare accuracy, latency, cost and failure recovery rather than assuming the most elaborate method wins.
Independent Exercise 1: Reasoning or Explanation?
A model states the correct answer first, then gives a plausible explanation. Does that prove the explanation caused the answer?
Answer
No. The generated explanation is inspectable output, not guaranteed faithful access to the internal computation that produced the answer.
Independent Exercise 2: Decomposition
A task asks whether two policies conflict. What useful subtasks exist?
Answer
Retrieve both current versions, extract relevant clauses, normalise terminology, compare conditions and identify actual contradictions with citations.
Independent Exercise 3: Tool Choice
A problem needs exact multiplication of several decimals. Which tool is appropriate?
Answer
A calculator or code execution tool. The model should focus on constructing the correct expression.
Independent Exercise 4: Self-Consistency
Ten sampled chains agree on an answer but all use the same wrong formula. Is majority vote enough?
Answer
No. Verify the formula or result independently. Agreement is not proof when errors are correlated.
Independent Exercise 5: Search
A reasoning tree keeps exploring after a formal solver has already verified a correct proof. What is wrong?
Answer
The stopping rule is weak. More search adds cost without useful uncertainty reduction.
Formal Reasoning Versus Natural-Language Reasoning
Natural-language reasoning is flexible but ambiguous. Formal reasoning translates a problem into symbols governed by explicit rules.
A theorem prover can establish whether a statement follows from axioms. A SAT solver can decide whether Boolean constraints have a satisfying assignment. A language model can help perform the translation between human language and the formal problem.
The strongest guarantee comes from the formal checker, not from the model’s confidence about the proof.
Worked Formal Logic Example
Premises: All mammals are warm-blooded. Whales are mammals. Goal: whales are warm-blooded.
A formal representation applies universal implication to the instance whale. The conclusion follows directly.
The language model can explain the proof, but a logic engine can verify the inference mechanically.
Reasoning in Mathematics
Mathematical reasoning combines definitions, symbolic transformations, theorem selection and calculation. Language models can propose solution paths while calculators, computer algebra systems and proof assistants verify parts of the work.
The system should separate algebraic equivalence from numerical checking. Testing three sample values is weaker than proving two expressions are equal for all permitted inputs.
Worked Algebra Example
Solve 3(x − 2) = 15. Expand to 3x − 6 = 15. Add 6 to both sides: 3x = 21. Divide by 3: x = 7. Substitute back: 3(7 − 2) = 15.
Each transformation preserves equality. The final substitution verifies the candidate solution.
A model can generate these steps; symbolic software can check them exactly.
Reasoning in Geometry
Geometry often depends on diagrams, hidden assumptions and theorem applicability. A drawn diagram may not be to scale.
The reasoning system needs explicit givens: parallel lines, equal angles, lengths or circle properties. Visual appearance alone should not create a theorem premise.
Reasoning in Code
Code reasoning includes tracing state, understanding control flow, predicting outputs, detecting bugs and designing algorithms.
Execution is a powerful verifier because the program can be run on tests. Yet a passing test suite proves only the covered behaviours, not every possible input.
Worked Code Trace
Function: start total = 0; for x in [2,4,6], add x to total; return total. Trace states 0 → 2 → 6 → 12.
For more complex programs, an interpreter can provide exact runtime state while the model explains the logic.
Reasoning About Exceptions
Many rules have exceptions. “Loans last three days unless an approved field project extends them to five.”
A reasoning system must identify whether the exception’s conditions are satisfied before applying it.
Missing the exception and over-applying the exception are different failure modes.
Rule Priority
Rules can conflict. A general rule may be overridden by a more specific rule or a later authoritative policy.
The system needs explicit precedence rather than choosing whichever sentence appears most recently in context.
Temporal Reasoning
Time introduces ordering, duration and validity windows. A policy can be published on Monday and take effect Friday.
Reasoning should distinguish event time, publication time and query time. “Current” means the rule valid at the relevant moment, not simply the newest document date.
Worked Temporal Example
Old rule valid through 30 September: five-day loan. New rule effective 1 October: three-day loan. Query on 30 September should use five days; query on 1 October uses three.
The system needs dates and effective-status logic, not just semantic similarity.
Spatial Reasoning
Spatial tasks track relative position: left, right, above, inside, near, connected.
A model can reason from text or images, but coordinates and maps can externalise state when precision matters.
Causal Reasoning
Correlation says two variables vary together. Causal reasoning asks whether changing one would change the other under an intervention.
Language models can propose causal hypotheses from patterns, but observational text alone does not establish causation. Experiments, domain models or causal inference methods provide stronger evidence.
Counterfactual Reasoning
A counterfactual asks what would happen if some condition had been different. This requires holding relevant background conditions stable while changing the specified variable.
Models can generate plausible counterfactual narratives, but their validity depends on a correct causal model.
Reasoning Under Uncertainty
Not every premise is known. A diagnostic system may maintain several hypotheses with probabilities rather than commit immediately.
Reasoning can choose the next observation that most reduces uncertainty.
Value of Information
A useful question is not only “What do I think?” but “Which new fact would change the decision most?”
A system can prioritise tests or searches that discriminate between competing hypotheses.
Worked Diagnostic Example
Laptop cannot connect. Hypotheses: Wi-Fi outage, wrong password, device adapter problem. Check whether other devices connect. If none connect, outage becomes more likely; if others connect, test credentials and adapter.
The reasoning system chooses observations strategically rather than generating a long list of possibilities.
Bayesian-Style Updating
When new evidence arrives, hypothesis probabilities can be updated. A formal Bayesian model makes assumptions explicit.
Language-model reasoning can approximate such updates in prose, but exact probabilistic tasks are stronger when delegated to numerical tools.
Heuristics
A heuristic is a shortcut that guides search without guaranteeing the optimal answer. Human and machine reasoning both use heuristics because exhaustive search is often impossible.
Good heuristics reduce the search space; bad heuristics systematically miss valid solutions.
Best-First Search
Best-first search expands the candidate state with the highest estimated promise.
A model or verifier can provide heuristic scores. If the scorer is miscalibrated, the search can spend compute on attractive dead ends.
Breadth-First Search
Breadth-first search explores all states at one depth before going deeper.
It can find short solutions but becomes expensive when branching factor is large.
Depth-First Search
Depth-first search follows one branch deeply before backtracking.
It uses less memory but can waste time down a bad branch without good stopping rules.
Monte Carlo Search Intuition
Sampling can estimate which branches lead to good outcomes by exploring simulated continuations.
This is useful when exact search is too large, but quality depends on the simulator and evaluation function.
Reasoning as Search in Program Space
For coding, the model can propose candidate programs, run tests and mutate failures. The search space is source code rather than prose.
Execution creates objective feedback that can guide the next generation.
Reasoning as Search in Proof Space
For theorem proving, the system can propose lemmas or tactics and let a proof assistant accept or reject them.
Formal verification prunes invalid branches immediately.
Reasoning as Search in Action Space
An agent chooses among actions: search, read, calculate, edit, ask, stop. The environment returns observations.
The agent loop is therefore a reasoning process over external state, not only a text continuation.
State Compression
Long tasks accumulate observations. Carrying every detail forever can exceed context capacity.
The system can compress state into structured summaries: completed steps, open questions, verified facts, failed attempts and resource IDs.
Compression must preserve information needed for later decisions.
Reasoning Memory Versus User Memory
A scratchpad containing intermediate calculations is task memory. A stored preference remembered next month is persistent user memory.
Conflating them creates privacy and correctness problems.
Reasoning Traces as Training Data
Models can be trained on worked solutions or generated traces. Good traces can teach decomposition and verification patterns.
Bad traces teach plausible errors. Trace quality needs independent checking.
Synthetic Reasoning Data
A strong model can generate reasoning problems and solutions for a smaller student. Formal tools can verify mathematics or code automatically.
This connects synthetic data, distillation and reasoning into one training pipeline.
Outcome Reward
A model can receive reward only for the correct final answer. This allows many solution paths but gives sparse feedback about where a failure occurred.
Outcome supervision is cheap when answers are easy to verify.
Process Reward
Process supervision scores intermediate steps. It provides denser feedback and can discourage invalid local moves.
Human annotation of every step can be expensive; automated verifiers work well only in domains with checkable structure.
Reasoning and Preference Learning
Humans may prefer concise correct reasoning over rambling output. Preference data can shape presentation and helpfulness.
Preference should not replace correctness checks. People can be persuaded by confident wrong explanations.
Reasoning and Adversarial Inputs
A problem statement can contain irrelevant detail designed to distract the model or malicious instructions embedded in retrieved sources.
Robust reasoning separates task data from authority and filters irrelevant evidence.
Reasoning and Contradictory Evidence
When sources disagree, the system should not average them into one confident statement.
Compare source authority, date, scope and direct support. Preserve unresolved conflict if evidence cannot settle it.
Source Triangulation
Several independent sources can strengthen a claim. Ten pages copying one original do not provide ten independent confirmations.
Reasoning systems should track provenance and derivative relationships where possible.
Reasoning and Citation Support
A citation should support the exact nearby claim. A source about the topic generally is weaker than a source that establishes the proposition.
Citation checking is therefore a reasoning task over entailment and provenance.
Reasoning and Numerical Precision
A chain can be conceptually correct but lose accuracy through rounding. Numerical tasks should define precision and rounding rules.
Use exact arithmetic where possible and round at the appropriate stage.
Reasoning and Units
3 metres + 5 seconds is not a meaningful direct addition. Units constrain valid operations.
A reasoning system should preserve units through calculations and use dimensional analysis as a verification layer.
Worked Unit Example
Distance = 120 km, time = 2 h. Speed = 60 km/h. Converting time to minutes without adjusting units can produce a wrong numerical interpretation.
Explicit units make the error visible.
Reasoning and Approximation
Some tasks require estimates rather than exact answers. The system should label approximations and identify assumptions.
False precision—such as reporting 12.347821 when inputs are rough estimates—misrepresents certainty.
Reasoning and Commonsense
Everyday tasks rely on background regularities: cups hold liquids, people cannot be in two distant rooms simultaneously, tomorrow follows today.
Models learn many such patterns from data, but edge cases and cultural variation make “commonsense” an empirical capability rather than a fixed rulebook.
Reasoning and World Models
Planning improves when a system can predict how actions change the environment. A world model represents those transition dynamics.
Simulation can test plans before real execution, but simulated results inherit model errors.
Reasoning and Robotics
A robot must reason under continuous feedback: observe position, choose action, move, sense again, correct.
Latency and physical safety make real-world reasoning more constrained than text-only problem solving.
Reasoning and Human Collaboration
The system can surface options, uncertainty and evidence while a person supplies goals, values and domain judgment.
A strong collaborative workflow makes the handoff explicit: which parts are machine proposals, which are verified facts and which decisions remain human.
Reasoning Evaluation by Difficulty
Build a ladder: direct retrieval, one-step transformation, multi-step arithmetic, conflicting evidence, tool use, planning under constraints and adversarial variants.
Performance should degrade gracefully rather than collapse unpredictably as difficulty increases.
Reasoning Evaluation by Transfer
Change names, numbers and surface wording while preserving the underlying structure.
A system that memorises benchmark templates will fail transfer even when its original score is high.
Reasoning Evaluation by Robustness
Add irrelevant facts, reorder premises, insert misleading but non-authoritative text and remove one required input.
A robust system should preserve valid reasoning and flag missing information.
Reasoning Evaluation by Verification
Measure not only final answer accuracy but whether the system uses available tools, cites evidence and recognises when verification fails.
A correct lucky guess and a verified correct solution should not be treated as identical process quality.
Independent Exercise 6: Temporal Logic
Rule changes on 1 October. Query asks about 30 September. Which rule applies?
Answer
The rule valid on 30 September, unless the policy explicitly states retroactive effect.
Independent Exercise 7: Value of Information
Three hypotheses explain a failure. One quick test would eliminate two. Should the system write a long diagnosis first?
Answer
No. Perform or request the high-information test if authorised and practical.
Independent Exercise 8: Formal Verification
A generated proof sounds convincing but the theorem prover rejects step 4. What should control the result?
Answer
The formal checker for the encoded theorem. Repair the invalid step or the formalisation.
Independent Exercise 9: Conflicting Sources
Two official documents disagree and neither is clearly newer. What is the correct reasoning behaviour?
Answer
Preserve the conflict, identify each source and seek an authoritative resolution rather than inventing a merged rule.
Reasoning Quality and Answer Presentation Are Separate
A strong reasoning process can produce a concise final answer. A weak process can produce an elaborate essay. Evaluation should therefore separate internal task success from presentation quality.
For many users, the best interface shows the result, key evidence and any uncertainty without exposing every intermediate generated token.
Reasoning and Faithfulness
A generated chain may not faithfully describe the model’s hidden computation. Researchers study this gap because a plausible explanation can be post-hoc.
Applications should avoid claiming that visible chain-of-thought is a literal transcript of the model’s internal reasoning. Treat generated rationales as one artefact that can be checked.
Reasoning Without Exposing Private Scratch Work
A system can use internal structured state, tool results and verifiers while returning only a concise user-facing explanation.
This preserves inspectability through evidence and outputs without relying on raw private chain-of-thought as the audit mechanism.
Reasoning and Error Localisation
The best diagnostic output identifies the first broken step: wrong premise, wrong source, wrong transformation, wrong tool input or wrong verification.
This is more useful than a final confidence score because it points directly to a repair.
Reasoning and Recovery
When one path fails, the system can backtrack, choose another tool, request missing information or return partial progress.
Recovery behaviour is part of reasoning competence. Endless repetition of the same failed step is not progress.
Reasoning and Stopping Rules
A system should stop when the task is verified, when the remaining uncertainty requires human information, or when the compute budget is exhausted.
It should not keep searching merely because more branches are available.
Reasoning Budgets
Budgets can limit tokens, model calls, tool calls, search branches and wall-clock time.
A good budget is task-sensitive: high-stakes difficult tasks deserve more verification than trivial transformations.
Adaptive Reasoning Depth
The system can estimate task difficulty and allocate more work only when needed.
Simple lookup: one pass. Ambiguous policy comparison: retrieval plus verification. Difficult proof: search over candidate derivations and formal checking.
This adaptive allocation leads directly into test-time compute.
Reasoning and Cost Transparency
A user may see one answer while the system ran twenty hidden inference calls. Cost, latency and energy therefore depend on the reasoning algorithm, not only output length.
Operational reports should measure total runtime work.
Reasoning and Benchmark Saturation
When models perform near-perfectly on familiar benchmarks, small score differences can become less informative. Harder private tests, transfer cases and contamination controls matter more.
Benchmark success should lead to better evaluation, not the assumption that reasoning is solved.
Reasoning and Dataset Contamination
A reasoning benchmark can be memorised from training data. The model may reproduce a known solution rather than solve the structure anew.
Paraphrases, new instances and private tasks help test genuine transfer.
Reasoning Under Distribution Shift
A model trained on textbook maths may struggle with messy real invoices. A planning model trained on clean simulated environments may fail with incomplete real-world state.
Reasoning should be evaluated where it will actually be used.
A Complete Reasoning Record
For consequential tasks, record the task definition, evidence sources, tools used, intermediate structured state when appropriate, verification results and final status.
The record does not need private hidden chain-of-thought. It needs enough operational evidence to reproduce or investigate the outcome.
Worked End-to-End Example: Research and Calculation
Question: “Did the new policy reduce the maximum normal loan period, and by how much?”
Retrieve current policy: three school days. Retrieve archived policy: five school days. Validate effective dates. Compute 5 − 3 = 2. Final answer: yes, by two school days, with citations to both versions.
The reasoning is inspectable because each stage has an evidence object and a check.
Worked End-to-End Example: Code Repair
Bug report: negative-only input returns zero instead of the largest negative value. Inspect code. Hypothesis: max_value initialised to zero. Patch initialisation to first element. Run tests [3,8,2] → 8 and [−5,−2,−9] → −2. Verify no empty-list regression.
The system reasons through hypothesis, patch and execution evidence rather than merely generating a plausible explanation.
Worked End-to-End Example: Scheduling
User asks for Friday after 6 PM with same tutor. Extract constraints. Query schedule. Candidate 6:30 available. Ask approval. Recheck state. Execute reschedule. Read back new booking.
Reasoning crosses language interpretation, database state, planning, permission and verification.
Independent Exercise 10: Recovery
A search returns no official source. What should a reasoning system do?
Answer
Try an appropriate alternate authoritative route, ask for the source, or report the limitation. It should not fabricate evidence.
Independent Exercise 11: Budget
A simple arithmetic problem launches 100 model samples. Is that good reasoning?
Answer
Not necessarily. The extra compute is wasteful if a calculator solves the task exactly. Reasoning depth should match the problem.
Independent Exercise 12: Faithfulness
A visible rationale sounds logical but a tool trace shows the answer was obtained from a database lookup. Which evidence describes the actual operational route?
Answer
The tool trace and database result. The rationale may be explanatory prose rather than a faithful account of the runtime computation.
Frequently Asked Questions About Machine Reasoning
Do language models reason?
They can exhibit problem-solving behaviours that are reasonably described operationally as reasoning. This does not settle philosophical questions about consciousness or human-like cognition.
Is chain-of-thought always better?
No. It can help some tasks and hurt others. Long chains can propagate mistakes or add cost.
Does a visible reasoning trace show the model’s true internal reasoning?
Not necessarily. Treat it as generated intermediate output that can be inspected, not perfect access to hidden internal computation.
What is self-consistency?
Sampling multiple reasoning paths and selecting an answer supported by repeated outcomes.
What is ReAct?
A pattern that interleaves model reasoning or task state with actions and observations from tools or environments.
What is Tree of Thoughts?
A family of methods that explores multiple candidate intermediate states using search rather than one linear generation path.
Why use external tools?
They provide exact calculation, current evidence, authoritative state or formal verification that text generation alone may not provide reliably.
How should machine reasoning be evaluated?
Use tasks with known acceptance criteria, verify intermediate constraints where possible and measure quality, cost, latency and generalisation.
Machine Reasoning Is Inference Organised Around a Goal
The base model still produces tokens and scores. Reasoning emerges when the runtime system organises those capabilities into decomposition, state tracking, search, action and verification.
The strongest systems do not equate longer text with deeper reasoning. They spend additional computation only when it reduces uncertainty or improves a checkable outcome.
Continue through the How Super Intelligence Works hub. Next: 034 — Test-Time Compute.
A mature machine-reasoning system should therefore be judged by disciplined problem solving rather than by how much intermediate prose it emits. It should preserve the task, identify constraints, use tools when they provide stronger evidence, verify consequential steps, recover from failed branches and stop when the result is established or the remaining uncertainty cannot be reduced. A short verified solution can be better reasoning than a long speculative chain. A database lookup should remain a lookup, exact multiplication should use a calculator when available, and a formally checked proof does not need endless additional sampling. The reader should also be able to distinguish proposals from observations, calculations from generated claims, and verified results from plausible explanations. That operating discipline creates the next mechanism in the series: test-time compute. Once a system can explore several candidate paths, the next engineering question is how much runtime work to allocate, where to allocate it and how to measure whether the extra computation genuinely improves the answer rather than merely making the process longer.
A Final Machine-Reasoning Floor Check
Before moving into test-time compute, preserve one final distinction: a reasoning system should be evaluated on solved tasks, verified intermediate state and recovery from failure, not on the volume of hidden or visible deliberation. The useful unit is progress toward a checkable result. Additional tokens, branches or tool calls earn their place only when they reduce uncertainty, satisfy a constraint or strengthen verification. This completion rule is the floor carried into Article 034.
