Spatial reasoning is how an AI system relates objects, coordinates, viewpoints and possible movement through an environment. It may detect an item in an image, represent that item in a coordinate frame, estimate relationships to other objects and plan a route under stated constraints. The test is not whether the description sounds convincing; it is whether the geometry, reference frames and proposed actions remain correct when a viewpoint, obstacle or physical dimension changes.
This chapter teaches that difference through complete worked examples. We turn a pixel location into normalised coordinates, plan a shortest route on a fictional classroom grid, rotate a target from a robot’s local frame into a room frame and test a doorway that is too narrow for a cart. You will then solve an independent transfer problem with a changed orientation. It fills the Spatial Reasoning gap in eduKateSG’s How Super Intelligence Works curriculum. The title is an editorial label for examining contemporary AI mechanisms, not a claim that today’s spatial systems are artificial superintelligence.
If you know how to label x and y on a Mathematics graph, you already have an entry point. The next step is to ask what the coordinates mean, whose viewpoint supplies them, how much uncertainty they contain and whether they justify the proposed conclusion. Those habits turn an attractive picture into a testable model.
Contents and reading routes
Start with the mechanism · Open the grid laboratory · Check the shortest route · Try the independent case · Read sources and connections
- 1. What spatial reasoning actually asks
- 2. Separate perception, representation, reasoning and action
- 3. Coordinates require origin, axes and units
- 4. Image coordinates and world coordinates are not interchangeable
- 5. A complete pixel conversion example
- 6. Build a scene with objects and relations
- 7. The fictional Study Cart laboratory
- 8. Work out a route and a lower bound
- 9. A grid path is not yet a collision-safe trajectory
- 10. Turn local movement into global position
- 11. Frames, transformations and timestamps
- 12. Occlusion creates limits, not permission to guess
- 13. Object identity across different views
- 14. Mapping, planning and control operate at different speeds
- 15. Test the model by changing one spatial assumption
- 16. Distinguish a description from a usable action plan
- 17. The human-in-the-loop boundary
- 18. Independent transfer: a doorway and a new reference frame
- 19. Full transfer answers
- 20. Diagnose five recurring mistakes
- 21. How to evaluate a spatial-reasoning system
- 22. Frequently asked questions
- 23. Authoritative sources and wider eduKate reading
1. What spatial reasoning actually asks
Spatial reasoning is the ability to work with position, orientation, distance, size, connectivity and the relationships among objects in a scene. A model may identify a door and a chair in an image, but the more demanding task is to work out which door is reachable, where the chair blocks movement and what changes when the observer turns around.
In an AI application, spatial reasoning may combine visual observations with maps, geometric calculations, coordinate transformations, object identities and physical constraints. A system that produces a fluent description of a photograph has not necessarily proved that it can place a robotic gripper within a safe tolerance. The tasks share information but have different acceptance tests.
Humans routinely use spatial language such as ‘to the left of’, ‘behind’ and ‘near’. These expressions are useful but incomplete without a viewpoint and scale. Left of the teacher is not necessarily left of the pupil facing the teacher. Behind a cup from the camera view may be in front of it from the table’s opposite end. The first job is to state the reference frame.
2. Separate perception, representation, reasoning and action
Perception gathers observations: pixels, depth measurements, detections and motion estimates. Representation stores a usable account of the scene: object lists, bounding boxes, grids, poses or more elaborate maps. Reasoning evaluates relations and possible routes. Action translates a chosen route into commands, if the system has permission and a suitable controller.
Errors can enter at each boundary. A detector may miss a low bag; a map may place a table one cell too far right; a planner may treat the robot as a point; a controller may have a delayed estimate of its present pose. A good system does not call all these problems ‘bad reasoning’ and leave the repair unspecified.
The distinction also matters for students. Identifying coordinates on a graph is not the same as deciding whether a route through those coordinates is possible. The second task needs constraints. This chapter develops both skills through one complete fictional floor-plan exercise and then changes the setting in an independent transfer case.
3. Coordinates require origin, axes and units
A coordinate such as (3,2) is meaningless without the coordinate system. It could mean three metres east and two north of a room origin, or column three and row two in an image array, or a point measured from a moving robot. Always name the origin, directions, measurement units and coordinate order.
Our main teaching map will use metres and a fixed room frame. The x-axis increases to the east, the y-axis increases to the north, and each marked grid-cell centre is one metre from its horizontal or vertical neighbour. The robot’s location is a point at a cell centre for planning; its physical radius must be checked separately.
Image data often uses a different convention, with the vertical coordinate increasing downward. The same pair of digits can refer to a different location if the order is [row,column] instead of (x,y). A reliable AI output should declare its conventions rather than leaving a reader to guess.
4. Image coordinates and world coordinates are not interchangeable
A point in a picture can be represented by pixel coordinates, but the picture is a projection of a three-dimensional scene. Moving a camera changes the image coordinate of an object even when the object has not moved in the room. Occlusion can hide part of an object and perspective changes apparent size.
Some vision APIs return normalised image coordinates rather than raw pixels. For example, a documented robotics interface uses coordinates scaled from zero to one thousand, with point output ordered [y,x]. That is an API-specific convention, not a universal rule of geometry. A different implementation might use [x,y], relative fractions from zero to one, or a camera-centred three-dimensional vector.
To convert a point responsibly, record the image width, height, crop, resizing and coordinate ordering. Without those details, placing a detection onto a floor map can be wrong by metres. A model may be very good at identifying a pen in an image and still need calibrated geometry before moving a real gripper toward it.
5. A complete pixel conversion example
Take an imaginary image that is 800 pixels wide and 600 pixels high. A marker is located 200 pixels from the left and 450 pixels from the top. Its pixel coordinates in the familiar (x,y) form are (200,450). Dividing x by 800 and multiplying by 1000 gives 250; dividing y by 600 and multiplying by 1000 gives 750.
In a [y,x] normalised convention, the point is therefore [750,250], not [250,750]. If the image is cropped or padded before inference, the mapping must include that transformation. Dividing the same pixel pair by the new crop dimensions without tracking the origin would describe a different point.
This simple calculation exposes a common AI failure: a number can be formatted exactly and still refer to the wrong geometry. Inspect image dimensions, coordinate order and pre-processing metadata alongside the prediction. A correct formula is not useful when fed dimensions from the wrong image.
6. Build a scene with objects and relations
A scene representation should distinguish observations from assumptions. In a classroom, the system might detect three tables, four chairs and a backpack near a doorway. It may infer likely free space between tables but not know what is behind a closed cabinet. Store those categories separately.
Useful relation types include left-of, adjacent-to, contained-in, supported-by, in-front-of, intersects and reachable-from. Some relations depend on viewpoint while others depend on physical contact or a fixed map. A scene graph can represent objects as nodes and their relationships as labelled links, but the graph is only as reliable as the detections and geometry that support it.
The reader should be able to ask which edge in the relation graph or which measurement proves a claim. ‘The aisle is clear’ requires a definition of aisle width, robot size and the latest observation. A picture alone may not resolve a hidden obstacle or a transparent surface.
7. The fictional Study Cart laboratory
Our main exercise uses the fictional Study Cart in a small training room. It is a classroom mathematics model, not a tested robot or a set of physical control instructions. The room is represented by grid-cell centres with integer coordinates x from 0 to 7 and y from 0 to 5. Every horizontal or vertical move between adjacent centres has a length of one metre.
The cart starts at (1,1). The target is (6,3). Two columns of shelving occupy the cells whose x coordinates are 3 and 4 and whose y coordinates are 1 and 2. In set notation, blocked centres are (3,1), (4,1), (3,2) and (4,2). Other cells shown in the packet are provisionally free. The cart is permitted to move one cell north, south, east or west per grid step, not diagonally.
The grid first asks a geometric question, not a mechanical one: can we construct a sequence of adjacent unblocked centres connecting start to goal? Physical execution requires further checks for dimensions, sensing and controller behaviour. By writing the abstraction down before planning, we keep the educational calculation separate from an unsupported claim about a real machine.
8. Work out a route and a lower bound
A valid candidate route is (1,1), (2,1), (2,2), (2,3), (3,3), (4,3), (5,3), (6,3). Every consecutive pair differs in exactly one coordinate by one. None of the listed points is in the blocked-centre set. There are eight points but seven moves, giving a grid-centre travel distance of seven metres.
Why is seven a shortest possible length under the four-direction model? From x=1 to x=6 requires a net gain of five eastward steps. From y=1 to y=3 requires a net gain of two northward steps. Even without shelving, at least five plus two, or seven, unit moves are needed. Our candidate meets that lower bound and avoids the blocked cells, so it is a shortest path in this simplified discrete model.
The conclusion is deliberately narrow. It proves a grid-route property under the stated movement rule. It does not account for rotation cost, wheel slip, obstacles between cell centres, uncertain localisation or a moving pupil entering the aisle.
9. A grid path is not yet a collision-safe trajectory
A real cart has width, length and a turning envelope. Our fictional cart is approximated as a circular footprint with a radius of 0.3 metre. At a distance of 0.5 metre from a straight obstacle-cell boundary, its nominal side clearance would be 0.2 metre. That simple difference does not assess corner geometry, sensor uncertainty or the cart’s actual shape.
To move from a grid plan to a physical plan, one approach expands obstacles by a safety margin related to the robot footprint and localisation uncertainty. Then the planner finds a path for the reference point through the remaining free region. A robust design also considers stopping distance, speed, dynamic obstacles and what to do when a sensor disagrees with the map.
Never treat the existence of a mathematically shortest path as authority to move hardware around people. A physical controller needs explicit limits and appropriate safety engineering. The shortest route may be unacceptable if a longer one has better clearances or avoids unpredictable human traffic.
10. Turn local movement into global position
Now suppose the same fictional cart stands at world position (3,2), facing north. Define its local forward direction as +x in the robot’s own frame. Relative to the world’s +x east direction, the cart has rotated anticlockwise by ninety degrees. A target one metre directly ahead is the local point (1,0).
A ninety-degree anticlockwise rotation maps the local vector (1,0) to the world displacement (0,1). Adding the cart’s world translation (3,2) gives the target world position (3,3). The operation needs both rotation and translation. Simply adding (1,0) to (3,2) would yield (4,2), which is east of the cart rather than ahead of it.
Robotics systems track such relationships through transformations between coordinate frames, often updating them over time. The calculation is not advanced for its own sake; it prevents a command intended for the cart’s own forward axis from being misread as the world’s east axis.
11. Frames, transformations and timestamps
A robot might use several frames: room map, movable base, camera, wrist and gripper. A target detected by the camera must be transformed into a frame useful for action. The required transformation depends on the camera’s calibrated pose relative to the base and the base’s pose relative to the room at the appropriate moment.
Timestamps matter when the device moves. A camera observation captured before a turn may not describe the same relative geometry after the turn. Applying the current transform to stale image coordinates can produce a neat but incorrect target. A system should track observation age and use suitable interpolation or declare that the information is too old.
The ROS 2 tf2 documentation describes tracking relationships between multiple coordinate frames over time. This is a practical example of why spatial intelligence requires bookkeeping rather than just intuition. The map must remain connected to real sensors and the relevant time.
12. Occlusion creates limits, not permission to guess
An object that is partially hidden may still be identifiable, but its hidden dimensions and exact position can be uncertain. A shelf may hide a bag that the camera cannot see. A flat image may not reveal whether a passage is wide enough for a moving cart.
A careful system distinguishes visible, inferred and unknown space. It may request a different camera angle, a depth observation or human confirmation. Labelling unseen space ‘free’ because no obstacle was detected confuses absence of evidence with evidence of absence. The reverse is also possible: a shadow may be falsely detected as a solid obstacle.
Spatial reasoning benefits from explicit uncertainty bounds. A good result can be ‘the path is feasible under this map; clearance at the doorway is unverified’. That sentence is more informative than an unqualified ‘route clear’. Students can apply the same habit when sketching geometry from incomplete diagrams.
13. Object identity across different views
A camera sees a blue box, then the camera rotates and sees another blue box. Are they the same object? Colour alone does not establish identity. A robust tracker combines spatial continuity, shape, timing and other evidence to maintain or revise object identity.
With multiple cameras, observations may have different coordinate systems, fields of view and occlusions. A fusion system needs calibration and correspondence rules before asserting that two detections refer to one object. Two visually identical notebooks in neighbouring trays provide a simple example: merging their identities would corrupt inventory counts even if the object detector worked perfectly.
Do not ask a language model to resolve these ambiguities through confidence alone. Keep a record of matching evidence and permit an ‘unresolved’ identity. Later measurements may disambiguate the scene. Again, the intelligence lies partly in refusing an unjustified assumption.
14. Mapping, planning and control operate at different speeds
A static floor plan can be prepared slowly. Obstacle detection during movement may need fresher observations. Low-level motor control can operate under tighter timing requirements than a language model’s planning conversation. Combining everything into one slow text loop is often inappropriate for physical control.
An embodied AI architecture can therefore allocate work: perception estimates the current situation, a planner chooses a constrained objective, and a controller handles time-critical actuation within safety boundaries. The planner should be able to stop, replan or request human help when conditions change.
Performance must be evaluated per layer. A model might answer spatial questions accurately offline but still be unsuitable for real-time control. Conversely, a robust controller can execute safe movements without understanding the human purpose unless the task planner connects it to the correct goal.
15. Test the model by changing one spatial assumption
Suppose the Study Cart’s corridor at y=3 is blocked at (4,3). Our original seven-move route is no longer valid. The lower bound of seven steps still holds mathematically, but meeting that bound requires some other path. A new shortest-path search must account for the changed obstacle map.
If a planner returns the old route merely because it was successful yesterday, it is using a stale world representation. A good evaluator changes one obstacle, changes the start orientation or introduces a new clearance constraint, then checks whether the system appropriately revises its conclusion.
Spatial evaluation should test viewpoint changes, mirrored layouts, occlusion, unexpected obstacles and the difference between reachable and merely visible. Success on one familiar picture is not evidence of general spatial competence. Tests need ground truth and well-defined acceptance rules.
16. Distinguish a description from a usable action plan
An answer such as ‘go around the bookshelf and stop near the table’ may be helpful to a person, but it lacks distances, clearance requirements, exact targets and a stopping rule for an automated cart. An actionable plan needs a representation that downstream components can interpret and check.
Even a detailed sequence of waypoints does not prove that a robot can execute it. Hardware limitations, localisation, traffic and environmental changes may invalidate the waypoints. The system must verify its present pose and observe progress, not assume every issued command was completed.
The same distinction appears in software agents: stating ‘click the correct button’ is not an observed action, and a completed click is not proof that the intended record changed. Spatial reasoning supplies part of a larger observe-decide-act-check loop, not a substitute for outcome verification.
17. The human-in-the-loop boundary
Moving equipment in a room with people introduces potential physical harms. A language model’s inference should not be treated as a safety-certified control decision. Appropriate sensing, tested safety systems, speed limits, emergency stops, supervision and domain-specific standards matter for real deployments.
The fictional Study Cart does not move in this article; the exercises are paper-and-pencil models. The point is to teach how spatial representations can be checked, not to supply instructions for deploying an autonomous device. When real systems are used, competent engineering and the applicable safety and legal requirements take precedence.
For education, that boundary itself is a valuable concept. Pupils learn that a mathematical route, a simulated motion and an actual safe movement are separate achievements requiring separate evidence.
18. Independent transfer: a doorway and a new reference frame
Consider a different fictional mobile cart whose footprint is approximated as a circle of radius 0.30 metre. A measured clear doorway is 0.55 metre wide. Assume the measurement is exact for this abstract exercise, and ignore any additional safety margin. Does the cart fit through the opening under the circular-footprint model? Explain using widths, not an image impression.
Next, put the cart at world position (4,1) facing west, one hundred and eighty degrees anticlockwise from the world’s east axis. A target one metre directly ahead in the cart’s local forward direction is described by local coordinates (1,0). What is its world position? Explain which part of the earlier transformation rule still works and which numerical result changes.
Finally, a screenshot locates the doorway at pixel x=600, y=150 in an 800-by-600 image. If the detector’s point convention is normalised [y,x] on a zero-to-one-thousand scale, what pair should it report? Try all three questions before reading the answers.
19. Full transfer answers
The circular cart’s diameter is twice its radius, or 0.60 metre. The clear doorway is 0.55 metre, so the cart is nominally 0.05 metre wider than the opening and cannot pass through under the stated rigid circular model. Real deployments require additional margins, not less. The correct result is a rejection, not a hopeful manoeuvre.
Facing west rotates the cart’s local forward vector (1,0) into the world displacement (-1,0). Add that to (4,1) to obtain world position (3,1). The method is unchanged from the earlier ninety-degree example: transform the local direction, then add the world translation. Only the orientation and resulting vector differ.
For the screenshot, y=150/600×1000=250 and x=600/800×1000=750. The [y,x] output is [250,750]. If the API expected [x,y], the pair would be reversed. A spatial model must know the convention before it presents these digits as coordinates.
20. Diagnose five recurring mistakes
First, treating image pixels as metres ignores camera projection and calibration. Second, mixing [x,y] and [y,x] swaps axes. Third, confusing the robot frame with the room frame sends motion in the wrong direction. Fourth, considering the robot a zero-width point can turn an impossible doorway into an apparently easy route.
Fifth, trusting an old observation after a change in the room mistakes the past for the present. Each error has a corresponding repair: document projection and crop, state coordinate order, compose the frame transformations, account for the physical footprint and track observation timestamps.
Many sophisticated errors are ordinary foundational errors hidden inside larger systems. Teaching the basics carefully is not an obstacle to advanced AI; it is how a learner becomes capable of inspecting the advanced system’s claims.
21. How to evaluate a spatial-reasoning system
Build a test set that distinguishes object detection, relation recognition, coordinate transformation, route planning and verified action. For each problem, print the initial map, observations, movement rules, units and acceptance conditions. A system should not get credit for a correct-looking final location if it violated constraints along the route.
Measure failures as well as successes. Record wrong-frame errors, collisions in simulation, excessive detours, uncertain targets, stale observations and inappropriate certainty. Where human safety matters, offline tests are preliminary evidence only; real-world validation belongs to appropriately qualified teams with domain-specific controls.
A useful educational assessment also asks students to explain why an answer fails after an assumption changes. If an algorithm can solve the original grid but cannot notice a narrower doorway, its apparent skill has not yet transferred.
22. Frequently asked questions
Can an AI system understand space from text alone? It can reason about stated relations and geometry, but text alone cannot supply every missing real-world measurement. The validity of its answer depends on the description and assumptions.
Does detecting an object mean knowing its distance? Not automatically. Two-dimensional detection and metric depth estimation are different outputs, with different sensing and calibration needs.
Does the shortest route always make the best route? No. Clearance, safety, uncertainty, turning and energy costs can favour a longer route. Specify the objective function and constraints.
Are robot planning systems examples of actual artificial superintelligence? No. Contemporary perception, planning and robotics tools are examples of AI technologies. Artificial superintelligence is a separate, speculative capability concept and should not be inferred from an impressive demonstration.
23. Authoritative sources and wider eduKate reading
The ROS 2 tf2 documentation explains coordinate frames and time-aware transformations in robotics. Google’s official spatial reasoning documentation describes example interfaces for points, bounding boxes and trajectories. Its robotics overview describes contemporary embodied-reasoning models; product capabilities, previews and supported interfaces can change. The NIST AI Risk Management Framework provides broader voluntary risk-management context, not certification for the Study Cart.
Read the adjacent eduKate material on Vision, Video, Planning and Computer Use. For formal verification of computations, see the newly developed Code Execution chapter. Use the full SI curriculum, How X Works hub and Gold Standard of Education as connected starting points.
All floor grids, images, Study Cart details, routes, test results, safety constraints and transfer cases here are invented teaching fixtures. No physical movement, camera observation, model benchmark or product trial was performed for this article. Source review: 22 October 2026.
Continue to Sensors and Edge Intelligence to see how a measured signal becomes a local decision, or back to Research Agents when a spatial claim needs better provenance. These companion lessons keep map geometry, sensor evidence and interpretation distinct.