VIEW THIS AS

Auto mode follows the Route Engine until you choose a viewpoint.

YOU ARE HERE

ROUTE CHECK

CONNECTED TO

WHAT NEXT

Use the canonical route for this room, or HELP if you are unsure.

How Intelligence Works | When the Spiral Goes Down — How Systems Learn the Wrong Lesson and Become Worse

HOW INTELLIGENCE WORKS · FAILURE · FEEDBACK · VERSIONING · ADAPTATION · eduKateSG

When the Spiral Goes Down

How systems learn the wrong lesson and become worse.

The Spiral Closed Loop gives intelligence a compelling geometry: act, encounter reality, receive evidence, learn, retain change and return as a new version. But there is a trap hidden inside the picture. We naturally draw the spiral upward.

Nothing in a feedback loop guarantees that direction.

A system can observe the world and misunderstand it. It can assign credit to the wrong cause. It can optimise the measurement rather than the underlying job. It can preserve a bad correction. It can become extremely competent at producing the wrong outcome. Experience alone does not guarantee wisdom.

The spiral goes upward only when the information returned from reality is interpreted, tested, retained and transferred well enough to improve future capability.


1. Version 2 can be worse than Version 1

Imagine our car manufacturer receives complaints that a suspension feels too soft. Engineers increase stiffness. The next model feels sharper during short test drives and receives enthusiastic early reviews. Months later, owners discover that the change makes long journeys tiring and increases wear elsewhere.

The company did not ignore feedback. It used feedback.

The failure was that a local signal was interpreted as a complete objective. One dimension improved while the system-level result deteriorated.

V1 → feedback → wrong interpretation → bad update → V2↓

This is the downward spiral: closed-loop change that reduces broader capability.

2. Feedback is not truth

Feedback is evidence about consequences. It is not a complete explanation of those consequences.

A customer complaint tells us that a customer experienced a problem. It does not automatically tell us its cause. A falling examination mark tells us that performance changed under particular conditions. It does not tell us whether the cause was knowledge, timing, question interpretation, fatigue or something else. A sensor reading tells us what the instrument reported, not whether the sensor was calibrated or whether the variable is the one that matters.

This is why the Intelligence architecture contains Evidence Weighting, Precision Weighting, Causal Reasoning, Credit Assignment and Outcome Evaluation. Feedback must pass through judgement before it becomes an update.

3. The proxy takes over

Many downward spirals begin because the real objective is difficult to measure.

A school wants deep learning but can easily count test scores. A company wants durable customer value but can easily count clicks. A hospital wants health but can easily count throughput. A factory wants reliable products but can easily count units passing a particular inspection. An AI system is given a measurable reward that approximates what designers actually want.

The proxy begins as a useful signal. Then incentives gather around it. People and machines become better at increasing the proxy. Eventually the measurement can detach from the underlying job.

When a measure becomes easier to optimise than the reality it was meant to represent, intelligence can improve the score while degrading the system.

4. Local intelligence can create global stupidity

Suppose every department in an organisation improves its own metric. Procurement lowers component cost. Manufacturing raises throughput. Warehousing reduces inventory. Customer service shortens call time. Engineering reduces development time.

Each optimisation can be locally rational and collectively disastrous. Cheaper components increase failures. Lower inventory removes resilience. Shorter calls leave problems unresolved. Faster development moves defects downstream.

This is a systems version of the dot problem. Each node sees only part of the map. Intelligence at the node does not automatically become intelligence at the network.

A genuinely closed organisational loop therefore needs consequences to travel across boundaries. The cost saved by procurement must remain connected to warranty failures. The time saved in development must remain connected to field reliability. Otherwise each department can spiral upward according to its own dashboard while the organisation spirals downward.

5. Overfitting experience

Another downward spiral appears when a system learns too specifically from its past.

A student memorises the surface forms of past examination questions and becomes excellent at recognising them. A company designs around the preferences of its current customers and becomes blind to emerging users. An algorithm fits historical data beautifully and fails when the environment changes. An organisation writes procedures for every incident until unusual situations become impossible to navigate.

The system appears to improve because performance on familiar cases rises. Transfer reveals the weakness.

This is why Transfer and Recomposition belongs near the top of the spiral. An update has not demonstrated robust improvement merely because it solves the case that caused the update. It has to survive changed surfaces and relevant new conditions.

6. The learning rate becomes unstable

A system can also deteriorate because it changes too quickly.

One bad week produces a complete strategy change. One unusual customer triggers a redesign. One surprising experiment overturns a well-supported model. One viral complaint becomes the apparent voice of the entire market.

Then the next signal pushes the system somewhere else.

The result is not learning but oscillation. Learning Rate Control exists because evidence has to change the model in proportion to its reliability, diagnostic value and relationship to what is already known.

7. The learning rate freezes

The opposite failure is rigidity. Past success becomes so heavily weighted that current evidence cannot move the model.

The company keeps building what used to sell. The student keeps using the method that worked at an easier level. The institution keeps a process because nobody wants responsibility for changing it. The model remains calibrated to an environment that no longer exists.

This is where Change Detection, Contextual Inference and Schema Revision become essential. Intelligence must preserve useful structure without turning preservation into paralysis.

8. Memory becomes technical debt

We often describe memory as the mechanism that lets a circle become a spiral. But memory can preserve bad structure as effectively as good structure.

A workaround becomes permanent. A temporary exception becomes policy. A defensive procedure remains after the original risk disappears. An obsolete assumption is copied into a new system because it existed in the old specification.

Now the past is not helping the future climb. It is charging the future rent.

Intelligent versioning therefore needs deletion, retirement, deprecation and reconciliation as well as preservation. A civilisation, organisation or mind that can only remember but cannot revise eventually becomes crowded with incompatible versions of itself.

9. Success can destroy the feedback that created success

A subtler downward spiral begins when improvement removes the conditions that kept the system capable.

Automation eliminates routine human practice, yet humans are still expected to intervene during rare failures. A reliable system produces so few incidents that operators lose diagnostic skill. A high-performing student receives increasing help and therefore gets fewer opportunities to practise independent recovery. An institution becomes so stable that nobody remembers how to rebuild it after disruption.

This connects to Capability Atrophy. The visible output can remain excellent while hidden recovery capability declines.

10. Creativity can accelerate the descent

Creativity is not automatically a rescue mechanism. A system with weak evaluation can generate new interventions faster than it can learn from old ones.

Schools launch initiative after initiative. Companies continually reorganise. Software teams replace architectures before the previous architecture has produced enough evidence. Learners switch study methods whenever a new technique appears online.

The dots move outward beautifully. The loop never closes.

This is why Creativity versus Intelligence matters. Generation expands possibility. Intelligence must decide which possibility deserves testing, how long the test should run, what evidence would count and whether the result should be retained.

11. A downward spiral can look like progress

The dangerous failures are not always obvious collapses. Sometimes every visible number improves.

Output rises while maintenance is deferred. Examination scores rise while transfer falls. Response times fall while unresolved cases accumulate. Production costs fall while resilience disappears. Engagement rises while understanding becomes shallower.

A system can therefore descend while its chosen dashboard points upward.

The most dangerous downward spiral is one whose measurement system calls it improvement.

12. How the spiral earns an upward arrow

The upward arrow has to be earned through several gates.

  • The relevant consequence must be observable.
  • The evidence must reach the system capable of changing the cause.
  • The signal must be weighted according to reliability.
  • The cause must be distinguished from coincidence.
  • The correction must target the mechanism rather than only the symptom.
  • The changed system must be tested.
  • The test must include transfer beyond the triggering case.
  • Useful improvement must be retained.
  • Obsolete knowledge must remain revisable.
  • System-level outcomes must remain visible beyond local metrics.

Only then does Vn+1 deserve to be called an improvement rather than merely a later state.

13. Education makes the downward spiral visible

A student repeatedly gets a question type wrong. The teacher supplies the correct method. The student copies it. The next similar question is correct. Everyone concludes that learning occurred.

Then the surface changes and the student fails again.

The feedback loop improved immediate performance without changing the transferable model. If the system now gives even more procedural help, the learner can become increasingly dependent while appearing increasingly successful during supported practice.

This is a downward spiral disguised as assistance.

Good teaching therefore asks not only, “Did the correction work?” but “What changed inside the learner, and does that change survive independence, delay and variation?”

Discover more from eduKate Singapore

Subscribe now to keep reading and get access to the full archive.

Continue reading