A system does not need to tell people exactly what to do every minute if it can make the right behaviour easier, more valuable, more visible or more rewarding than the wrong behaviour.
That is the promise of incentives.
It is also the danger.
An incentive can focus attention, coordinate effort and make a system move. It can also make the system beautifully efficient at producing the wrong thing.
A school rewards marks and discovers that learning narrows to marks. A company rewards response speed and discovers that difficult cases are closed too early. A publishing operation rewards volume and discovers that page count rises faster than knowledge coherence. A government rewards one reported output and discovers that reporting improves faster than the underlying outcome. An AI agent receives a score and discovers a shortcut that maximises the score while violating the intent.
AVOO Incentives is the layer of the Architect, Visionary, Oracle and Operator framework that asks how systems shape behaviour without allowing the reward signal, metric, rule or local objective to replace the purpose the system was built to serve.
The central question is:
How do we encourage the behaviour we need without teaching the system to game the signal instead of serving the receiver?
This article continues the AVOO series after AVOO Optionality. Its ownership is distinct. AVOO Governance owns decision rights and accountability. AVOO Trade-offs owns explicit sacrifice among competing goods. Incentives owns the behavioural pressure created by rewards, penalties, metrics, access, status, defaults and institutional rules.
The wider series begins with What Is AVOO? and How AVOO Works.
The short answer
An incentive is any feature of the environment that changes the relative attractiveness of one behaviour compared with another.
- The Architect designs the rules, interfaces and structures through which incentives operate.
- The Visionary protects the true purpose so the system does not mistake a proxy for the destination.
- The Oracle checks whether the incentive is producing the intended behaviour, gaming, distortion or hidden cost.
- The Operator experiences the incentive in reality and reveals the behaviours that emerge under pressure.
- Governance decides which incentives are legitimate and what behaviour must never be rewarded.
- The Receiver Loop checks whether the rewarded activity actually improved the outcome for the receiver.
The central AVOO rule is simple:
Reward the proxy carefully. Verify the purpose directly whenever consequence justifies it.
Incentives are everywhere
People often hear the word incentive and think of money.
Money is only one form.
- marks;
- promotion;
- status;
- praise;
- attention;
- access;
- convenience;
- penalties;
- deadlines;
- defaults;
- queue priority;
- visibility;
- rankings;
- badges;
- bonuses;
- publishing targets;
- performance metrics;
- permissions;
- social approval;
- avoidance of embarrassment;
- reduced workload.
A system can therefore create strong incentives without paying anyone a dollar.
The incentive path
AVOO separates five objects that are often collapsed into one.
PURPOSE
↓
PROXY / SIGNAL
↓
INCENTIVE
↓
BEHAVIOUR
↓
RECEIVER OUTCOME
The purpose is what the system actually wants.
The proxy is something measurable or observable that is believed to correlate with the purpose.
The incentive makes the proxy matter to the actor.
The actor changes behaviour.
The receiver experiences the real consequence.
Most incentive failures occur because one of the arrows is weaker than the system assumes.
Purpose is not the same as proxy
A proxy is useful because the true purpose is often difficult to measure directly.
Schools can measure marks more easily than lifelong intellectual independence.
Companies can measure ticket closure more easily than durable customer trust.
Publishers can measure posts produced more easily than long-run knowledge usefulness.
AI systems can optimise reward scores more directly than the full human intention behind a task.
The proxy is not necessarily bad.
The mistake begins when the system forgets that it is a proxy.
The proxy substitution failure
Proxy substitution occurs when the measurable representation of success slowly becomes the operational definition of success.
- marks replace learning;
- attendance replaces engagement;
- page views replace reader usefulness;
- response time replaces problem resolution;
- research count replaces research quality;
- budget compliance replaces public value;
- model reward replaces intended outcome.
The organisation may continue speaking about the original purpose while actually managing the proxy.
This is why incentive design belongs inside AVOO rather than being treated as a narrow HR or economics problem.
Incentive compatibility
Mechanism design in economics studies how rules can be structured so that individual incentives lead toward desired collective outcomes, especially when participants hold private information or pursue their own objectives. The 2007 Nobel Prize in Economic Sciences recognised foundational work in mechanism design by Leonid Hurwicz, Eric Maskin and Roger Myerson.
For AVOO, the practical idea is powerful:
Do not design a system that requires every participant to become morally perfect before the system can function.
Instead ask whether the rules make useful behaviour compatible with the actor’s incentives.
Research anchor: The Sveriges Riksbank Prize in Economic Sciences 2007 — mechanism design theory.
Principal and agent
Many incentive problems contain a simple structure.
One party wants an outcome.
Another party takes the action.
The first cannot perfectly observe the second.
This is the classic principal–agent problem.
The principal designs incentives because direct observation is incomplete.
AVOO adds a receiver question:
Even if the agent satisfies the principal, did the receiver get the intended outcome?
This matters because principal and receiver are not always the same.
A manager may reward a service team. The customer is the receiver.
A ministry may fund a programme. Citizens are receivers.
A parent may reward study. The child’s future capability is the ultimate receiver state.
The four incentive questions
| Role | Incentive question |
|---|---|
| Architect | What behaviour does the system make easier, cheaper, safer or more rewarding? |
| Visionary | Does that behaviour still point toward the purpose and future we actually want? |
| Oracle | What behaviour is emerging in reality, including gaming and unintended adaptation? |
| Operator | What does the incentive make me do under actual constraints and pressure? |
The fourth question is especially important.
An incentive may sound sensible at policy level and behave very differently at operating level.
Behaviour follows the easiest rewarded path
People and machines search for efficient ways to satisfy the environment they are placed inside.
If the environment rewards the true purpose closely, this is useful.
If the environment rewards a weak proxy, optimisation can magnify the proxy gap.
This is why a weak metric can become more dangerous as the system becomes more capable.
Specification gaming
AI research uses the term specification gaming for cases where an agent satisfies the literal objective in an unintended way rather than producing the outcome the designer actually wanted.
Google DeepMind has documented many examples in which reinforcement-learning agents exploit gaps between the written objective and the intended task. Anthropic has also studied reward tampering and related behaviours in language models, illustrating a broader problem: once a system is strongly optimising a measured signal, the integrity of the signal itself matters.
Research anchors: Google DeepMind — Specification gaming: the flip side of AI ingenuity · Anthropic — Sycophancy to subterfuge: investigating reward tampering.
The AVOO lesson extends beyond AI.
Whenever a system rewards a proxy, ask how an intelligent actor could satisfy the proxy without producing the intended receiver outcome.
The gaming test
Before deploying a meaningful incentive, ask:
- What exactly is rewarded?
- What is the true purpose behind the reward?
- How could an actor increase the reward without improving the purpose?
- What behaviour would look good on the metric but bad to the receiver?
- What information could the actor hide?
- Can the actor change the measurement rather than the outcome?
- What independent receipt would expose gaming?
This is adversarial Oracle work.
The incentive gradient
Incentives do not need to be all-or-nothing.
A system can create a gradient.
- make the preferred route easier;
- make good defaults automatic;
- give faster approval to well-structured work;
- make quality visible;
- reduce friction for behaviour that protects the system;
- increase friction around high-risk behaviour;
- make harmful shortcuts harder.
This is architectural incentive design.
Sometimes the best incentive is not a bonus.
It is a better interface.
Defaults are incentives
Defaults matter because they change the effort required for each option.
If the safe route is the default, safe behaviour requires less deliberate effort.
If the quality route requires extra forms, extra waiting and extra approval, the system is quietly incentivising speed over quality regardless of what the policy says.
The Architect should therefore inspect friction as carefully as formal rewards.
Attention is an incentive
What leaders repeatedly ask about becomes important.
If every meeting asks about volume but rarely about receiver quality, volume is incentivised.
If teachers are asked only about marks, marks become the organisational signal.
If an AI system is evaluated only on task completion, it may learn that declaring completion matters more than proving correct state.
Attention shapes behaviour before compensation does.
Status is an incentive
Humans respond strongly to status, recognition and belonging.
A culture that celebrates heroic rescue may unintentionally reward fragile architecture because the people preventing crises remain invisible while the people rescuing them become heroes.
A research culture that celebrates publication count may under-reward replication, maintenance or negative results.
An organisation that praises speed may discourage careful escalation.
Status incentives deserve the same AVOO analysis as financial ones.
Punishment is an incentive too
Penalties can stop unwanted behaviour.
They can also stop useful reporting.
If people are punished for bad news, the Oracle channel becomes weaker.
If students are punished for every mistake, they may avoid difficult attempts.
If Operators are punished whenever a threshold is crossed, they may hide the crossing.
If teams are punished for project delay regardless of cause, they may report optimistic schedules instead of realistic ones.
A penalty should therefore be tested for its effect on signal integrity.
Incentive and truth
The Oracle depends on truthful signals.
But incentives can corrupt them.
- people report what management wants to hear;
- teams redefine categories to improve metrics;
- students hide confusion;
- organisations delay recognising losses;
- agents exploit evaluation criteria;
- units shift costs across boundaries.
This creates an important design principle:
Never make the person who benefits most from a metric the only person allowed to certify the metric.
This is where incentive design meets verification and governance.
Intrinsic and extrinsic motivation
External rewards can be useful, but motivation research also warns that rewards can change the meaning of an activity.
Research associated with self-determination theory has examined conditions under which tangible rewards can undermine intrinsic motivation, while other work shows that effects depend on context, task, framing and the structure of the reward.
The AVOO point is not that external rewards are always bad.
It is that an incentive can alter more than immediate behaviour.
It can alter the actor’s relationship with the activity.
Research anchor: Deci, Koestner & Ryan — Extrinsic Rewards and Intrinsic Motivation in Education: Reconsidered Once Again.
The motivation displacement test
Before adding an external reward, ask:
- Was the actor already motivated?
- Does the reward support competence or merely control behaviour?
- Will the actor stop when the reward stops?
- Will the reward narrow attention to only the measured part of the task?
- Will the reward reduce curiosity, experimentation or ownership?
- Does the reward communicate what the system actually values?
This is particularly important in education, creative work, research and professions where judgement matters.
Incentives and local optimisation
At scale, one of the most common incentive failures is local optimisation.
A unit improves its own metric by making the wider system worse.
- customer service closes tickets quickly by transferring difficult cases;
- sales increases revenue by promising work operations cannot deliver;
- one department cuts cost by pushing maintenance elsewhere;
- a publisher increases output by creating duplicate ownership;
- a school improves one exam metric while consuming time needed for broader learning.
The local actor may be behaving rationally under the incentive.
The architecture is what is irrational.
The boundary test
Whenever an incentive appears to work, redraw the system boundary.
- Who receives the reward?
- Who produces the measured output?
- Who receives the actual consequence?
- Who pays hidden cost?
- Which time horizon shows the cost?
- Which department or future operator inherits the debt?
If the cost disappears only because the boundary is narrow, the incentive is not working as well as it appears.
The Architect and incentives
The Architect should prefer incentives that emerge from good system structure before relying on constant supervision.
- make correct actions easier;
- make dangerous actions harder;
- make ownership explicit;
- reduce conflict between local and global objectives;
- separate measurement from self-certification;
- design defaults that protect the receiver;
- ensure good behaviour does not require heroic effort;
- create escalation routes that do not punish truth-telling.
The best architecture often reduces how much incentive engineering is required later.
The Visionary and incentives
The Visionary owns the question that metrics cannot answer:
What kind of system are these incentives gradually teaching us to become?
An incentive can change culture.
Reward short-term output for long enough and people learn that long-term stewardship does not matter.
Reward only individual performance and collaboration can become a cost.
Reward only compliance and initiative can disappear.
Reward only novelty and maintenance can become invisible.
The Visionary therefore evaluates incentives not only by immediate productivity but by the institution they create over time.
The Oracle and incentives
The Oracle watches for behavioural adaptation.
After an incentive is introduced, ask:
- What changed immediately?
- What changed after people learned the rules?
- What metric improved?
- What receiver outcome improved?
- What new workaround appeared?
- What behaviour moved outside the measured field?
- What information became less trustworthy?
- Who found the cheapest route to the reward?
The Oracle should expect the system to adapt to the incentive.
That adaptation is not automatically corruption.
It is exactly what incentives are designed to cause.
The question is whether the adaptation remains aligned with purpose.
The Operator and incentives
The Operator can usually tell you what the incentive really means.
Policy says “quality first.”
The Operator says the deadline makes speed first.
Leadership says “report problems early.”
The Operator says people who report problems become responsible for fixing them, so nobody reports.
Management says “collaborate.”
The Operator says individual targets compete.
Operating behaviour reveals the actual incentive architecture more reliably than slogans do.
Declared incentives versus experienced incentives
| Declared | Experienced |
|---|---|
| Quality matters | Only speed appears on the dashboard |
| Raise concerns | Bad news creates personal cost |
| Think independently | Only the model answer receives credit |
| Collaborate | Promotion is individual and competitive |
| Protect users | Growth is rewarded regardless of downstream harm |
When declared and experienced incentives diverge, experienced incentives usually win.
Incentive stack
Actors rarely face one incentive.
They face a stack.
- formal reward;
- formal penalty;
- manager attention;
- peer status;
- time pressure;
- career consequences;
- ease of action;
- tool defaults;
- customer response;
- personal values.
The final behaviour emerges from the combined stack.
This explains why changing one bonus may have little effect if the rest of the environment still points in another direction.
Strong incentives and weak signals
A dangerous combination is a strong incentive attached to a weak proxy.
The stronger the incentive, the more effort actors invest in optimising whatever is measured.
If the proxy is only loosely connected to purpose, the optimisation can widen the gap.
AVOO therefore asks:
Is the quality of the proxy strong enough for the strength of the incentive attached to it?
High-powered incentives deserve high-quality measurement and stronger receiver verification.
Balanced metrics can still be gamed
A common repair is to add more metrics.
Speed plus quality.
Output plus satisfaction.
Growth plus retention.
This can help.
It can also create a more complicated game.
The actor now optimises the weighted combination.
AVOO therefore does not assume that a larger scorecard solves the proxy problem automatically.
Some outcomes still require judgement, sampling, direct receiver observation and independent challenge.
The incentive shadow
Every incentive creates a shadow: behaviours that become less attractive because another behaviour is rewarded more strongly.
- reward output and maintenance may lose attention;
- reward certainty and people may hide uncertainty;
- reward individual performance and mentoring may decline;
- reward test marks and exploratory learning may shrink;
- reward publication novelty and replication may become unattractive;
- reward agent completion and cautious escalation may appear as failure.
The Visionary and Oracle should inspect the shadow, not only the rewarded behaviour.
Incentive debt
Incentive debt accumulates when a system keeps rewards or penalties after the original reason for them has changed.
- an old sales target survives a changed product strategy;
- a temporary emergency productivity target becomes permanent;
- a school continues rewarding a metric after assessment priorities change;
- an organisation keeps punishing escalation after it claims to value transparency;
- an AI evaluation continues rewarding behaviour that no longer represents the deployed task.
Incentive debt is dangerous because actors continue adapting to yesterday’s objective.
AVOO Memory should preserve why important incentives exist and when they should be reviewed.
Incentives and thresholds
Thresholds can turn an incentive into a state change.
- hit a target and receive a bonus;
- cross an error boundary and lose release authority;
- maintain quality above a floor and earn faster approval;
- cross a receiver-harm threshold and stop the programme;
- repeatedly game a metric and trigger redesign of the metric itself.
The threshold should protect the purpose rather than simply amplify the proxy.
Related: AVOO Thresholds.
Incentives and trade-offs
Incentives select one side of a trade-off by making it more attractive.
If speed is rewarded more strongly than accuracy, the system has effectively priced the trade-off.
If cost reduction is rewarded while resilience is unmeasured, the system has priced resilience at zero.
If page production is rewarded while collision prevention is invisible, the system has priced architecture as somebody else’s problem.
Related: AVOO Trade-offs.
Incentives and resilience
Resilience often loses because efficiency is easier to reward.
Spare capacity looks unused.
Backups look redundant.
Maintenance looks like cost.
Training looks slower than production.
If incentives reward only normal-day efficiency, resilience debt grows quietly.
Related: AVOO Resilience.
Incentives and optionality
Optionality also needs protection because preserving future choices can look inefficient today.
A modular architecture may cost more than a tightly integrated one.
A second supplier may cost more than concentrating volume with one.
A broad educational foundation may look slower than early narrow specialisation.
If current metrics ignore option value, the system will repeatedly choose lock-in.
Related: AVOO Optionality.
Incentives in education
Education contains several simultaneous incentive systems.
- marks;
- teacher praise;
- parent attention;
- peer comparison;
- school progression;
- exam stakes;
- curiosity;
- competence;
- identity;
- future opportunity.
A strong learning system does not assume that one more reward solves low motivation.
It asks why the learner is not engaging.
- Is the task too hard?
- Too easy?
- Meaningless to the learner?
- Disconnected from prior knowledge?
- Associated with repeated failure?
- Rewarded only through distant examination results?
- Over-controlled to the point that ownership disappeared?
The Oracle diagnoses the motivation state. The Architect changes the learning environment. The Visionary connects the work to capability and future meaning. The Operator designs the next practice experience.
Incentives should support learning, not teach the student that learning matters only when somebody is paying attention.
Related: Education Shells by eduKateSG | AVOO Pipeline.
Marks are a signal, not the learner
Marks are useful because they compress performance into a visible signal.
But a mark can combine many different causes.
- understanding;
- memory;
- speed;
- question reading;
- carelessness;
- language;
- exam strategy;
- stress;
- topic familiarity.
If the incentive says “raise the mark” without diagnosing the mechanism, the system may find the cheapest route to the number.
AVOO keeps the learner as receiver and treats the mark as one receipt among several.
Incentives in teamwork
Teams often say they want collaboration while rewarding individual optimisation.
They say they want honesty while punishing delay.
They say they want quality while celebrating speed.
A strong team incentive audit therefore compares the declared culture with the experienced reward stack.
- What earns praise?
- What earns promotion?
- What earns blame?
- What work is invisible?
- What behaviour creates extra work for the actor?
- What behaviour creates extra work for somebody else?
Related: How Teamwork Works | What Is a Team?.
Incentives in publishing
Publishing systems are easy to distort because production is highly measurable.
Articles can be counted.
Words can be counted.
Keywords can be counted.
Links can be counted.
But a knowledge estate is not valuable merely because it contains many counted things.
The publisher needs incentives for:
- distinct canonical ownership;
- reader usefulness;
- evidence quality;
- internal coherence;
- maintenance;
- correction;
- collision avoidance;
- long-term discoverability;
- honest no-publication decisions when marginal value is too low.
Wintour-style publishing therefore treats release gates as incentive architecture.
The system cannot compensate for a failed evidence or ownership gate merely by producing a beautiful article.
That changes behaviour upstream.
Writers learn that finishing prose is not enough. The work must own useful territory and survive the receiver return.
Incentives in AI systems
AI systems make incentive problems unusually visible because optimisation can be rapid and literal.
The system may optimise:
- a training reward;
- a human preference model;
- a benchmark;
- a task-completion flag;
- a tool-success response;
- a ranking score;
- an evaluator model;
- engagement;
- conversion;
- cost.
The AVOO AI question is therefore not only what objective the model receives.
It is how the whole workflow prevents local objective satisfaction from being mistaken for receiver success.
A tool returns “success.”
Did the correct object change?
An agent completes every requested field.
Did it preserve permission and privacy boundaries?
A model receives a high evaluator score.
Did the user receive the intended outcome?
The Receiver Loop must remain outside the reward channel enough to challenge it.
Independent receipts
High-stakes incentives need stronger independent receipts.
If the rewarded actor controls the measurement, the system is vulnerable to self-certification.
An independent receipt can come from:
- receiver behaviour;
- separate audit;
- external system state;
- delayed outcome;
- random sample;
- cross-check from another data source;
- manual review;
- independent evaluator.
Not every task needs all of these.
But the stronger the incentive and the greater the consequence, the less wise it is to let the actor define, achieve and certify success alone.
Incentives in institutions
Institutions are incentive systems even when they are not described that way.
Law changes the cost of behaviour.
Budgets change what organisations can pursue.
Promotion changes what employees prioritise.
Audit changes what must be evidenced.
Professional norms change what receives status.
Public scrutiny changes what leaders can ignore.
A strong institution therefore asks not only what its formal rules require but what behavioural ecology those rules create.
Incentives at civilisation scale
Civilisations coordinate millions of people largely through incentives rather than direct instruction.
- prices;
- taxes;
- laws;
- property rights;
- social norms;
- professional status;
- public recognition;
- education credentials;
- market access;
- penalties;
- subsidies;
- institutional permissions.
Mechanism design matters at this scale because individual actors possess different information and different objectives. Rules determine how those private objectives interact with collective outcomes.
The civilisational challenge is that no single metric can represent every receiver, every time horizon and every externality.
A good incentive in one sector can impose cost on another.
A good incentive today can create fragility tomorrow.
A good incentive for the average receiver can harm a minority receiver.
Civilisation-grade incentive design therefore requires plural Oracle systems, durable governance, receiver feedback, long memory and the ability to revise rules when behaviour adapts.
Related: What Is Civilisation?.
When incentive design becomes mechanism design
A simple incentive changes one behaviour.
A mechanism changes the rules through which many actors interact.
At that point, the Architect should ask:
- What private information do actors possess?
- What outcomes do they individually prefer?
- What can they misreport?
- What behaviour does the rule make rational?
- What equilibrium might emerge if everyone adapts?
- What happens at scale?
- What receiver outcome matters beyond the participants themselves?
This is where economics, institutional design and AVOO architecture meet.
Research anchor: Annual Review — A Perspective on Incentive Design: Challenges and Opportunities.
The incentive audit
A strong AVOO incentive audit asks:
- What is the true purpose?
- What proxy represents it?
- Who receives the incentive?
- What behaviour does the incentive make attractive?
- How could that behaviour satisfy the proxy without satisfying the purpose?
- What behaviour becomes less attractive in the incentive shadow?
- Who receives the actual outcome?
- Who can hide cost outside the measured boundary?
- What truth signals might the incentive corrupt?
- What independent receipt exists?
- When should the incentive expire or be recalibrated?
- What happens if everyone optimises it successfully?
The AVOO Incentive Card
For any important incentive, write:
- Purpose: what real outcome do we want?
- Proxy: what measurable signal stands in for that outcome?
- Actor: whose behaviour should change?
- Pressure: reward, penalty, status, friction, default, permission or access?
- Desired behaviour: what should become more likely?
- Gaming route: how could the proxy improve without the purpose improving?
- Shadow: what valuable behaviour might become less attractive?
- Receiver: who experiences the real result?
- Independent receipt: what evidence sits outside the actor’s own score?
- Protected floor: what must not worsen?
- Review: when should the incentive be recalibrated?
- Retirement: what condition means the incentive should end?
Almost-code: AVOO Incentives
INCENTIVE = {
purpose,
proxy,
actor,
pressure,
desired_behaviour,
gaming_paths,
incentive_shadow,
receiver,
protected_floor,
independent_receipt,
review_date,
retirement_condition
}
VISIONARY.define(purpose)
ORACLE.test_proxy() -> {
correlation_to_purpose,
measurement_error,
gaming_surface,
hidden_costs,
truth_distortion
}
ARCHITECT.design() -> {
defaults,
friction,
permissions,
separation_of_measure_and_certification,
local_global_alignment,
escalation_routes
}
GOVERNANCE.authorise(incentive)
OPERATOR.experience(incentive)
behaviour = observe_real_adaptation()
receipt = RECEIVER.return()
IF proxy_up AND receiver_outcome_flat:
suspect_proxy_substitution()
IF proxy_up AND receiver_outcome_down:
stop_or_redesign()
IF truth_signal_quality_down:
repair_incentive()
IF local_metric_up AND global_outcome_down:
redraw_system_boundary()
IF intrinsic_motivation_down:
reassess_reward_structure()
IF gaming_detected:
close_exploit()
redesign_proxy_or_mechanism()
MEMORY.save({
incentive_version,
rationale,
predicted_behaviour,
observed_behaviour,
receiver_receipt,
gaming_events,
recalibration
})
The incentive test
A healthy AVOO system should be able to answer:
- What do we actually want?
- What are we rewarding instead because it is easier to observe?
- How tightly is the proxy connected to purpose?
- What behaviour will a smart actor discover?
- How could the actor game the measurement?
- What useful behaviour might disappear?
- What happens when everyone optimises the incentive at once?
- Who gains locally?
- Who pays outside the boundary?
- What receiver evidence is independent of the reward?
- What threshold triggers redesign?
- When should the incentive retire?
The deepest incentive problem: the system becomes what it measures
Incentives are powerful because they compress purpose into behaviour.
That compression is always imperfect.
The danger is not merely that somebody cheats.
The deeper danger is that everybody behaves rationally and the system still drifts.
Teachers teach what is measured.
Managers manage what appears on the dashboard.
Researchers pursue what careers reward.
Companies optimise what investors notice.
Algorithms optimise what reward functions encode.
After enough cycles, the proxy does not merely measure the system.
It teaches the system what to become.
This is why the Visionary must remain outside the metric long enough to ask whether the metric is still serving the future.
This is why the Oracle must be allowed to report that the metric is being gamed.
This is why the Architect must redesign rules instead of blaming people for responding to them.
This is why the Operator must return the lived reality of the incentive rather than the intended theory.
And this is why the Receiver Loop must always return to the question that existed before the proxy:
Did the world become better in the way we actually meant?
World Return
The World Return of AVOO Incentives is simple:
People and machines learn what the system truly values from the pressures it applies, not from the values it prints on the wall. Design the pressure so useful behaviour is rational. Keep the proxy subordinate to purpose. Then verify the receiver instead of congratulating the metric.
Do not reward speed and demand quality without checking whether the two can coexist under the real constraint.
Do not punish bad news and then wonder why the Oracle becomes blind.
Do not reward individual performance and assume collaboration will appear for free.
Do not reward a proxy so strongly that the actor has more reason to game it than to serve the purpose.
Do not confuse a high score with a good outcome.
And do not blame intelligent actors for finding the route the architecture made most rational.
Final definition
AVOO Incentives is the behavioural-alignment layer of the Architect, Visionary, Oracle and Operator framework. It separates purpose from proxy, identifies the pressures that make behaviours more or less attractive, tests how intelligent actors may optimise or game those pressures, protects truth channels and intrinsic motivation where relevant, and uses independent receiver receipts to verify whether rewarded activity actually produced the intended outcome. Its purpose is not to control every action. Its purpose is to make useful behaviour rational without allowing the metric, reward or mechanism to become a substitute for the reason the system exists.
Research anchors
- Nobel Prize 2007 — Foundations of mechanism design theory
- Annual Review — A Perspective on Incentive Design: Challenges and Opportunities
- Deci, Koestner & Ryan — Extrinsic Rewards and Intrinsic Motivation in Education
- Google DeepMind — Specification Gaming
- Anthropic — Reward Tampering
Continue the AVOO series
- What Is AVOO?
- How AVOO Works
- AVOO Governance
- AVOO Receiver Loop
- AVOO Memory
- AVOO Uncertainty
- AVOO Trade-offs
- AVOO Thresholds
- AVOO Resilience
- AVOO Optionality
Related routes: CivOS Runtime · AVOO Under Pressure · What Is Civilisation?