An AI agent cannot reliably learn whether its work helped simply by reading its own final answer or counting positive reactions. Real outcomes often arrive later, are never reported, or are influenced by the agent’s earlier recommendations. A useful feedback system distinguishes proposed actions, actual execution, observed user results, pending evidence and missing labels. Without those distinctions, an assistant can mistake selective praise, an incomplete tool receipt or its own influence on future data for proof that it was correct.
The practical method is to assign a stable task identity, define an externally meaningful outcome before acting, preserve the prediction and action history, wait for an appropriate observation window, distinguish missing data from negative results and investigate whether recommendations changed the population being measured. This chapter works through 120 fictional learner-support tasks, reconciles delayed labels and computes honest bounds on unknown outcomes. It also offers a changed-context transfer exercise and a safe monitoring design. Nothing here assumes artificial superintelligence exists.
This is a post-deployment companion to the How Super Intelligence Works curriculum. The agent experiments guide examines causal comparisons, while review sampling addresses population selection. This article focuses on how later real-world outcomes are obtained, attributed, interpreted and sometimes distorted by the AI system itself.
Choose a reading route
- Studying AI performance? Start with prediction versus observation, missing labels and the 120-task case.
- Managing a deployed agent? Examine feedback contracts, selective observations, leakage and post-deployment monitoring.
- Teaching Mathematics or Science? Solve the delayed-label fractions and the school-notice transfer challenge.
- Concerned about privacy and user rights? Focus on correction, recourse, selective feedback and what an outcome label can legitimately claim.
Contents
- 1. A final answer is not an outcome observation
- 2. Define prediction, intervention and observation separately
- 3. Feedback can arrive hours, weeks or never
- 4. Why missing feedback is not a negative label
- 5. The feedback loop changes the world it measures
- 6. Build a task-level feedback contract
- 7. A fictional 120-task follow-up laboratory
- 8. Resolve delayed cases without rewriting history
- 9. Report delayed labels in a proper table
- 10. Diagnose selection bias before ranking versions
- 11. Guard against training on the answer you want to predict
- 12. Distinguish monitoring from self-reinforcing optimisation
- 13. Make post-deployment monitoring actionable
- 14. Protect privacy and prevent feedback manipulation
- 15. A transfer exercise: delayed school-notice confirmations
- 16. Teach students to interpret missing data responsibly
- 17. Limits, residual uncertainty and the next safe decision
- Frequently asked questions
- Primary sources and research
- Learning pathways
1. A final answer is not an outcome observation
Suppose an assistant recommends a study plan to a Secondary school learner. The model might describe the plan as ‘optimised’, but that is a proposal, not evidence of learning. The learner may use part of the plan, change it, ask a teacher for help or stop practising altogether. A later improvement could reflect several influences; a disappointing result could have causes outside the proposed plan. An outcome system must observe what actually occurred and make clear what can and cannot be attributed to the agent.
Different tasks demand different outcomes. For a file update, independently verified saved content may be a meaningful immediate endpoint. For an email, recipient, sending permission and actual delivery or provider state matter. For learning, understanding and transfer can require follow-up work rather than a same-day thumbs-up. For a source summary, a domain reviewer may need to check factual support. A single universal ‘user satisfied’ signal collapses these distinct jobs into a vague number.
The agent can help collect evidence, but it should not be allowed to declare its own success based solely on generated text. If a connector says an action was accepted for processing, that is not the same as an observed completed effect. If a model says a student has mastered algebra after reading one explanation, it is making a stronger educational claim than the evidence supports. Set the outcome’s independent verification rule before deployment.
Feedback also carries a time dimension. A learner might find a lesson clear immediately but struggle with a related problem a week later. A corrected timetable may be accurate when written and obsolete after a policy change. A calendar reminder may not be judged useful until the scheduled event occurs. Store the relevant observation window and source version so that feedback is interpreted in the right context.
Return to contents or continue with Define prediction, intervention and observation separately.
2. Define prediction, intervention and observation separately
A prediction is what the system expected or proposed. An intervention is what the user or tool actually did. An observation is evidence collected after some event. Those three records should have distinct identifiers and timestamps. An agent might predict that a learner needs practice with fractions, recommend an exercise and later receive an independently checked fraction problem. Treating the predicted difficulty as the actual post-intervention outcome would be circular.
Consider an invented learning assistant that proposes two practice problems for a fictional student, Noor. The learner completes only the first, and a teacher later marks an independent third problem. The model’s suggested answer, the learner’s practice activity and the later assessment are different artifacts. Even if all three use the word ‘fractions’, they answer different questions. A useful dataset should not merge them into one text field called feedback.
Keep the assignment policy too. Why was a particular practice set shown to the learner? Was it randomly assigned, selected by a teacher or recommended because the model believed the learner was struggling? This matters when comparing outcomes. Students who receive extra help may initially have more difficulties than students who do not, so a simple comparison can make the intervention look harmful or helpful for reasons unrelated to its true effect.
Never infer that an agent’s proposed action took place simply because its plan contains a step. A system can recommend opening a document but the user may not open it; it can request publication but WordPress may retain a draft. Record the tool’s actual revision or the learner’s verified activity when permitted. Post-deployment measurements should relate to observed exposures and actions rather than the model’s aspirational workflow.
Return to contents or continue with Feedback can arrive hours, weeks or never.
3. Feedback can arrive hours, weeks or never
Many outcome labels are delayed. An appointment reminder may be judged after the appointment; a learning plan may require several practice sessions; a repair recommendation may be validated after the fault recurs or the equipment remains stable for a period. Reporting all still-pending cases as failures penalises activities whose observation windows have not matured. Reporting them as successes manufactures evidence. Give them an explicit pending state.
Choose an observation horizon appropriate to the task. A school event notice could be checked against an authoritative source before publication and then monitored for subsequent updates. A study-transfer task might use an independent changed-condition problem after a stated interval. A support query could be marked resolved only when its actual result is confirmed, not merely when the agent stops producing text. These choices should be stated before evaluating the results.
Not every delay is harmless. A feedback system can lose labels because a user cannot access the follow-up form, a vendor receipt is unavailable or a monitoring pipeline fails. Such missingness deserves investigation. If an organisation records feedback only for people who click a visible survey link, its reports may exclude those who use other input methods or leave the process early. Operational accessibility and sampling design therefore affect observed outcome quality.
Add a stable maturity field rather than calculating one giant percentage across mixed-age tasks. A one-day-old plan and a two-month-old plan may be at different stages of evaluation. A dashboard can show matured, still pending and not observable for the agreed reason, each with counts and definitions. If the source or evaluation criterion changes, retain the version boundary rather than silently reclassifying all history.
Return to contents or continue with Why missing feedback is not a negative label.
4. Why missing feedback is not a negative label
A missing response can mean many things. The user may have completed the task and moved on, become dissatisfied and left, lost connectivity, declined to provide personal information, used an inaccessible feedback widget or never reached the stage that the survey asks about. These causes have different relationships to success. Assigning zero success to every missing case assumes a meaning that has not been established.
Missingness may also be selective. People with unusually good or bad experiences may be more motivated to respond. Users with limited time may skip surveys. A provider may return more detailed outcomes for some action types than for others. If the probability of having a label depends on the unseen outcome or relevant user characteristics, a simple observed fraction can be biased. A neat confidence interval around that fraction does not cure the underlying selection problem.
Keep at least three statuses: observed positive under a predefined rule, observed negative under that rule and unobserved. For time-sensitive tasks, add ‘not yet mature’ rather than merge it with permanently missing. Explicitly store known inapplicable cases too. A source-verification test and a learner-memory assessment should not share one undifferentiated label simply because both use the words pass and fail.
A careful report can still be useful without full labels. State how many tasks were started, which reached the observation window, how many have valid follow-up, and what proportion of the labelled subset met the criterion. Then explain what is not known about the rest. Honesty about missing data helps people design better follow-up and accessibility rather than rewarding a system for suppressing inconvenient outcomes.
Return to contents or continue with The feedback loop changes the world it measures.
5. The feedback loop changes the world it measures
A model’s recommendation can affect the data that later trains or evaluates it. Google’s Machine Learning Crash Course describes feedback loops in production ML systems, including recommendation popularity that changes because the system promotes certain items. The same mechanism can appear in AI agents. An assistant that repeatedly recommends one study resource may increase clicks on it, and a later popularity-based ranker may interpret those clicks as independent evidence that it was inherently better.
Suppose a fictional study assistant displays Worksheet A far more often than Worksheet B. A naturally larger number of A clicks does not prove A is a better teaching resource. Students cannot click resources they never see, and the placement itself affects attention. The system has intervened in the opportunity to generate feedback. Compare results against exposure and relevant context, not merely total interactions.
Feedback loops can be helpful when controlled. A tutor may observe that learners struggle with a concept and adapt the next practice step. But adaptation should be grounded in independent work or qualified assessment rather than in the model grading its own demonstration. A self-referential loop can reinforce an error: the agent recommends based on an incorrect assumption, generates exercises reflecting it and then treats agreement with those exercises as confirmation.
Human behaviour also adapts. Users may learn how to obtain a favourable model response, avoid giving feedback on difficult tasks, or rely on the system so heavily that later independent performance changes. Monitoring must ask whether the agent’s outputs altered who was exposed, which outcomes could occur and which labels were subsequently available. A feedback signal is not a passive mirror of an untouched world.
Return to contents or continue with Build a task-level feedback contract.
6. Build a task-level feedback contract
Define the intended outcome in language a responsible reviewer can test. A task contract might say ‘the saved article contains the correct approved title, full text, published status and validated links’ or ‘the learner correctly solves an independent transfer problem without viewing the worked answer’. The contract should identify the observation time, allowed evidence, data access, who can adjudicate disputes and what counts as an unknown result.
Record task ID, proposal version, actual intervention, source revision, tool or learning evidence, observation window, label status, reviewer and outcome revision. Keep enough information for a different person to reconstruct the judgement while minimising personal data. An event trace may show that a tool call occurred, but it is not the same as observing a successful user outcome. Preserve those layers separately.
Do not use model-generated explanations as ground-truth labels merely because they are available at scale. A second model can assist with triage or find suspicious claims, but it may share biases and hallucinations with the system under review. For consequential cases, use appropriate independent sources, reliable tool records or qualified human judgement, and label automatic proxies as proxies rather than facts.
Define an appeal or correction path. A student might challenge an assessment because the prompt was ambiguous. A parent might identify a timetable source that had been superseded. A vendor may later deliver a missing receipt. The outcome dataset should support revisions with provenance instead of overwriting an earlier judgement as though it never existed. An accurate history is essential for understanding monitoring changes.
Return to contents or continue with A fictional 120-task follow-up laboratory.
7. A fictional 120-task follow-up laboratory
Use an invented Riverbend Learning Desk that delivers 120 study-support task proposals over a defined review window. At the first observation cut-off, seventy have a valid independently checked follow-up outcome, thirty are still inside the planned observation delay and twenty have no usable follow-up evidence. The three mutually exclusive categories sum to 70 + 30 + 20 = 120. The population is task instances, not distinct children; no real learner records are used.
Among the seventy matured and labelled outcomes, forty-nine meet the predefined independent follow-up criterion and twenty-one do not. The observed positive fraction is 49 ÷ 70 = 70%. This is a conditional percentage among the observed subset. It is not proof that seventy percent of all 120 tasks succeeded or that the agent caused the improvement. The model’s proposal may not have been followed, and the labelled subset may be selective.
The count of independently confirmed positive outcomes relative to all 120 proposals is 49 ÷ 120 ≈ 40.83%. That number is a coverage description: about forty-one percent of proposed tasks already have a positive recorded outcome under the exercise’s observation rules. It is not an estimate that only forty-one percent ultimately helped. Some pending tasks may become positive, and some missing outcomes may remain unknowable. State what the denominator represents.
Similarly, the twenty-one observed negatives cannot automatically be described as twenty-one harmful agent interventions. A learner might not complete the plan, an external event might disrupt practice, or the outcome measure might reveal a concept that was already difficult. The label says whether a specific predefined criterion was met under the observed conditions. Attribution and causality require additional study.
Return to contents or continue with Resolve delayed cases without rewriting history.
8. Resolve delayed cases without rewriting history
Suppose the thirty pending tasks later mature. Eighteen produce positive follow-up evidence and twelve produce negative evidence. The observed positive total becomes 49 + 18 = 67. The observed negative total becomes 21 + 12 = 33. There are now one hundred labelled outcomes and twenty still without usable feedback. The original review cut-off is preserved as a historical snapshot; the later figures are a new version, not a silent correction.
The observed positive fraction among labelled outcomes is now 67 ÷ 100 = 67%. The initial observed fraction was 70% among seventy labels. The difference is three percentage points, but neither figure alone establishes a change in the underlying agent’s performance. The later thirty tasks may represent a different task mix or delay profile, and the twenty missing labels may be systematically unlike the observed ones.
For the full 120 proposals, the confirmed positive count is now 67, giving 67 ÷ 120 ≈ 55.83% confirmed-positive coverage. If all twenty currently unobserved outcomes were ultimately classified negative, the final positive fraction would be 67/120. If all twenty were positive, it would be (67 + 20)/120 = 87/120 = 72.5%. Under the exercise’s binary outcome definition, these are logical extreme bounds, not a probabilistic confidence interval.
Any narrower estimate for the missing twenty requires further assumptions or new evidence about how missingness relates to outcomes. The model cannot responsibly fill the gap by predicting the most convenient labels and then calling them observed. A useful next step is targeted follow-up, selection-bias analysis or a redesigned measurement system that increases observable outcomes without intruding on users.
Return to contents or continue with Report delayed labels in a proper table.
9. Report delayed labels in a proper table
Use rows for observed positive, observed negative, not yet mature and missing evidence. Show initial and later counts with explicit totals. A reader can verify that both snapshots cover the same 120 task proposals without interpreting a blank cell as zero performance. The table should also name the follow-up criterion and date. It is a measurement record, not a promotional scorecard.
Separating pending from missing is especially important. The thirty pending tasks were expected to become measurable later; the twenty missing tasks lacked usable evidence at that cut-off. The later snapshot resolves the pending thirty but not the missing twenty. A dashboard that lumps all fifty as ‘unsuccessful’ at the initial time manufactures a sixty-eight? No: it would fabricate fifty negative outcomes, and obscure the true count of twenty-one observed negatives. The correct arithmetic keeps observed and unknown categories distinct.
An operator can calculate label completeness. Initially seventy of 120 proposals had valid outcomes, or 70 ÷ 120 ≈ 58.33%. Later one hundred of 120 had labels, or 100 ÷ 120 ≈ 83.33%. Completeness improves by twenty-five percentage points. That progress describes observability, not necessarily model quality. Better measurement can reveal more failures without making the product worse.
The table should direct work. Pending items need scheduled rechecks at their legitimate observation horizon. Missing-evidence items need diagnosis of why no label exists. Negative outcomes may need source review, human assessment or user support. Positive outcomes may still be checked for reliability and unexpected harmful side effects. A mature feedback programme turns measurement categories into responsible actions rather than collapsing them into one colour.
| Outcome state | First review | Later review |
|---|---|---|
| Observed positive | 49 | 67 |
| Observed negative | 21 | 33 |
| Awaiting observation window | 30 | 0 |
| Missing usable feedback | 20 | 20 |
| All proposed tasks | 120 | 120 |
Return to contents or continue with Diagnose selection bias before ranking versions.
10. Diagnose selection bias before ranking versions
Suppose one agent version encourages users to leave feedback by showing a prominent survey, while another hides the option at the end of a long workflow. The first version may collect more complaints and more praise simply because feedback is easier. Comparing raw feedback totals is not a fair quality comparison. Record the denominator of eligible users, actual exposure to the survey, completion rate and other relevant selection factors.
Feedback can also depend on difficulty. An adaptive tutor may show harder tasks to stronger students and remedial tasks to learners who are struggling. The groups do not start with the same needs. A naive comparison of later scores can reflect the assignment policy rather than the effectiveness of the suggested practice. Establish the causal question before choosing a comparison method.
Randomised assignment or carefully designed experiments can help answer whether a change caused an improvement under suitable ethical and operational constraints. The agent experiments chapter develops those methods. This guide’s immediate job is to preserve and qualify the outcome evidence that such an experiment would need, including exposure, delayed observation and missing labels.
Where randomisation is inappropriate, use careful observational analysis and acknowledge confounding. Keep versioned task strata, source state and selection logic. Do not claim that a new model improved learning because high-performing students were more likely to receive it. An honest observational result may suggest where to investigate next without proving causation.
Return to contents or continue with Guard against training on the answer you want to predict.
11. Guard against training on the answer you want to predict
Label leakage occurs when information from the outcome or a future event enters features that would not have been available at the moment of prediction. Google’s production ML guidance advises monitoring leakage and ensuring that training resembles serving conditions. Agent systems can suffer a related error when a model is evaluated on data containing the final human review or tool result it is supposedly trying to predict.
Imagine training an assistant to predict whether a helpdesk case will need escalation. A dataset accidentally includes a field populated only after the case has been escalated. Offline accuracy looks spectacular, but the field does not exist at the real decision time. A model may learn the proxy instead of recognising genuine risk. The problem is not solved by using a more capable model; it requires correct time alignment and feature control.
Agent-generated content can leak into its own labels. If a model drafts a summary and a second model judges that the summary agrees with the draft’s conclusions, the system may reward internal consistency rather than external truth. To measure source accuracy, the evaluation must inspect the actual cited material and claim scope. To measure task completion, the evaluation needs the real target resource state. Self-consistency is a different, weaker property.
Version data by availability time. Store when the source was published, when a feature became observable and when the outcome label was created. If retrospective data are used for training, ensure the model’s features reflect what it could have known at action time. A factual source published after an earlier decision may be useful for later evaluation but should not be smuggled into the historical prediction as though it had been available then.
Return to contents or continue with Distinguish monitoring from self-reinforcing optimisation.
12. Distinguish monitoring from self-reinforcing optimisation
Monitoring asks whether deployed behaviour remains reliable in actual conditions. Optimisation changes future behaviour based on feedback. These are connected but separate activities. A team may monitor a new assistant for evidence of source drift without automatically retraining the model whenever a user clicks a rating. Updating a model or routing rule should follow a controlled process that assesses bias, new risks and whether the label means what designers think it means.
Google’s production ML materials warn that model predictions can change the world from which later training data come. A learning-resource recommendation is a practical example: suggested materials gain exposure, exposure creates clicks, and clicks can influence future suggestions. If popularity is taken as independent educational quality, the loop can repeatedly favour already-visible items. Track what was shown as well as what was selected.
An unchecked feedback loop may narrow educational opportunities. If an assistant initially believes one learner needs only easy questions, it may serve increasingly easy material and treat good scores on those questions as proof that the learner is progressing. Independent changed-condition problems can reveal whether understanding transfers. Otherwise the system may mistake improved performance on self-selected, familiar tasks for broad mastery.
Mitigation can include stable assessment items appropriate to the learner, controlled exploration under suitable safeguards, diverse source review and independent educator judgement. The right approach depends on the stakes and population. Do not introduce experimentation on consequential real-world actions merely because a feedback model would benefit statistically. Human consent and ethical limits remain part of the design.
Return to contents or continue with Make post-deployment monitoring actionable.
13. Make post-deployment monitoring actionable
NIST’s AI Risk Management Framework Manage function describes post-deployment monitoring, input from users, appeals, override, incident response, recovery and change management as relevant risk-management activities. NIST’s March 2026 discussion of deployed AI monitoring also emphasises that practical monitoring methodologies remain challenging and context-dependent. These sources support careful system design; they do not establish a universal formula for rating every AI agent.
Create signals for outcome completeness, time-to-label, source drift, appeal frequency, correction rate, actual external-state mismatches and changes in task mix. Each should have a defined numerator, denominator and accountable owner. A rise in negative labels might reflect deteriorating quality, a newly reachable complaint channel or better measurement of previously invisible failures. Investigate mechanisms before attributing cause.
Do not chase every chart fluctuation with a model update. A small sample may be noisy, a source may have changed, or a new population may now use the service. Compare stable versioned slices where possible and preserve the history of selection rules. For high-severity confirmed incidents, act on direct evidence even if a sample size is too small for a useful prevalence estimate. Monitoring is a decision-support process, not a decorative collection of metrics.
Build recourse into the feedback system. If a parent reports that an education rule was misrepresented, a responsible reviewer should check the actual official source and reply through an appropriate channel. The correction should be linked to the original task and any affected downstream content. If the error reveals a systemic failure, repair the source pipeline and retest related cases. Listening without corrective action is not an operational feedback loop.
Return to contents or continue with Protect privacy and prevent feedback manipulation.
14. Protect privacy and prevent feedback manipulation
Feedback can contain personal details, quoted conversations, test scores or information about children. Collect only what is necessary for the stated quality purpose and manage access, retention and deletion under relevant rules. A monitoring programme should not create an uncontrolled shadow archive of complete learner conversations. Task IDs, outcome categories and limited evidence pointers often suffice for statistical reporting while detailed material remains in a protected review environment.
A system can also be manipulated through feedback. An attacker may send repeated ratings, fabricate success receipts or place instructions into an external result that the model later treats as training guidance. Do not allow untrusted feedback text to override operating instructions or tool permissions. Verify the claimed source of consequential feedback and distinguish qualitative comments from authenticated external outcomes.
Even legitimate human ratings may reflect preferences unrelated to task correctness. A user might reward an answer that agrees with them, or penalise a correct but unwelcome explanation. Evaluate factual support and real-world outcomes separately from satisfaction. Both can matter to usability, but one is not a substitute for the other. A well-designed learning assistant should not become more misleading simply because misleading reassurance earns positive ratings.
Protect the ability to contest a label. A reviewer could misunderstand a student’s reasoning; a tool could report an old file version; an outcome assessment could be applied to the wrong person. Store the provenance and revision of the judgement, with a route to correction where appropriate. Updating a record should not erase the history of why an earlier decision was made.
Return to contents or continue with A transfer exercise: delayed school-notice confirmations.
15. A transfer exercise: delayed school-notice confirmations
Transfer the method to a fictional school-information publisher that sends sixty authorised notice updates through an asynchronous platform. At the initial check, thirty-six have confirmed saved-and-published receipts, fourteen have legitimate pending jobs and ten have no trustworthy acknowledgement. Check the count: 36 + 14 + 10 = 60. The confirmed-outcome fraction is 36 ÷ 60 = 60%, but it would be false to say the remaining forty percent definitely failed.
At the next check, eight of the fourteen pending jobs are verified published, four are verified failed and two remain pending. The ten originally unknown outcomes are still unresolved. The revised mutually exclusive categories are forty-four published, four verified failed, two still pending and ten unknown, totalling sixty. The new confirmed-publication coverage is 44 ÷ 60 ≈ 73.33%. That number describes observed state across these fictional attempts, not whether the article content was accurate or useful.
Now change the system: the platform returns an ‘accepted’ receipt for a new attempt, but a previously unknown request may already have published the same notice. Should the assistant simply send another update? No. It must query the canonical post or operation ID, compare the intended content and status, and decide whether a retry is actually missing. The idempotency chapter explains this recovery mechanism in detail.
Change the user decision too. Suppose the notice was cancelled before the delayed acknowledgement arrived. A later successful publish receipt is not necessarily a desirable user outcome; it may reveal a violation of the current cancellation state. The evaluator must compare the actual action time with the approval and cancellation history. Feedback is useful only when the label reflects what the task was authorised to accomplish, not merely whether a tool returned a green result.
Return to contents or continue with Teach students to interpret missing data responsibly.
16. Teach students to interpret missing data responsibly
Students can practise with a paper table containing 120 invented cases. Have them write three denominators: all proposed tasks, matured labelled outcomes and confirmed positives. Then ask why 49 ÷ 70, 49 ÷ 120 and 67 ÷ 100 answer different questions. The arithmetic is simple, but the meanings are not. This is a powerful critical-thinking exercise for media claims, education studies and everyday dashboards.
Ask learners to imagine the twenty missing labels were systematically from users who had trouble accessing the follow-up form. Would the observed positive percentage represent the whole population? Not necessarily. The missingness mechanism could be related to user experience and outcome, and better accessibility might change which outcomes are observed. An honest analyst would seek additional evidence rather than assign invented labels.
Move the example into Science. A researcher records plant growth after a fertiliser treatment, but some pots have no final measurement because labels detached. Missing pots should not automatically be entered as zero growth. Investigate why measurements are missing, which pots were affected and whether the loss might relate to treatment or conditions. The reasoning method transfers even though the domain has changed completely.
Move it again into English reading comprehension. An AI recommends a short passage and measures only whether a student clicks Next. That is not an independent measure of comprehension. A short changed-context question with explained feedback may be more educationally meaningful, though still not a complete measure of long-term learning. Define the real outcome before designing the metric.
Return to contents or continue with Limits, residual uncertainty and the next safe decision.
17. Limits, residual uncertainty and the next safe decision
No feedback architecture can guarantee that every meaningful outcome will be observed. Some effects take too long, some depend on people outside the service, and some cannot be ethically measured through continuous monitoring. A disciplined system reports these limits and chooses reasonable proxies without relabelling proxies as truth. The need to measure more is not a licence to collect unnecessary personal data.
A model can also change the questions people ask, the materials they encounter and the decision pathways they follow. Evaluation in such an environment needs attention to causal interference and selection, not merely more ratings. Controlled experiments or expert studies may be appropriate for some tasks, while observational monitoring with clear limitations is better for others. A responsible organisation chooses methods suited to risk rather than claiming one technique is sufficient everywhere.
Beware of overconfident automated remediation. If a dashboard detects a temporary fall in observed satisfaction, an agent might automatically change recommendations in a way that further alters the future feedback. Require versioned change control, human authorisation for consequential updates and a test plan that checks whether the repair addresses the real mechanism. Better observability is valuable even when the correct response is to refrain from immediate changes.
The deepest lesson is that being able to generate an answer is not the same as learning whether the answer helped. A trustworthy assistant distinguishes intention, authorised action, independently observed result, delayed or missing evidence and future adjustment. Learners can apply the distinction to study plans; engineers can apply it to deployed systems; and families can use it to ask better questions about claims of AI success. No claim becomes true merely because the agent repeated it confidently.
Return to contents.
Frequently asked questions
Can a thumbs-up rating prove an AI answer was correct?
No. Satisfaction, factual support and actual user outcomes are different measures. Independent evidence may be needed.
Should pending feedback count as failure?
Not automatically. A task still inside its declared observation window is pending; a task with no usable evidence is missing. Both should be reported separately from observed negative outcomes.
Why can the observed positive rate change from 70% to 67%?
The denominator grew from seventy labelled cases to one hundred after delayed results matured. The change alone does not prove model quality deteriorated.
Is 67 out of 120 the final system success rate?
No. It is the confirmed positive count relative to all proposals in the fictional example, while twenty outcomes remain unknown. The final binary positive fraction is not identified.
How do feedback loops cause misleading results?
An agent can change which resources users see or which tasks they attempt, causing later click or success data to depend on earlier recommendations.
Can the agent use its own generated labels as ground truth?
Self-assessments may help triage but should not be treated as independent validation of factual or real-world results without appropriate checks.
Primary sources and research
Sources reviewed 23 October 2026. Riverbend Learning Desk and every task count are fictional teaching examples. The NIST March 2026 research is background on monitoring challenges rather than evidence that this invented system was studied.
- Google Machine Learning Crash Course — production questions: Feedback loops, changing upstream signals and model effects on subsequent data.
- Google Machine Learning Crash Course — monitoring pipelines: Serving-time data, label leakage, bias in observed outcomes and ongoing real-world measurement.
- Google Machine Learning Glossary: Definitions of feedback loops and agent feedback stages.
- NIST AI RMF Playbook — Manage: Post-deployment feedback, user recourse, monitoring, incidents and corrections.
- NIST — Challenges to monitoring deployed AI systems, March 2026: Contextual challenges in observing real-world system behaviour after deployment.
- W3C PROV-O: Modelled relationships among records, activities and accountable participants for evidence lineage.
Learning pathways
Continue with Agent Experiments, Review Sampling, Evidence Ledgers, Agent Test Oracles, the How X Works hub and Gold Standard of Education.