An AI service-level objective, or SLO, defines a target for the quality of a user-visible service over a specified population and time window. An error budget expresses how much failure the team has decided to tolerate against that objective while balancing reliability improvements with change. The model’s internal speed is not itself the SLO.
This chapter follows a fictional Beacon Homework Answer Service with ten thousand eligible requests, a 99.5% supported-answer objective and a strict two-second completion target. We calculate allowable misses, track the remaining budget, examine a small overshoot and solve an independent changed-context example.
The article is an unnumbered technical companion to How Super Intelligence Works. Super Intelligence is the site’s educational umbrella, not a claim that contemporary AI has achieved artificial superintelligence.
Contents and reliability reading routes
1. Begin with what the user calls a good response · 2. SLI, SLO and SLA are not interchangeable · 3. Choose a user-centred eligibility population · 4. The complete Beacon service packet · 5. Compute the total permissible errors · 6. Early in the window, forty-two are missed · 7. The final two thousand add eleven misses · 8. Compare target, observed result and budget separately · 9. An error budget is a release decision instrument · 10. A fast but wrong answer counts as bad · 11. A slow but correct answer may also count as bad · 12. First token is not the completed result · 13. Correct abstention needs an explicit contract · 14. Counts and eligible events must reconcile · 15. One incident can spend a large portion of the budget · 16. Burn rate depends on the comparison window · 17. Observability supplies evidence for the indicator · 18. A source error is not always a model error · 19. Dependencies can share failure modes · 20. Quality must be sampled or checked appropriately · 21. Error budget exhaustion is not permission to conceal incidents · 22. The independent Sunrise Museum exercise · 23. Transfer with a tighter objective · 24. An SLO should not be a claim about intelligence · 25. A practical reliability review before release · 26. Frequently asked questions · 27. Official sources and connected eduKate curriculum
1. Begin with what the user calls a good response
A homework assistance service is not successful merely because a server replies with text. The pupil needs a response suitable for the assigned task, grounded in the applicable sources and delivered within a useful time. A fluent answer containing an invented solution can be worse than an appropriately acknowledged limitation.
An SLO is useful only if its measured good events represent the intended user experience. The team should define eligibility, quality and timing before looking at the results, rather than changing the denominator to make its dashboard look healthy.
2. SLI, SLO and SLA are not interchangeable
A service-level indicator, or SLI, is a defined measurement such as the fraction of eligible requests that receive an accepted supported answer within two seconds. An SLO is the desired threshold for that indicator over a specified window. A service-level agreement, or SLA, is a separate commitment or contract that may have consequences.
The Google SRE workbook example SLO document illustrates explicit indicators, objectives, error budgets and measurement caveats. A published objective without a trustworthy measurement definition is only an aspiration.
3. Choose a user-centred eligibility population
Beacon’s fictional objective covers standard-information homework requests that the service is authorised and expected to answer automatically. Cases requiring a teacher’s individual decision are routed through a separate review path and excluded by the predeclared scope, not selectively removed after a difficult result occurs.
The operator must document how it determines eligibility. Excluding every slow or wrong result as ‘out of scope’ after observing it would manipulate the rate. Eligibility rules belong to the service contract and should remain stable during the measurement period.
4. The complete Beacon service packet
In a fictional fixed reporting window, Beacon receives exactly ten thousand eligible user requests. A good request must produce a supported, task-appropriate answer, with the full response delivered no later than two seconds after the specified start event. Any eligible request failing that combined criterion counts as a miss.
The SLO target is 99.5% good requests over the full ten-thousand-request window. All counts are invented teaching values; no actual education service, learner or model was monitored or tested.
5. Compute the total permissible errors
A 99.5% target permits 100%−99.5%=0.5% bad eligible outcomes over the stated window. Convert 0.5% to 0.005 and multiply by 10,000 to obtain fifty allowable misses.
The error budget is fifty failed eligible outcomes under this request-count objective. That is not permission to harm learners or publish incorrect material knowingly. It is a quantitative reliability-policy mechanism that lets teams examine how much observed unreliability remains under the accepted target.
6. Early in the window, forty-two are missed
After the first eight thousand eligible requests, the packet records forty-two misses. For the planned ten-thousand-request window, this leaves 50−42=8 misses available before the final objective would be exceeded. This is a provisional calculation assuming the window ends with the stipulated full ten thousand.
The early segment’s measured rate is (8000−42)/8000 = 99.475%, slightly below 99.5%. Budget burn can therefore look concerning before the window ends, even though the final result is not yet known.
7. The final two thousand add eleven misses
Under the fictional packet, the next two thousand eligible requests include eleven additional misses. The full window therefore contains 42+11=53 misses and 10,000−53=9,947 good outcomes.
The achieved good-outcome share is 9,947/10,000=99.47%. That is below the 99.5% target by 0.03 percentage points, and the error budget is exceeded by three failed eligible requests.
8. Compare target, observed result and budget separately
The target is 99.5%. The observed-within-fixture value is 99.47%. The allowed number of misses is fifty; the actual count is fifty-three. Each figure describes the same fictional window from a different view, and the report should show all of them.
A dashboard that rounds 99.47% to a whole number would show 99% or 99.5% under different formats and risk hiding the specific failure. Keep precision appropriate to the decision rather than relying on a celebratory green indicator.
9. An error budget is a release decision instrument
Google’s example error budget policy uses budget consumption to guide reliability work and release decisions. It explicitly frames the policy as protecting customers and balancing stability with new features, rather than punishing staff for an SLO miss.
A real organisation should agree on its own release and escalation response with responsible stakeholders. This invented Beacon case does not declare that every team must freeze all software changes after three additional misses.
10. A fast but wrong answer counts as bad
Suppose an eligible Beacon answer arrives in 0.2 seconds but its key mathematical statement is unsupported. Under the defined combined criterion it is a miss. The answer’s latency does not cure the quality failure.
This distinction matters because a dashboard measuring only HTTP success or first-token response time could call the same interaction successful. The SLI should match the user task, not the easiest server statistic to obtain.
11. A slow but correct answer may also count as bad
Imagine a complete supported explanation arrives in 2.5 seconds when the service promised no more than two seconds for eligible standard requests. It fails the stated timing criterion even though its content is correct.
Whether a slower response is truly harmful depends on the application and stakeholder needs. The team should choose an appropriate target through evidence rather than assume every pupil requires an arbitrary two-second limit.
12. First token is not the completed result
An AI assistant may stream its first token quickly and finish its full answer later. A time-to-first-token measure describes that first visible output, whereas the user’s task may require the complete checked explanation. The measurement must define its endpoint.
The Latency and Throughput chapter explains these distinctions. An SLO using first-token timing should not be advertised as a final-answer completion guarantee.
13. Correct abstention needs an explicit contract
Some legitimate requests may lack a required source. If the authorised service is expected to acknowledge missing information and route to human help, a correct abstention can be a good outcome for that task class. If the objective requires an answer supported by a known current source, an unsupported invented answer is bad.
Define task classes and success rules before the test. Otherwise the team may count every refusal as a model failure or every fluent guess as a success, producing a misleading service measure.
14. Counts and eligible events must reconcile
A useful audit records total eligible requests, good outcomes, missed outcomes, exclusions by predeclared rule and any instrumentation gaps. The equation good+bad=eligible should hold for a complete fixed cohort under the simple binary criterion.
If some requests are genuinely unresolved at the measurement cutoff, the operator needs a documented policy for whether they are still pending or count as deadline misses. Do not silently omit them from the denominator to protect the headline percentage.
15. One incident can spend a large portion of the budget
Imagine an invented outage caused twenty of Beacon’s fifty-three misses. Those twenty would consume 20/50=40% of the originally allowed error budget. This helps explain why investigating a concentrated failure may be more valuable than optimising tiny model-level timing differences.
The packet does not establish the outage’s technical cause. A trace or incident review would need actual source, queue and application observations. The error budget shows magnitude relative to a target, not why the failure occurred.
16. Burn rate depends on the comparison window
A team may compare the rate of bad events over a short period with the allowed rate under its longer SLO objective. Fast budget burn can signal trouble, but the alert should follow the service’s actual traffic and impact characteristics.
Low-traffic systems may produce noisy percentage swings, while high-volume services can exhaust budgets rapidly. Google’s SLO alerting guidance discusses these considerations. An arbitrary universal page threshold is not suitable for every education application.
17. Observability supplies evidence for the indicator
A trace can show whether source retrieval, inference, checking and delivery occurred. Metrics can count good and bad eligible outcomes. Logs can help explain a particular failure. None alone automatically validates the answer’s educational correctness.
The Agent Observability chapter gives a worked critical-path model. Here the focus is the user-facing reliability policy built on appropriately validated measurements.
18. A source error is not always a model error
Beacon might retrieve an outdated school marking rubric and answer it accurately. The model’s computation succeeded, but the final answer is wrong for the current task. The correct SLI should count the user-facing failure while the incident investigation identifies the stale source.
Improvement might require better source versioning rather than more model parameters. An SLO helps define the impact; the technical diagnosis locates the right repair.
19. Dependencies can share failure modes
One retrieval service can feed many agent workers. If its source index becomes unavailable, numerous requests may fail together. Multiplying assumed independent stage probabilities might understate correlated failures.
A reliable service objective therefore needs measured end-to-end outcomes, not only a product of component uptime rates. Component reliability is useful for diagnosis but not a substitute for the person’s actual experience.
20. Quality must be sampled or checked appropriately
It may be straightforward to count whether a response arrived within two seconds, while correctness requires a validated rubric, deterministic checks or authorised human evaluation. The measurement design should reflect that distinction rather than invent a universal automated judge.
If only a sample of answers is quality-checked, the report should explain sampling and uncertainty. A model grading its own answers without validation is not automatically an independent measurement.
21. Error budget exhaustion is not permission to conceal incidents
A team that has already exceeded its budget should not remove inconvenient requests from its logs or relabel wrong answers as accepted. Doing so may improve the metric while worsening the service.
An appropriate response includes accurate status, investigation of meaningful causes, safe corrective releases and stakeholder decisions under the agreed policy. Honest error accounting is more valuable than an artificially perfect dashboard.
22. The independent Sunrise Museum exercise
A second fictional information service handles four thousand eligible requests with a 99% good-response objective. It allows 1% failures, or forty misses. Suppose it initially records thirty-one misses, leaving nine under the full-window budget.
If twelve further misses occur during the remaining part of the same stipulated four-thousand-request window, total misses are forty-three, exceeding budget by three. Good outcomes are 4,000−43=3,957, for 3,957/4,000=98.925%.
23. Transfer with a tighter objective
Return to Beacon’s ten thousand eligible requests, but imagine an organisation changes its agreed objective to 99.9% while keeping the observed fifty-three misses. The new allowable budget is 0.1% of 10,000, or ten misses. The same observations exceed that proposed budget by forty-three.
That does not prove a stricter objective is appropriate. It shows how a target is a policy choice: the same system measurements can be compared against different declared expectations, but the change should not be hidden after seeing results.
24. An SLO should not be a claim about intelligence
A 99.5% useful-response objective concerns a particular application, task class, population and window. It does not certify the model as AGI or establish reliability on scientific research, healthcare or untested problems.
The series separates capability benchmarks from institution-specific accepted outcomes. The service objective tells a school what its particular workflow is trying to deliver, not what AI can do everywhere.
25. A practical reliability review before release
Identify the legitimate task and eligible request set, define good and bad results, choose an appropriate observation window, obtain reliable data and calculate the error budget. Then agree on how the team responds to budget burn, concentrated incidents and proposed releases.
Recheck the source and review process after model or tool changes. A successful deployment should improve the user-facing objective under representative conditions without introducing unsafe shortcuts or unsupported claims.
26. Frequently asked questions
Is an SLO the same as an SLA? No; an SLO is an objective, while an SLA can create contractual obligations. Does a 99.5% SLO mean exactly 99.5% of all possible future queries will succeed? No; it concerns a defined indicator and measurement window.
Can a 200 response be wrong? Yes. Does high uptime prove a pupil learned independently? No. Can a correct abstention be successful? It depends on the predeclared task acceptance criteria.
Applied reliability accounting: eligibility, overlapping errors, burn and release decisions
The numerator must describe good user outcomes
A service that measures HTTP success codes may show excellent uptime while giving unsupported answers. The reliability indicator should reflect the relevant task rather than only the transport layer. For Beacon, an eligible answer is good only when it is grounded and fully delivered within the defined response target.
A fast hallucination is therefore a missed user objective even though the network request technically completed. A correct explanation delivered after the agreed deadline may also be counted as bad under that particular service contract.
Decide eligibility before viewing outcomes
The operator should define which requests are covered before calculating the indicator. If teacher-approval cases are deliberately outside the standard-answer path, that distinction should be encoded using relevant task criteria rather than inferred from whether a difficult request happened to fail.
Moving problematic cases into an exclusion category after observing them would inflate reliability artificially. A transparent report records which events were excluded under the approved rules and why.
The total error budget is not a daily allowance
A 50-miss budget for Beacon’s defined ten-thousand-request window does not necessarily mean the service may have an unrelated fixed number of failures each day. Traffic volume changes, and the team’s monitoring window and burn calculations may follow different conventions.
The service should compare bad-event rates with the permitted fraction and relevant recent traffic, not divide the final allowance into invented daily portions without support from the SLO policy.
Early burn can predict an exhausted budget
After eight thousand requests, Beacon has already recorded forty-two failures, a 0.525% missed-outcome rate. The target would permit 0.5% over that segment if the same proportion applied. The early observation therefore warrants investigation even before the final ten thousand are complete.
A team should not interpret eight remaining misses under the ultimate fifty-miss allowance as proof that recent service behaviour is healthy. It describes remaining nominal headroom, not the trajectory of new errors.
A release policy needs an agreed response
An error budget can help a product team decide when to prioritise reliability work rather than feature releases. The policy must say who is accountable for decisions and which urgent fixes remain permitted during an SLO miss. It should not be created only after an incident to punish a particular employee.
Google’s published example uses a four-week window and escalation arrangements; it is an illustrative policy, not a mandate for every tutoring service. Beacon’s fictional organisation would need its own approved process.
Different SLOs can be evaluated separately
An AI service might have one indicator for availability and another for correctness of supported answers. Those can have different targets and budgets. A combined acceptable-answer indicator may be more directly user-centred but harder to instrument reliably.
The application should be clear whether it counts a request bad for any one reason, or separately counts failures under multiple objectives. Do not add two budgets as though they represented disjoint events when a single request can violate both.
Double-counting the same failure is a common trap
Imagine one Beacon response is both unsupported and late. For the combined good/bad definition, it is one bad request, not two different failed user outcomes. For diagnostic reporting, it may have two failure tags. Those classifications need different counting rules.
A dashboard that sums all failure tags to compute unique misses can exaggerate the number of requests affected. A dashboard that drops overlapping failures without explanation can conceal root causes. Report events and findings separately.
Task difficulty can make one average misleading
Short factual lookups and long mathematics explanations may have different expected latency and correctness profiles. If they share one undifferentiated SLO, a flood of easy questions can hide repeated failures on the tasks that matter most.
Segment results by justified task family and examine where the service actually fails, using appropriately protected data. A task mix difference between months can change aggregate performance without a model revision. Explain the denominator before making a trend claim.
Student learning is not a serving SLO
A system can meet an answer-delivery target yet fail to help a pupil understand the method independently. The learning outcome belongs to a separate educational evaluation with tasks, baselines and changed-context practice. A service reliability SLO measures a defined software experience, not all human benefits.
The two measures can inform one another without being merged into a misleading universal score. The school’s educational goal should remain visible alongside technical service quality.
Sampled answer checking has uncertainty
When semantic correctness is checked only on a subset of requests, the proportion estimated from that sample may have uncertainty and sampling bias. The operator should not simply count every unchecked answer as correct or claim an exact population-wide rate without a justified estimator.
A genuine reliability report records sampling methods, reference validation, any missing data and the limitations of automated graders. The model’s own endorsement of its output is not independent ground truth.
Review queues may create legitimate latency
If an exceptional task requires human approval, the acceptance rule for that task should reflect the permitted workflow rather than pretending it is equivalent to a standard two-second lookup. The service may correctly escalate and wait, which is progress but not a fully answered standard request.
An application should establish those task classes in advance. Otherwise staff may be encouraged to bypass review simply to satisfy a technical latency target.
Timeouts can distort the measurement boundary
A client may stop waiting at two seconds while the server continues computing. If the user received nothing within the defined target, the event is bad under Beacon’s latency criterion, even if the server completes after three seconds.
The system should not count the server’s later successful execution as another independent good user request. Task identity and actual delivery timing matter, especially when retries submit multiple computational attempts for one original question.
A trace can diagnose but not certify semantic correctness
A tracing system may show that the retrieval step used document revision R-14, model synthesis completed in 200 ms and delivery finished in 500 ms. That helps locate a slow stage or an unexpected tool invocation. It does not prove the chosen document supported the answer.
Link the trace to source validation and a meaningful result check. The Agent Observability companion provides a separate critical-path laboratory.
Correlated failures deserve incident-level analysis
A single bad document update can lead many independent requests to return the same wrong answer. The missed-user-outcome count may jump even though model inference remains fast and stable. A task-level SLO captures the impact but not the shared cause.
Investigate source freshness, permissions, retrieval and deployment changes. Adding more agent workers would not necessarily fix a common upstream error. The recovery plan should target the actual dependency.
An alert threshold should be actionable
An alarm that pages staff for a single harmless rounding fluctuation can create fatigue. One that waits until many pupils receive wrong explanations may be too slow. Teams should set alerting criteria using the chosen SLO, traffic characteristics and meaningful response actions.
For a small educational pilot, carefully chosen synthetic checks or a longer observation window may be more informative than high-volume production-style paging. Source data and stakes should guide the decision.
Not every reliability failure has the same consequence
Two missed answers may both count as bad under an indicator, but one might involve a simple typo and another a dangerously misleading instruction. Error-budget accounting does not replace severity assessment and incident response.
The team should classify consequential cases under its responsible governance process while retaining a consistent SLO denominator. Counting events equally for one numerical objective does not mean all harms are ethically equivalent.
A stricter SLO requires more than changing a number
If Beacon’s objective rises from 99.5% to 99.9% over ten thousand eligible requests, the allowed errors fall from fifty to ten. Achieving the stronger target requires demonstrated improvements or changes in the legitimate service design, not merely editing the dashboard threshold.
A new objective may be justified by user needs and risk, but it may also be infeasible without resources. Agree on it using actual measurement and stakeholder expectations rather than a fashionable percentage.
A low-traffic example needs different care
Suppose a separate fictional school service processes only one hundred eligible requests with a 99% objective. One failure exhausts its one-error budget. A second failure causes a miss. Percentages can change sharply with only a few observations.
The team should report counts, sample size and uncertainty. Its monitoring and release decisions may require different practical handling from a platform with millions of requests, even if the mathematical indicator shares the same formula.
A final changed-context budget check
In the Sunrise transfer, forty misses are allowed among four thousand eligible requests. Thirty-one early failures leave nine nominal headroom. Twelve later failures produce forty-three total, exceeding by three. The achieved good share is 3,957/4,000=98.925%, below 99%.
A learner should be able to recalculate these figures without copying Beacon’s ten-thousand-request denominator. That independent transfer demonstrates understanding of an error budget as a ratio tied to a specific service window.
The final reliability decision card
A useful card states the service, eligible user population, good-event definition, measurement window, total eligible events, good and bad counts, target, remaining or exceeded budget, known instrumentation gaps and accountable next action. It separates observed facts from investigation hypotheses.
The team should decide whether to continue releases, apply a safety correction or revise the measurement plan according to its approved policy. An AI can organise the evidence but should not invent the source data or unilaterally change the reliability commitment.
Continue the four-part agent reliability route: Durable Workflows · Event-Driven Agents · Agent Sagas. Together these laboratories trace a saved task across failures, event delivery, multi-service compensation and measured complete-task reliability. See the completed 100-topic series hub and Agent Observability for the corresponding full-system checks.
27. Official sources and connected eduKate curriculum
Use Google SRE’s example SLO document, error budget policy and SLO alerting guidance for foundational reliability mechanisms. They are not measurements of Beacon or Sunrise.
Connect to Agent Observability, Deployment and Monitoring, Evaluations and How Super Intelligence Works. All objectives, error counts and outcomes in the exercises are invented. Sources reviewed 22 October 2026.