An AI agent changes whenever its model, instructions, tools, source material, retrieval index, permissions or business rules change. Agent change control is the discipline of describing each proposed change, testing what it actually affects, gaining the right authorisation, releasing it to a bounded audience, watching real outcomes and reversing or containing failures. The aim is not to freeze improvement. It is to make improvement observable and recoverable before a seemingly small prompt edit quietly changes consequential decisions.
A safe rollout requires more than a successful demonstration in a chat window. Start with a versioned bundle describing the exact model and tool interfaces, prompt templates, policy checks and source revisions used for each task. Compare the candidate against a stable baseline on representative cases, including difficult and permission-sensitive cases. Then release gradually with explicit stop conditions and rollback instructions. This chapter calculates the results of a fictional 120-case agent evaluation, shows why an improved overall pass rate can hide a worse result for sensitive records, and finishes with a changed-condition transfer exercise.
The examples concern current tool-using AI engineering, not an assertion that artificial superintelligence already exists. This specialist lesson follows How Super Intelligence Works, deployment, monitoring and versioning where available, and the verified agent incident-response guide. All organisations, task counts and release results in the worked examples are invented teaching data.
A route through this chapter
- If you are learning software delivery, begin with a release bundle and follow the baseline-versus-candidate calculation.
- If you operate an AI assistant, concentrate on permissions, prompt injection resilience, tool-schema compatibility, canary cohorts and rollback.
- If you review policy changes, examine who owns the rule, how an expired rule is identified and what cannot be undone after an agent acts.
- For independent practice, calculate the subgroup results before consulting the answer and use the final transfer case to design a safer rollout.
Contents
- 1. What counts as a change
- 2. Versioning the full system
- 3. Worked 120-case release packet
- 4. Canary, shadow and staged rollout
- 5. Diagnosis, containment and rollback
- 6. Transfer and practice
- 7. Questions and next reading
1. A change is any altered assumption the task relies upon
Picture a parent asking an educational assistant whether a particular study guide can be sent to a class group. The model might summarise the guide correctly, but the permission depends on a school’s consent rule and which recipients are allowed to receive it. Changing the prompt wording is plainly an application change; so is changing the model’s default behaviour. Less visibly, switching the document index, modifying a connected tool’s fields or replacing a cached consent policy also changes the system’s possible actions. If nobody can say which version produced a result, accountability becomes guesswork.
The engineering unit worth controlling is therefore a complete task path, not a prompt string. A task path comprises input, source selection, policy check, computation, proposed action, human review when required, tool execution and durable receipt. A change may touch one component while affecting another. For example, a source search improvement could increase the chance that the agent finds the latest rule but also expose malicious instruction text inside retrieved documents. An apparently useful change should be assessed by the outcomes it creates and the boundaries it crosses.
Separate improving an answer from authorising an action
A model can be more fluent and yet less reliable at deciding whether to call a tool. Fluency helps readers, but it is not an authority signal. A rule that forbids publication without a current approval must be enforced at the appropriate action boundary, ideally by independently checkable controls. The model’s explanation can assist a human reviewer, but changing the explanation does not silently update permissions. Keeping that separation makes it possible to change language quality without accidentally changing who is authorised to receive student information.
Imagine a new prompt says, “When an approval seems obvious, avoid unnecessary interruptions.” The product team may intend a friendlier interface. However, “seems obvious” is not a verified approval event. If it causes the agent to skip a required review, the new text is a security or governance regression even if users complete tasks faster. A release test should include a case in which the requested action appears harmless but formal consent is absent. Success is declining or holding the action, not composing the most persuasive reason to proceed.
Make each release hypothesis falsifiable
A release description such as “better intelligence” is too vague to test. A useful hypothesis names the target users, task family and measurable outcome: “The candidate reduces irrelevant citations in parent-facing lesson summaries without increasing unsupported facts, expired-permission actions or total time to verified completion.” That statement can generate sample cases, measurements and stop conditions. If the candidate performs well on one dimension while violating another, the release should fail until the conflict is resolved or consciously re-scoped by authorised owners.
Avoid building a test set exclusively from examples that motivated the change. If engineers repeatedly edit the prompt until it passes twenty familiar prompts, the team may have fitted a narrow rehearsal rather than improved general behaviour. Use held-out cases, boundary conditions, negative examples, time-shifted sources and adversarial but safe inputs. Preserve an evaluation record that distinguishes the exploratory cases used during development from the acceptance cases used to approve release. The distinction matters when learning whether a change transfers beyond its original demonstrations.
2. The version bundle: what must travel together
For a deterministic script, the executable version may be enough to reproduce behaviour given fixed data. An AI agent often needs more. Record the base model identifier, inference settings, system and task instructions, tool definitions, retrieval configuration, source snapshot or watermark, policy engine version, relevant feature flags and the application code version. Some providers may change behaviour under an evolving service alias, so the effective release identity should include what the provider makes observable and the test date. The bundle is a map of dependencies, not a guarantee that stochastic outputs will be byte-for-byte identical.
Suppose the agent uses a new model while its tool adapter still expects the old JSON field “recipient_email” and the updated tool now requires “recipient_id”. A model benchmark may improve, yet publication fails because the action schema changed. That is an interface compatibility error, not evidence that the model forgot how to write. Release testing should validate accepted tool schemas with representative requests and error responses. The preferred outcome on an unknown field is an explicit and safe validation failure, not an improvised guess about which person the message should reach.
The change manifest
| Versioned element | Owner and change type | Release check |
|---|---|---|
| Model and inference settings | Model/inference team | Task-quality evaluation and latency bounds |
| Prompt and task instructions | Product/editorial owner | Instruction precedence, exceptions and negative cases |
| Retrieval index and source watermark | Data owner | Current revisions, superseded records and attribution |
| Tool schemas and credentials | Integration/security owner | Schema test, scopes, least privilege and expiry |
| Business or consent policy | Responsible policy owner | Formal approval, current effective date and denial cases |
| Agent workflow application | Engineering owner | Idempotency, state transitions, rollback and receipts |
A manifest should also record what did not change. If the policy code is unchanged but the source index was rebuilt, the reviewer can concentrate on source freshness and the model’s handling of untrusted content. If both policy and model changed together, a disappointing result is harder to diagnose: was it the text generator, the retrieved document, a stale authorisation, or the interaction among all three? Small reversible releases are easier to attribute than simultaneous sweeping changes. Sometimes a coordinated migration is necessary, but its dependencies should be explicit.
Configuration drift is a real release
Configuration drift occurs when running instances no longer use the intended configuration. Perhaps one worker has a newly updated system instruction while three still use the previous version. Perhaps a manual emergency change affected only one region. Requests may alternate between apparently inconsistent answers without a model-training issue. The investigation needs the effective per-request configuration digest and tool version, not just the version that the deployment dashboard says should be active. Link observed user outcomes to the bundle that actually served them.
Google’s Site Reliability Engineering discussion of configuration specifics provides a useful operational lens: configuration has lifecycle, distribution and validation concerns analogous to code. For AI workflows, include the text and tools that shape behaviour in that discipline. Secrets require separate secure handling; do not embed raw credentials in a public release manifest simply to make the bundle look complete. Record a controlled reference and verify access with a safely designed test instead.
A prompt that changes an agent’s default answer length may be low-risk in isolation, but the same prompt could affect whether a recipient sees an important caution or whether the model calls a tool. Classify changes based on plausible effects, not based on whether the modified file looks small. A one-line safety check may have wider consequences than a thousand-line documentation edit. Risk determines review depth and staged exposure, while technical size helps estimate engineering complexity.
Prompt regressions and policy regressions are different
A prompt regression is a decline in instruction-following or task performance after prompt changes. A policy regression concerns whether authorised and prohibited actions remain correctly classified under the organisation’s actual rules. Some prompt regressions produce policy violations, but a policy change may also be intentional and officially approved. Treat them separately: the product team tests clarity, completeness and tool-choice behaviour; the policy owner certifies the underlying rule and effective date. The incident record should identify which boundary failed rather than describing every problem as “AI hallucination.”
Example: a staff member updates a policy document to say that an old mailing list may no longer receive individual learning reports. An index refresh correctly brings the new document into retrieval. The model continues quoting an older cached summary because that passage ranks highly. The release technically used fresh data but failed to enforce authoritative supersession. A robust change contract includes tests that the newer policy revokes previously valid actions. More documents retrieved does not necessarily mean more accurate permission decisions.
3. The fictional Maple Learning Desk evaluation
Maple Learning Desk is an invented service that explains course materials and prepares authorised communications. A change proposes a new retrieval strategy and friendlier instruction template. The team creates a 120-case acceptance packet: forty-eight ordinary lesson summaries, forty-eight multi-step source comparisons and twenty-four sensitivity-dependent cases involving restricted recipients or expired permissions. Each case is scored as fully correct only if the answer’s factual content, citations, action decision and recorded status all satisfy the relevant acceptance contract. A merely plausible paragraph does not count.
The stable version completes 111 of the 120 cases correctly. Of the ninety-six ordinary and multi-step tasks, it passes eighty-nine. Of the twenty-four restricted cases, it passes twenty-two. The proposed version completes 114 overall, with ninety-five successes in the ninety-six ordinary and multi-step tasks and nineteen in the restricted group. These numbers are an intentionally constructed teaching packet. They are not a published assessment of any commercial model or a forecast of what a new model would achieve.
Work the denominators
| Packet group | Existing version | Candidate | What changes |
|---|---|---|---|
| Ordinary and multi-step, 96 | 89/96 = 92.71% | 95/96 = 98.96% | +6 correct cases |
| Restricted, 24 | 22/24 = 91.67% | 19/24 = 79.17% | −3 correct cases |
| All 120 | 111/120 = 92.50% | 114/120 = 95.00% | +3 correct cases |
The headline accuracy rises from 92.50 percent to 95.00 percent, an increase of 2.5 percentage points. Do not call that a 2.5 percent relative improvement: relative improvement would be the percentage-point difference divided by the original 92.50 percent, about 2.70 percent. More importantly, the candidate loses three successful restricted cases while gaining six in the less sensitive population. If permission mistakes have a high consequence, the aggregate gain cannot justify release. The evaluation is making a governance decision visible, not computing a single magic score.
Inspect the three new restricted failures. One case falsely reuses a consent that expired yesterday. Another attempts to send a source excerpt to a recipient outside the approved list. The third supplies a superficially convincing explanation but records “sent” without a provider acknowledgement. These are distinct mechanisms requiring different repairs: consent-version checks, recipient-bound approval verification and external receipt reconciliation. Merely asking the model to “be careful” addresses none of those at the authoritative boundary.
Why weighted averages can conceal damage
A change can improve the largest, easiest population and still become unsafe in an underrepresented subgroup. Overall scores are weighted by the counts in the test packet, so a small high-consequence category can be submerged. The remedy is not to manipulate the weights until the candidate looks good. Define protected gates in advance: all prohibited-action tests must pass, sensitive-action cases must meet an appropriate threshold and any observed unauthorised live side effect blocks the release. The exact thresholds belong to responsible risk owners rather than a number invented for this article.
If an evaluation report contains “98 percent passed” but omits which two percent failed, it is incomplete for release approval. Two harmless style failures are different from two unauthorised sends. Categorise failures by mechanism, impact and reversibility; preserve original examples and acceptance evidence. Separate classification mistakes from infrastructure timeouts. An infrastructure error may merit a reliability fix, while a permission breach may require revoking a capability. Neither is repaired by inflating the success denominator.
Measure complete outcomes, not attractive intermediate states
Suppose ninety cases receive polished answers and eighty-five complete the required action-and-readback path. Which completion rate applies? For a contract that requires verified external completion, the denominator is all ninety intended cases and the numerator is eighty-five; the complete rate is 94.44 percent. Counting only the eighty-five that reached the tool would hide failed or held work. State when a task is legitimately ineligible, when it is deferred and when a reviewer has explicitly denied it. A safe refusal is sometimes a successful policy outcome even though it is not a completed send.
Evaluation cases should exercise both ordinary success and safe failure. For a prompt-injection example, use a harmless document containing a line asking the assistant to ignore task rules; the correct response is to treat that as source text rather than authority. For a missing source, success may be identifying the gap and asking for a verifiable source instead of fabricating a citation. For an expired permission, success may be a human handoff. The acceptance contract should specify each expected outcome before testing to prevent retrospective score inflation.
What can this sample prove?
A 120-case packet can demonstrate specific failures on those cases. It does not prove the true failure rate for all users or future task distributions. Repeated improvements on the same packet create overfitting risk, and a small sensitivity subgroup has substantial uncertainty. Therefore the right conclusion is “the candidate fails named restricted-case acceptance criteria,” not “we know exactly how often it will fail in production.” Strong engineering combines targeted unit and integration tests, held-out evaluations, observed task results and sustained monitoring after controlled release.
When statistical comparisons are useful, specify sampling design and uncertainty rather than reciting a point estimate. If high-risk outcomes are too rare to observe meaningfully in ordinary canary traffic, direct negative tests and hard action constraints become especially important. No practical canary sample can establish an absolute zero probability of harm. It can establish that a predeclared set of conditions was satisfied and that escalation is available when a novel condition occurs.
4. Shadow mode, canaries and measured expansion
Shadow mode routes a copy of genuine-looking inputs to a candidate system while preventing it from making external effects. It is useful for comparing answers, citations and proposed actions against a baseline. Its limitation is important: a shadow response is not proof that the candidate would perform correctly after taking an action, and copied real input may contain personal data. Use a lawful, approved, protected evaluation route, masking or synthesising data where appropriate. Never quietly duplicate sensitive live communications into an experimental model outside the authorised data boundary.
A canary release exposes a bounded portion of appropriate real traffic to the candidate, with an explicit release cohort, monitoring and rollback owner. “Five percent of requests” is an allocation rule, not an impact ceiling. A small proportion of high-value or sensitive tasks may still have outsized consequences, and one tool call might alter many records. Canary design should therefore consider recipient class, capability scopes and irreversible action types, not merely total request volume. High-risk sends can remain behind a stable service or manual review during the experiment.
How the canary receives and reports work
Imagine Maple repairs the three sensitive regressions and tests a second candidate in an isolated environment. The owners approve a controlled canary of twenty low-risk summaries without external sends, and reserve twenty comparable tasks for the stable version. In the invented exercise, the candidate passes seventeen of twenty complete acceptance tests while the baseline passes eighteen of twenty. The rates are 85 and 90 percent respectively. This small observation is insufficient to quantify broad comparative effectiveness, but the candidate has failed the team’s predeclared rule that the new version must not regress on the particular summarisation gate. Pause; do not invent a statistical exception because the prose looked more engaging.
The team reads the three candidate failures and discovers that the new instructions truncate citations when the source exceeds a defined length. Fix the truncation interface and add a length-boundary test; then repeat a genuinely new acceptance packet rather than merely scoring the same examples until they pass. If the release is permitted to continue later, preserve routing cohort identifiers so incident reviewers can identify which user saw which effective version. Hidden version switches make both debugging and consent auditing harder.
Stages protect real tasks from uncontrolled experiments
A sensible release sequence is not one universal percentage ladder. It starts with offline tests, then protected simulation or shadowing, then narrowly scoped real traffic with hard action limits, and only then progressively broader use. Each stage has a named decision maker, entry criteria, monitoring window, pause trigger and safe return path. The owner may need longer observation for tasks with delayed outcomes; a five-minute green chart cannot prove the quality of a document that a reviewer will inspect next week. Choose the window based on outcome latency rather than convenience.
Google SRE’s guide to canarying releases explains why a canary needs a useful measurement signal and why differences between canary and control conditions matter. AI assistants add special signals: unsupported citations, incorrect permission decisions, external tool duplication, mislabelled task state, human escalation burden and source-version mismatch. A canary with only CPU utilisation and HTTP success can look excellent while delivering unusable or unapproved work.
Avoid misreading canary metrics
A lower token count may look like reduced cost, but if the candidate omits a critical source or stops before recording a provider receipt, the saving is fictitious. A faster p50 response time might coexist with unacceptable tail delays on difficult cases. A high thumbs-up rate could be skewed toward easy readers while restricted cases are safely blocked for a reason users do not like. Identify metric definitions, outcome units and selection effects before drawing conclusions. Use complete-task cost and verified outcomes where possible.
Canary cohorts should be comparable. If the stable group handles challenging documents and the candidate group handles short summaries, the apparent difference is dominated by case mix. Stratify by task category, document length, permission requirements and currentness constraints. If randomisation is appropriate and permitted, specify it. If not, call the evidence observational and avoid causal certainty. Workable engineering does not require pretending all data were collected in a perfect experiment; it requires being explicit about what changed and what could explain the result.
5. Stop conditions, rollback and repairs
A rollback returns the effective application configuration to an approved state. It can reverse prompt text, model routing or software version, but it cannot erase an email already delivered, a confidential record already disclosed or an incorrect instruction already acted upon. That asymmetry makes side-effect containment essential. Prepare an emergency disable switch for the particular external capability, an owner who can revoke scopes and a way to identify affected intentions and recipients. A rollback procedure without external action reconciliation is only partial recovery.
When to stop an experiment
Stop conditions should be recorded before rollout: any unauthorised tool action, missing required approval, materially unsupported sensitive claim, incorrect currentness for a high-consequence source or failure to record a required receipt can trigger immediate halt. Other conditions, such as slower median responses, might call for review rather than automatic shutdown. The operator should not have to improvise the risk policy under time pressure. Logging the precise trigger supports a blameless review later.
A warning signal also needs a clear source. A model-generated explanation that it is “probably fine” is not an independent check on whether the tool changed a real record. Look at the authoritative system, the approval ledger and the action receipt. If a release has already made a harmful or irreversible change, containment may require correcting records or notifying the responsible parties under applicable rules. Do not attempt to hide the effect by rewriting internal logs. Evidence preservation is part of recovery.
Diagnose which component caused the regression
Suppose the new model and old prompt work on a case, but the old model and new prompt fail. This suggests a prompt change could contribute, though further experiments are needed because model behaviour is not perfectly deterministic. In a controlled offline test, vary one factor at a time: prompt, source revision, tool schema, policy input or model. Preserve effective versions, seeds when supported, input cases and tool outputs. A factor that repeatedly predicts the failure gives engineers a concrete repair target; an anecdote gives only a suspicion.
Do not treat the four possible combinations of two model versions and two prompts as automatically sufficient for causality. Stochastic sampling, source freshness, dynamic tool results and upstream service behaviour can change across tests. Freeze what can legitimately be frozen, repeat meaningful cases and record uncontrolled variation. If the problem manifests only in a changed environment, include that environment in the release contract. An honest “not yet isolated” is preferable to assigning blame to whichever team touched the last file.
A release ledger makes investigation possible
| Record | Purpose | What a reviewer should be able to check |
|---|---|---|
| Change request and approval | Names intention and owner | What changed, why and by whose authority |
| Bundle digest | Identifies effective combination | Prompt, model, tools, policy and source revision |
| Acceptance packet | Records pass/fail mechanisms | Task counts, cases and subgroup gates |
| Release assignment | Shows who received candidate | Cohort, time, feature scope and fallback |
| Action receipts | Establishes real external effects | Original task keys and provider confirmations |
| Pause, rollback and follow-up | Preserves repair trace | What was stopped, verified and retested |
If a version bundle is rolled back, retest an end-to-end task under the restored state. An assumed-good old version might not be compatible with a changed schema or data migration. Some state transformations are not reversible by restoring old code alone. Plan forward-compatible migrations or an explicitly rehearsed reversal procedure; verify the actual data format. Declaring “back to yesterday’s model” does not solve today’s missing column or withdrawn authorisation.
A released prompt might refer to a source that later disappears or moves. Even if its language remains unchanged, its outcomes can drift because the world and its dependencies change. Monitoring needs to detect freshness and access changes, not only software deploys. An official school admission page updated for a new cohort is a content change with operational consequences; users should receive date-accurate guidance even when the agent’s model file has not changed. Date-sensitive references are data-version events.
6. Transfer exercise: the candidate passes overall but fails a consent boundary
The fictional Cedar Art Archive tests a release on sixty cases: thirty ordinary descriptions, twenty source comparisons and ten requests that require explicit publication permission. Its old version passes twenty-seven ordinary, eighteen comparison and ten permission cases. Its new version passes twenty-nine ordinary, twenty comparisons and eight permission cases. Calculate the overall rates, identify the significant governance concern and decide whether the team should ship the candidate merely because its total score improves.
The old version passes 27 + 18 + 10 = 55 of 60, or 91.67 percent. The candidate passes 29 + 20 + 8 = 57 of 60, or 95 percent. Overall performance gains two successful cases, or 3.33 percentage points. However, the permission group drops from ten of ten to eight of ten, and the new failures concern publication without current consent. The correct decision is to block that capability’s release while repairing the rule enforcement, regardless of the superficially better aggregate. If the original authorisation checks are implemented by a separate protected gate, verify that the gate actually prevented any publication, not just that the model described it.
Changed conditions: yesterday’s canary becomes tomorrow’s rule
Now imagine all sixty evaluation cases pass, the controlled canary finishes without incident and a school changes its student-data sharing rule the next morning. Does yesterday’s green rollout authorise future sharing? No. A release approval applies to the tested effective conditions, not to a world frozen in time. The agent needs a policy version and effective-date check at the action boundary. It may remain operational for ordinary summaries while sensitive sharing pauses for a new compliance check. The right answer adapts scope to the changed condition, not the other way round.
Independent practice with explained answers
Question one: an agent becomes faster but produces fewer citations. Is that automatically a regression? It depends on the specified task. If required claims remain sufficiently supported and the new response meets its evidence contract, fewer citations may be appropriate. If shortened output removes evidence needed for a consequential decision, speed cannot excuse the omission. Measure correct, complete user outcomes rather than maximising citations or minimising tokens in isolation.
Question two: a canary sees no harmful actions in five low-risk tasks. Has it proved safe high-risk behaviour? No. There was no meaningful exposure to the high-risk capability, so the observation says little about it. Use specific prohibited-action tests, protected gates and approval routes. Question three: a failed canary is rolled back but three records have already changed. Is recovery complete? No. Reconcile the records, validate the external consequences and document necessary correction; reverting configuration cannot rewind the world.
Question four: a model version is fixed, but an index refresh changes its answers. Should this count as a release? Yes, if it changes effective behaviour and user outcomes. Record index revision or watermark and verify compatibility. Question five: a prompt is unchanged but an administrator grants the agent broader tool scope. Does it need testing? Certainly. The system’s action authority changed, potentially more significantly than the model text.
Common misunderstandings and practical repair
“The model told us it followed the rule” is not the same as a signed policy decision or an approved tool action. “The old version was safer” does not prove the current system can roll back without data loss. “No customer complained” does not mean a silent privacy disclosure did not occur. “Our benchmark rose” does not settle what happened to sensitive cohorts. Each mistaken shortcut hides a distinct check. Repair the decision process by connecting each assertion to a named authoritative record and rerunning the complete task rather than rewarding fluent explanations.
The most useful practice is to build a small release review sheet. Write the intended improvement, the current failure being repaired, the exact change manifest, the task families exposed, the protected gates, the acceptance results by subgroup, the human authoriser, the canary scope, the stop rule and the confirmed rollback path. A student could apply the same reasoning to a spreadsheet model or school project: change one assumption, test representative examples, preserve the original and verify the new outcome instead of calling every newer version “better.”
7. Frequently asked questions
Is a new prompt always a new product version? It should be a versioned behaviour change when it affects tasks people rely on. An editorial punctuation fix may need light review, whereas a changed instruction governing a tool call needs much stronger acceptance testing. The release identity should permit a reader or incident reviewer to discover which instructions, model and policy applied at the time of the result.
Can an AI agent approve its own policy changes? A model can draft proposals and assemble impact evidence, but whether it may authorise a change is a separate institutional decision. High-impact capabilities ordinarily need an accountable human or formally authorised governance process, with an independent record. This chapter does not assume that model confidence supplies the legal or organisational authority to revise a rule.
Why not route every task through the most capable model? A model choice is only one variable. Tasks still depend on correct source revisions, tool schemas, human permissions, stable state and verification. A more capable generator cannot repair an inaccessible receipt or grant consent. An appropriate release evaluates the complete workflow, including cost and time to a correctly finished task, not solely text quality.
What does “canary” mean for privacy-sensitive work? It means bounded real exposure only where lawful and explicitly approved; it does not mean sacrificing a few sensitive people to see what happens. Protect high-impact actions with hard policy controls and testing in safe environments first. Where a controlled live trial is inappropriate, remain in offline or synthetic conditions and choose a different evidence route.
Can rollback undo a model-generated instruction that a person already followed? No. The product may correct the published advice and contact affected readers where appropriate, but it cannot erase their past action. Distinguish reversible software configuration from real-world consequences. Treat the incident according to impact and documented responsibilities, not merely deployment status.
Sources, limits and next lessons
Read Google SRE: Canarying Releases for controlled release and signal selection and Google SRE: Configuration Specifics for why configuration is operationally important. NIST AI RMF’s Govern, Map, Measure and Manage functions provide a risk-management framework rather than a promise that a particular prompt change is safe. Google Cloud’s reliability guidance contextualises testing and recovery. All Maple and Cedar tasks and outcomes above are original teaching scenarios, not evaluations of any named AI provider.
For the next layer, read controlled fault testing, data consistency, incident response and the new disaster-recovery laboratory. The How X Works hub and Gold Standard of Education develop the wider habit behind every successful change: explain what you think will improve, test what actually improves, and retain the evidence needed to correct yourself.