Super Intelligence master guide › Energy, work, learning and safety series › Article 0018
AI agent containment limits the resources and actions available to an agent. Independent monitoring records and examines its behavior through a process the agent does not control. Together, they help connect a capable system with a defined operating scope, observable consequences and a way to respond when something goes wrong.
An agent differs from a simple answer interface when it can use tools and continue work across several steps. It may read documents, edit files, query services or initiate changes. The usefulness of those capabilities depends partly on the environment surrounding the model: permissions, credentials, policy enforcement, logs and recovery.
A written instruction to behave carefully is valuable, but it is not the same as an enforced access boundary. Likewise, a detailed account generated by the agent is not necessarily an independent record of what occurred. A dependable design treats instructions, restrictions and observation as complementary controls.
The Super Intelligence series uses SI as a broad theme for machine intelligence. Current agents should be assessed through their demonstrated behavior within particular permissions and tasks. Hypothetical technical superintelligence does not justify unlimited authority, nor does it make today’s containment mechanisms infallible.
Map the path from request to consequence
A useful starting point is to trace one action through the system. A user asks for a result. The model selects a step. A tool request is constructed. An access-control layer decides whether the operation is permitted. The service performs or rejects the action, and the outcome returns to the agent.
Each stage has a different responsibility. The model interprets the task. The access-control layer enforces authority. The tool carries out an operation. The monitor preserves evidence. Confusing these roles can create a gap: a model may describe an action as acceptable without possessing permission to perform it.
A fictional document agent illustrates the chain. It can propose a revision, request access to a file and attempt to write the change. The file service should still enforce the allowed location and operation. The user’s intention to improve one document does not imply permission to alter every accessible file.
Mapping the chain also reveals where controls belong. A restriction that exists only in a conversation can be bypassed by an accidental tool request if the tool itself accepts it. The relevant boundary must be enforced where the consequence becomes possible.
Controls first require an explicit account of intent and authority; the system must identify which actions serve the task and which resources the user is entitled to authorize.
Use least privilege as a task-design principle
Least privilege means giving a process the access necessary for its assigned function. NIST’s least-privilege glossary entry defines the principle in terms of minimum necessary authorizations and resources. For agents, the practical implication is to begin with the task and derive permissions from it.
A fictional research assistant may need to read approved documents and create a new report. It does not automatically need permission to edit the source collection, send messages or access unrelated records. Removing those permissions reduces the consequences of a mistaken step while preserving the intended work.
Permissions should be specific enough to express the real boundary. Read access and write access differ. Creating a new file and deleting an existing one differ. Inspecting a draft and publishing it differ. A broad label such as document access can conceal these distinctions.
The principle also helps evaluation. A narrow permission set makes the operating arrangement easier to describe and test. If the task later requires more authority, the additional operation can receive a focused review.
Least privilege is not a reason to block every useful action. It is a method for making useful authority explicit and ensuring that additional access has a purpose.
Separate identity, authentication and authorization
Identity describes which user or process is involved. Authentication provides evidence supporting that identity. Authorization determines which action that identity may perform on a resource. These questions are connected, but success at one does not settle the others.
An agent acting on behalf of an authenticated account may still lack permission for a particular operation. A service should examine the operation, destination and conditions rather than infer universal authority from a successful login.
NIST’s Zero Trust Architecture explains that assets and accounts do not receive implicit trust solely because of network location or ownership. It distinguishes authentication and authorization and focuses on protecting resources. Applying that reasoning to agents supports checking access where the resource is reached.
This is a design connection, not a claim that every agent implementation is a complete zero-trust architecture. The principle helps frame the question: what evidence supports this action now?
A fictional scheduling agent can be authorized to inspect availability while lacking permission to change another calendar. Authentication establishes its account relationship; authorization establishes the specific scope. Keeping them separate prevents convenience from becoming an accidental expansion of authority.
Containment should follow the resource boundary
A sandbox can isolate an agent’s execution environment, but the important question is what remains reachable from inside it. If broad credentials or unrestricted network tools are available, isolation of the local process may leave substantial external authority.
Containment therefore needs a resource map. Which files can be read or written? Which services can be contacted? Which credentials are available? Which operations can those credentials authorize? The answers determine the practical scope.
A fictional code assistant may operate in a temporary workspace with sample data. That arrangement limits the impact of mistakes within the workspace. If it also receives credentials for a production service, the scope changes significantly even though the workspace still looks isolated.
The design should examine the complete set of routes to consequences. A local restriction does not automatically limit an external API. A tool restriction does not automatically limit data exposed through another tool.
The useful measure is the authority that survives all available paths. Containment is effective when the configured boundaries match the intended task and remain enforceable across those paths, rather than when an environment carries the reassuring label sandbox.
The mechanism requires boundaries inside an operating production service; read access, proposals, approvals and execution should remain distinct even when they share a model or interface.
Keep untrusted content from becoming authority
Agents may encounter documents, web pages and tool outputs containing text that looks like instructions. That material can be useful evidence for the task without becoming a legitimate source of authority over the agent.
NIST’s Adversarial Machine Learning taxonomy includes prompt-injection threats and distinguishes attacker control over queries, resources and other parts of the system. The security issue is relevant when an agent processes external content while also holding tools and permissions.
A fictional source document might contain a statement asking the assistant to send unrelated files elsewhere. The document’s presence in the research task does not authorize that action. The operating system should preserve the difference between content to be analyzed and instructions from an authorized source.
A clear instruction hierarchy helps communicate this distinction. Enforcement outside the model is also important: even if the agent misinterprets the content, the resulting action should remain subject to resource and policy checks.
The aim is to prevent imported material from expanding authority. Reading a document can provide facts, questions or claims. It should not silently change the agent’s permissions, destination constraints or obligations to the user.
Design monitoring around observable events
Monitoring should identify what the system actually attempted and what the environment permitted. A useful event can record the action type, resource, time, policy decision and outcome. The exact detail depends on the task and data sensitivity.
The agent’s own explanation may help interpret an event, but it should not be the sole evidence. An independent service can record that a write was attempted and rejected even if the agent later describes the task differently.
Monitoring also needs to distinguish intention from execution. A proposed action, a submitted request and a completed operation are different events. Confusing them can create a misleading account of progress or risk.
A fictional publishing agent may draft a page, request publication and receive a failure. The monitor should not record the page as published merely because the request was made. Conversely, a successful operation should remain observable even if the agent’s response is interrupted.
This event-based view makes investigation possible. Reviewers can connect requests, decisions and effects without relying entirely on a narrative produced after the fact.
Independence requires a separate trust boundary
A monitor is independent when the agent cannot freely alter its rules, disable its observation or rewrite its evidence. Separate hardware can support this arrangement, but independence is an architectural property rather than a guarantee supplied by a particular chip.
The relevant questions concern control. Who can configure the monitor? Where are records stored? Can the agent modify them? Does the monitor observe tool activity through a path independent of the agent’s own report?
A fictional agent may run in one service while tool events are recorded by another. That separation can be useful if the agent lacks authority over the recording process. Placing both services on the same machine does not automatically defeat independence, just as placing them on different machines does not automatically establish it.
Common credentials can weaken the boundary. If the agent can use the same administrative account that manages logging, its practical control may extend beyond the apparent separation.
Design should therefore examine privileges and failure paths, not only physical placement. The purpose of independence is evidence that remains trustworthy enough to support response when the agent’s own behavior is in question.
Worked example: a research agent with a narrow workspace
Consider a fictional assistant asked to prepare a comparison from six approved documents. Its permitted operations are reading those documents and creating a report in a designated workspace. It has no authority to edit the source material or distribute the report.
The task brief identifies the comparison criteria and asks the assistant to mark missing information. The access layer enforces the read-only source boundary and write location. Tool events record which documents were opened and which output file was created.
If the agent attempts to edit a source document, the operation is rejected and logged. The response process can determine whether the attempt resulted from a misunderstanding, tool-selection error or instruction hidden in a document.
The final review examines the report’s quality separately from the access controls. A contained agent can still misunderstand a source or make a poor comparison. The user checks consequential claims against the documents and assesses whether the requested criteria were covered.
This example shows the distinct roles. Containment limits possible changes. Monitoring provides evidence about behavior. Review evaluates the usefulness and correctness of the final work. A successful workflow requires all three.
Worked example: a scheduling agent with conditional authority
A fictional scheduling agent can read availability and propose an appointment. It may create an entry only after receiving the required authorization for a specific date, participants and destination calendar.
The permission should not be inferred from a vague request to arrange the week. The system can prepare alternatives while the actual commitment remains unresolved. Once authorization is provided, the action should be checked against the authorized details.
A useful design connects the approval with the exact proposed operation. If the date or participants change afterward, the earlier approval may no longer describe the action. This avoids treating approval as a reusable permission for every similar request.
Monitoring records the proposed operation, authorization, attempt and confirmed result. If the calendar service reports a conflict or rejects the request, the agent should communicate that outcome accurately.
The workflow also needs correction. A duplicate entry, partial failure or changed availability can require review. The agent should have only the authority necessary for the agreed repair. Conditional access enables useful action while keeping the commitment understandable.
Worked example: a code agent before production access
A fictional code assistant modifies a small application in an isolated development environment. It can inspect relevant files, write changes and run permitted checks. Production credentials are absent from this stage.
The initial evaluation examines whether the changes meet the specified behavior and whether the agent remains within the workspace. A test that passes does not itself authorize deployment. The production step has different consequences and should be governed separately.
The assistant can prepare a reviewable change, document the behavior and identify unresolved checks. An authorized deployment process can then decide whether the result is ready and perform the release under appropriate permissions.
Monitoring should preserve the difference between running a local check and changing a live service. Both may appear as tool operations, but their significance differs.
This separation makes failures easier to contain and diagnose. A mistaken edit can be corrected in development. A deployment error requires an operational response. Keeping the stages distinct supports productive autonomy without granting the agent every capability at the beginning of the task.
Make escalation actionable
An alert is useful when it reaches a process capable of responding. Escalation should identify the condition, relevant evidence and available actions. A stream of unexplained warnings can create fatigue without improving control.
A fictional agent repeatedly requesting access to an unrelated folder may trigger a pause and review. The reviewer needs to know the intended task, attempted resource and policy decision. That information helps determine whether the permission is missing legitimately or the request is outside scope.
Responses can include rejecting one operation, pausing the task, revoking access or continuing with a narrower scope. The appropriate response depends on the consequence and the reliability of the evidence.
Escalation should also communicate what happened to the user. A task may remain incomplete because an action was rejected. Reporting completion despite that rejection would create a separate correctness failure.
Define ownership before an incident. If no service or responsible role can act on the alert, monitoring has identified a problem without supplying a response. The operational design should connect detection with a decision and a usable recovery path.
Understand the limits of detection
A monitor can examine events that reach its observation points. It may not know every internal reason for an action or every consequence outside the instrumented environment. Its coverage should be described explicitly.
Detection also takes time. An alert issued after an irreversible operation cannot prevent that operation, though it may support response. Where prevention is required, the relevant policy must be checked before the action is completed.
The design therefore distinguishes preventive controls from detective controls. A destination restriction can reject an unauthorized transfer. A log can preserve evidence of the attempt. Both are valuable, but they perform different jobs.
False alarms and missed detections should be examined within the task. An overly broad rule can block legitimate work; an overly permissive rule can overlook a meaningful departure. The balance cannot be assessed without understanding the action and consequence.
The monitor’s own health matters as well. Missing events, delayed delivery or storage failures can weaken the evidence. A system should be able to recognize when observation is impaired and apply an appropriate operating response.
Test containment with permitted and forbidden cases
A containment test should establish that legitimate work remains possible and that prohibited operations are rejected. Testing only the second can produce a secure-looking environment that cannot complete its assigned task.
A fictional report agent can be tested on reading an approved source and creating an allowed output. Separate cases can attempt an unrelated read, a source edit or an external transmission. The expected result should be specified before the test.
Tests should examine the available paths to the same consequence. A file restriction may work through one tool and fail through another. The scope of the evaluation should follow the actual configuration rather than a generic list.
Record the configuration with the result. Permissions, tool versions and policy settings can change. A passed test for one arrangement cannot establish the behavior of an altered arrangement without further examination.
Testing also needs a realistic workflow. A narrow isolated operation may behave correctly while a longer sequence reveals an unexpected permission interaction. The relevant question is whether the controls remain effective during the kind of work the agent is expected to perform.
Separate content correctness from action safety
An agent can stay within permissions and still produce a wrong answer. It can also produce a correct answer while taking an unauthorized action. These are independent dimensions of evaluation.
A fictional research report might faithfully preserve file boundaries yet compare inconsistent units. The containment system has done its job; the analytical result still needs correction. Another agent might produce an accurate summary after reading a document outside its approved scope. Accuracy does not erase the access violation.
This distinction helps organizations avoid exaggerated claims. A secure runtime does not prove that every response is true. A strong benchmark does not prove that every tool action is authorized.
The complete evaluation should therefore include output quality, permission behavior and operational response. Each needs evidence suited to its purpose. Combining them into one reassuring label can hide important weaknesses.
The practical benefit is clearer repair. Improve reasoning or evidence handling when the content is wrong. Improve enforcement when authority is exceeded. Improve detection or recovery when the system cannot establish what happened. The mechanism determines the intervention.
Preserve logs without creating unnecessary exposure
Logs can contain sensitive data, credentials, document excerpts or personal information. Recording everything indiscriminately may create a new risk while attempting to manage another one.
A useful logging plan identifies the evidence required for review and the data that should be minimized or protected. An event may need a resource identifier and policy decision without retaining the entire document content. Another task may require a controlled record of a changed field.
Access to the logs should also be scoped. A reviewer may need to inspect events without permission to modify them. An operator may need to maintain the logging service without routine access to all sensitive content.
Retention should follow the purpose and applicable requirements. Keeping records indefinitely is not automatically more responsible than deleting them too soon. The organization needs a reasoned policy and a way to apply it.
A fictional investigation can often begin with metadata and obtain additional protected detail only when necessary. This approach preserves useful evidence while recognizing that the observation system is itself part of the data environment.
Carry controls from evaluation into deployment
Evaluation establishes behavior in a configured environment. Deployment should preserve the relevant controls or explicitly account for the changes. Otherwise, the evidence may describe a system that is no longer operating.
Adding a new tool can create a new path to resources. Expanding a permission can change the impact of a mistaken action. A model update can change tool selection or interpretation. Each alteration should be examined in terms of the task and affected controls.
A fictional assistant that passed testing with read-only access should not inherit broad write access merely because it is now considered ready. Readiness concerns the evaluated arrangement.
The deployment record can identify the approved task, permissions, monitoring and response process. It should also explain what change requires renewed review. This makes the boundary maintainable rather than dependent on memory.
Controls should remain usable during ordinary operation. A review process that is too slow or unclear may encourage workarounds. The design should therefore connect safety with the practical flow of work, preserving legitimate capability while making unauthorized expansion difficult.
A consequential application is scoped authority in a healthcare workflow; evidence about clinical performance does not by itself grant permission to disclose records or change a care process.
Plan recovery before granting consequential authority
Some actions can be reversed easily; others require coordination or leave lasting effects. Permission design should consider that difference before the agent acts.
A fictional file-editing task can preserve an earlier version for restoration. A scheduling task may need a method for removing a duplicate entry and notifying affected participants. A publishing task may need a clear route to correction. These arrangements should be scoped to the actual workflow.
Recovery should not rely entirely on the agent that caused the problem. An independent operator or service needs enough authority to pause the task, revoke access or restore an appropriate state.
The plan should also account for partial completion. A multi-step operation may succeed in one service and fail in another. The system needs evidence of which changes occurred before deciding how to continue.
A recovery rehearsal can expose missing permissions or unclear ownership. The value lies in demonstrating that the response is possible under realistic conditions. An emergency instruction written in a document is less useful if the necessary tools or authority are absent when needed.
Bound the duration and quantity of authority
Permission has a time dimension as well as a resource dimension. A task may need access for one session or one approved operation. Leaving that access available afterward can expand the period in which a mistake has consequences.
A fictional publishing assistant might receive authority to update one page after review. The authorization can be tied to that page and the approved change, rather than remain a general publication permission. Expiry or revocation should be enforced by the surrounding service where the design supports it.
Quantity also matters. An agent permitted to make one test request is not necessarily authorized to repeat it without limit. A resource budget can constrain the number of operations, duration or expenditure associated with a task.
These controls need observable outcomes. If access expires, the agent should receive a clear result and avoid pretending that the operation succeeded. If a budget is exhausted, the task should report what was completed and what remains.
The purpose is precise delegation. A bounded capability can support useful work while giving the organization a clearer account of how much authority was available and when it ended.
Test the monitor as a component in its own right
The monitoring service needs evaluation, not only the agent it observes. A useful test asks whether expected events arrive, whether their order can be reconstructed and whether missing observation becomes visible.
A fictional rehearsal can perform an allowed write, a rejected write and a partially completed operation. Reviewers then compare the service’s confirmed outcomes with the records available to the monitor. If one operation is absent from the record, the gap should be investigated before treating the logs as complete evidence.
The rehearsal can also simulate an observation failure without exposing real data or systems. The question is what the environment does when logging is unavailable. Some tasks may pause; others may continue under a narrower permitted scope. The choice should reflect the consequence and documented operating policy.
A monitor can be accurate yet unhelpful if its events arrive too late or lack the detail required for response. Testing should therefore include the response process, not merely count recorded messages.
This creates an assurance chain: the action boundary is tested, the observation path is tested and the response is rehearsed. Each component has an inspectable purpose.
The assurance consequence is audit evidence for the deployed control system; reviewers need evidence that monitors, escalation and enforcement work together under the tested conditions.
Keep the task’s stopping condition visible
An agent that can continue across steps needs a definition of completion. Without one, it may keep searching, revising or requesting access after it has produced a usable result.
A stopping condition can describe the deliverable and required checks. A fictional research task may end when the comparison covers the approved criteria, unresolved information is identified and the report is saved in the designated location. It does not require unlimited investigation into every related topic.
Failure also needs a stopping condition. If a required source is unavailable or an essential operation is rejected, the agent should preserve completed work and report the unresolved dependency. Repeating the same attempt indefinitely creates activity without progress.
The monitor can help identify behavior inconsistent with these conditions, such as repeated requests producing the same rejection. A response may narrow the task or request review.
Completion should be communicated through evidence: the actual output, confirmed operations and remaining uncertainty. This keeps autonomy connected to a defined result. The useful agent can continue long enough to finish authorized work and stop when further action no longer serves that work.
Frequently asked questions about AI agent containment
Is a sandbox enough to make an agent safe?
A sandbox limits some parts of the environment, but its usefulness depends on what resources remain reachable. Broad credentials, network tools or service permissions can create consequences outside the local workspace.
Examine the complete authority map. Containment should cover the operations relevant to the task, while monitoring and output review address behavior and correctness. A sandbox is one component of the operating arrangement, rather than a universal guarantee.
Why should monitoring be independent of the agent?
The agent’s report may be incomplete or inaccurate, particularly when its own behavior is being investigated. Independent records provide evidence of attempts, policy decisions and confirmed effects.
Independence concerns control over observation and records. Separate hardware may help, but the key questions are whether the agent can alter the monitor, disable logging or rewrite evidence. The trust boundary should be examined directly.
Does limiting permissions make an agent less useful?
It can if the permissions do not match the task. The aim is to provide the authority required for legitimate work while excluding unrelated operations.
Test successful tasks as well as rejected actions. If a needed operation is missing, revise the scope through an appropriate authorization process. Clear permissions can support usefulness by making capability predictable and its consequences easier to manage.
What should happen when an action is rejected?
The system should preserve the reason and explain the effect on task completion. It may continue work that remains authorized, ask for the missing decision or pause where the rejected operation is necessary.
A rejection should not be hidden behind a claim of success. The response process needs to distinguish an outside-scope action from a legitimate task requiring additional access. Evidence about the operation helps make that distinction.
How can an organization know whether controls still work?
Maintain a record of the configuration, test realistic allowed and prohibited operations, monitor actual behavior and review relevant changes. Include a way to detect impaired observation and to recover from failure.
No single successful test establishes permanent safety. The operating environment changes, and the control evidence should change with it. The goal is a maintained understanding of authority, behavior and response.
Continue through the series
Return to Super Intelligence: Energy, Work, Learning and Safety. Explore everyday delegation in 0015 — AI Literacy for Work and Everyday Life, then examine the policy question in 0019 — Values Judgment and the Intent of the User. Organizational accountability continues in 0020 — Safety Audits and the Conditions for Faster Progress.
Previous: 0017 — Faster Answers and Deeper Learning · Next: 0019 — Values, Judgment and the Intent of the User