Super Intelligence master guide › Energy, work, learning and safety series › Article 0020
AI safety audits examine whether a configured system behaves within its intended scope and whether the organization has evidence, controls and response processes appropriate to its consequences. Their purpose is to support decisions about use, improvement, restriction or withdrawal. A reassuring label is insufficient without an inspectable connection between the claim and the evidence.
Safety can enable faster progress when it makes useful deployment more dependable. Clear permissions can reduce uncertainty about action. Reliable evaluation can identify where a system belongs. Monitoring and correction can make experimentation easier to manage. These benefits arise from the specific control and workflow, rather than from the word safety alone.
Some safeguards also add cost or friction. The relevant question is whether the complete arrangement improves the ability to deliver useful work within acceptable limits. Measuring only release speed overlooks failures and repairs; measuring only restrictions overlooks legitimate work that never becomes possible.
The Super Intelligence series uses SI as a broad theme for machine intelligence. Current AI systems should be judged through evidence about their configured tasks and operating environments. Hypothetical technical superintelligence would intensify questions of authority and accountability; it does not establish that today’s systems have reached that threshold.
Connect safety with a concrete use
An audit begins with the intended use. A system summarizing approved documents, an agent changing records and an application influencing a consequential decision have different operating requirements. The evaluation should follow the actual task rather than a generic category.
A fictional business assistant may prepare draft replies but lack authority to send them. Its evidence should cover the drafting task, source fidelity and review process. If sending is added, the permission and delivery process require further examination.
The intended use also identifies who relies on the output and what failure means. An inaccurate heading in a private draft differs from an incorrect commitment in a customer message. The task boundary gives the audit a basis for prioritizing evidence.
A useful description includes the input, output, user role, tools, permissions and limits. It should identify assumptions that must remain true for the evidence to apply.
This makes safety operational. The question is whether the system can support a particular use under particular conditions, with a response when those conditions change. A broad claim that the model is safe cannot replace that description.
Use a framework without turning it into a certificate
NIST’s AI Risk Management Framework Core organizes risk management through govern, map, measure and manage. Governance is cross-cutting; the functions are not a mandatory ordered checklist. The framework emphasizes continuing attention across the system lifecycle and allows organizations to apply it according to their context.
Its practical value is to connect purpose, evidence and response. A project needs an accountable operating arrangement, an understanding of its effects, measurements relevant to those effects and decisions based on the findings.
NIST’s Generative Artificial Intelligence Profile provides a companion resource addressing risks associated with generative AI. It is guidance for organizing risk management, not a universal product approval.
An organization should therefore describe what it actually did. Naming a framework does not establish that the relevant activities occurred or that a particular application meets its intended conditions. The audit needs records, test results and evidence of response.
The methods in this chapter are practical proposals for turning that connection into a reviewable process.
Document product intentions before evaluating behavior
Product intentions describe what the application is supposed to achieve and what it is not authorized to do. They should be specific enough that an evaluator can recognize a departure.
A fictional research assistant’s intention might be to compare approved documents and create a report with clearly identified uncertainty. An intention to be helpful is too broad to determine whether an unsupported recommendation belongs in the task.
The documentation should include expected users and operating conditions. A specialist workflow may depend on professional review. A public interface may need a different explanation and control arrangement. The same model can support different products with different evidence requirements.
Intentions also need version control. If the task changes, the earlier evaluation may no longer support the current claim. The record should identify what changed and which evidence needs reconsideration.
Clear intentions make disagreements more productive. Reviewers can ask whether the system met its stated purpose, whether that purpose is appropriate and whether the surrounding controls support it. Those are different questions, and documentation gives each a concrete starting point.
Map consequences before selecting metrics
Metrics should follow the consequence being assessed. Counting correct answers may be useful, but it cannot reveal every effect of an agent that can alter a record or distribute information.
A fictional support assistant can fail by inventing a commitment, omitting an exception or sending a response to the wrong destination. These failures concern content, meaning and action. An evaluation that measures only grammatical quality would miss all three.
Consequence mapping can also include burdens. A tool that generates many drafts may increase review effort. A control that blocks routine authorized work may push users toward an unsupported workaround. The system’s practical effect includes these interactions.
The map should distinguish plausible effects from established ones. An imagined severe outcome can guide investigation without being reported as something that occurred. Actual incidents should be documented with evidence.
This helps prioritize evaluation. Focus on the failures that would materially affect the intended use, then add broader tests when new information justifies them. A large collection of metrics is less valuable than a smaller set connected with the decisions the organization must make.
An audit first requires the purposes and priorities behind acceptable risk; choosing a test threshold already reflects judgments about whose outcomes matter and who can make the decision.
Measure the configured system rather than a model name
A deployed application includes prompts, retrieval, tools, permissions, interfaces and human review. The model is one component. Evaluation should describe the complete arrangement that produced the result.
A fictional assistant might use a fixed collection of approved documents in one configuration and unrestricted external search in another. The second configuration has a different evidence environment even if the model name is unchanged.
The record should therefore identify the relevant versions and settings. It should explain what input was available, what actions were possible and what review occurred. This allows a result to be reproduced or interpreted when the system changes.
The evaluation should also include representative workflows. A model may answer isolated questions accurately but fail when a task requires maintaining several conditions across steps. Conversely, a well-designed workflow may improve reliability through retrieval and checks.
This broader view avoids both exaggerated optimism and exaggerated pessimism. A weak result may arise from a repairable configuration problem. A strong result may depend on controls that must remain present. The audit needs to determine which arrangement the evidence actually supports.
The operating mechanism is testing prevention, detection and recovery together; a promising model result cannot establish whether permissions, monitors and recovery actually constrain the deployed workflow.
Build an evidence register with traceable claims
An evidence register connects each important claim with its supporting material and limitations. It can be simple: the claim, evaluation method, configuration, result and unresolved uncertainty.
A fictional claim that an assistant reduces review effort should be supported by a comparison that includes review time. A claim that it preserves confidentiality should connect with access tests and data-handling evidence. A claim that it remains within permissions should connect with actual enforcement behavior.
The register should avoid replacing an unsupported claim with a larger document. Quantity is not the measure. The useful question is whether another reviewer can follow the connection and assess its strength.
Evidence can have different forms. Quantitative tests, qualitative review, incident records and user feedback can each contribute when interpreted appropriately. A testimonial about convenience does not establish an error rate. A test score does not explain every user’s experience.
Recording limitations is part of the evidence. If a relevant condition was not tested, the claim should remain scoped accordingly. An explicit gap supports a better decision than a vague assurance that everything was considered.
Use internal review without relying on self-assessment alone
Internal review has access to the design and can identify issues early. It can examine changes before release, test assumptions and investigate failures. Its proximity is valuable, but it can also share the development team’s blind spots.
A useful arrangement gives review a defined role and enough independence to challenge the implementation. The reviewer should be able to report an unresolved issue without treating delivery as the only acceptable outcome.
A fictional product team may believe a new permission is necessary for convenience. Internal review can ask whether the task requires that permission and whether a narrower operation would suffice. The review is stronger when supported by actual workflow evidence.
The process should record the finding, response and remaining risk. A discussion that produces no traceable decision is difficult to evaluate later.
Internal review is therefore one layer of accountability. It should contribute to the evidence chain, while additional scrutiny examines assumptions that the organization may not notice or may have incentives to minimize.
Define what independent review can actually inspect
External or independent review is useful when it has a meaningful scope and access to relevant evidence. The label independent does not establish those conditions by itself.
A fictional assessor reviewing an agent needs to know the intended task, configuration, permissions and evaluation results. If the assessor sees only a demonstration chosen by the developer, the review supports a narrower conclusion than a full operational audit.
Independence also concerns incentives and reporting. Who selects the scope? Who can suppress a finding? Can the assessor examine unfavorable cases? How are limitations communicated? These questions help interpret the result without assuming that every review arrangement is identical.
Confidentiality can restrict access, but the restriction should be recorded. A reviewer may still assess some claims while leaving others unresolved. The final statement should reflect that boundary.
A useful audit conclusion describes what was examined, what was found and what remains outside scope. This gives decision makers evidence they can use without turning the assessor’s name into an unlimited endorsement.
Worked example: audit a document assistant
Consider a fictional assistant used to prepare internal reports from approved records. Its intended task is to summarize and compare, with no permission to edit the source collection or send material externally.
The audit begins with the task definition and configuration. It examines whether the assistant can access only the approved material, whether outputs preserve uncertainty and whether the final report can be checked against sources.
Representative cases include a missing term, conflicting versions and inconsistent units. These cases test the work the assistant actually performs. Access tests separately examine whether prohibited edits and transmissions are rejected.
The audit also considers the review process. Can a user locate the evidence behind a consequential statement? How long does correction take? Are repeated omissions reported and used to improve the workflow?
A scoped conclusion might support continued use for internal drafting with specified review. It would not authorize publication or unrestricted access. The evidence is valuable because it describes a concrete arrangement and gives the organization a basis for maintaining or revising it.
Worked example: audit an agent that can change records
A fictional operations agent can update selected inventory fields under defined authorization. Its useful capability includes action, so the audit must examine both decision quality and enforced scope.
The evaluation can test approved updates, rejected outside-scope requests and incomplete operations. It should establish whether the service confirms the actual result and whether the agent reports that result accurately.
A multi-step case may reveal partial completion. The agent updates one field but fails to update another. The audit examines whether the workflow recognizes the inconsistency, preserves evidence and routes the case for repair.
The organization also needs a recovery method. A mistaken update should have a defined correction process, and the agent’s authority to perform that correction should be explicit.
The audit conclusion should connect permissions, observation and response. A successful content benchmark cannot establish action safety, and a successful access test cannot establish that the updates are substantively correct. Both dimensions belong in the operating evidence.
Worked example: assess a release decision
A fictional team wants to release a new version of an assistant with broader retrieval and an additional tool. The previous version passed its evaluation, but the changed configuration creates new questions.
The team identifies which claims remain supported and which need renewed evidence. A broader retrieval source may affect factual quality. The new tool may expand action authority. The interface may need to explain a different operating boundary.
The release review examines tests relevant to these changes. It also considers unresolved findings and the response available if deployment reveals a problem. The decision can approve a narrow stage, require repair or defer the expansion.
This is where safety can support speed. A clear process prevents the team from restarting every question from scratch while ensuring that changed assumptions receive attention. It preserves earlier evidence where it applies and focuses new work where it does not.
The release should communicate its scope. A successful limited deployment does not automatically support every later extension. Progress becomes more manageable when evidence travels with the configuration it describes.
Connect governance with decisions and resources
Governance defines who is accountable, who can authorize use and who can respond to findings. It should connect review with actual decision-making authority and the resources needed to carry out that decision.
A fictional oversight committee may receive reports but lack authority to pause deployment. That arrangement provides visibility without complete control. Another committee may have authority but receive information too late to act usefully.
The organization should therefore define both information flow and decision rights. Findings need a route from the evaluator to a responsible role, with an expected response appropriate to the consequence.
Resources matter as well. Monitoring requires maintenance. Review requires time and expertise. Correction requires access and ownership. A policy that assigns duties without supporting them can create an appearance of accountability while leaving the work incomplete.
Governance should also include third-party dependencies. A service update or changed data source can affect the application. The organization needs a way to learn about relevant changes and reconsider the evidence where necessary.
Make escalation more than a reporting habit
Escalation should lead to a decision. A finding can be accepted with documented limits, repaired, mitigated through a narrower use or treated as a reason to pause. The appropriate response depends on evidence and consequence.
A fictional audit identifies that an assistant omits exceptions in a particular document format. The response could restrict that format, revise preprocessing and add evaluation cases. Simply noting the issue in a report does not change the operating risk.
The escalation record should explain who considered the finding and what action occurred. It should also identify unresolved parts. This preserves accountability when a decision is revisited.
Timing belongs in the process. A concern that affects current operations may need a faster route than a low-impact improvement suggestion. A uniform reporting schedule can be inappropriate if it delays a necessary response.
The aim is not to escalate every imperfection to the highest level. It is to ensure that significant evidence reaches a role capable of acting and that the decision remains understandable afterward.
Monitor the application after release
Release changes the evidence environment. Actual inputs, users and workflows may differ from the evaluation set. Monitoring examines whether the intended arrangement remains useful and within its limits.
The plan should connect observations with response. A repeated error, changed input pattern or unexpected tool use may trigger review. The trigger should reflect the task rather than a generic demand to log more information.
User feedback can reveal problems that automated measures miss. A summary might be technically faithful yet omit a detail needed for the actual decision. Feedback should enter a process that can examine the claim and update the evaluation.
Monitoring should also identify impaired observation. Missing events or delayed records can make the application harder to assess. The operating response may need to change when the evidence path is unavailable.
Post-release monitoring therefore extends the audit. It does not replace careful evaluation before use, but it provides information about conditions that cannot be fully captured in advance.
Manage change as a change in evidence
A system’s claim remains credible only while its supporting conditions remain relevant. Model updates, new tools, changed policies and different data sources can alter those conditions.
Change management should identify what changed and which evidence is affected. A minor wording adjustment may need a limited check. A new external action may require a broader review of permission, monitoring and recovery.
A fictional application introduces an automatic sending step after previously producing drafts only. That change is more than a convenience feature. It moves the workflow from information production to external action and creates a new authority question.
The organization can preserve unaffected evidence while examining the changed component. This is more efficient than either retesting everything without reason or assuming the earlier result covers the entire new system.
A clear change record supports later investigation. If an incident occurs, reviewers can identify the configuration and decisions relevant to it rather than reconstructing the history from memory.
Share practices without claiming identical contexts
Sharing evaluation methods and failure lessons can help organizations learn beyond their own experience. A useful shared practice explains the task, configuration, failure mechanism and repair, with appropriate protection of sensitive information.
A fictional organization may discover that conflicting source versions cause unreliable summaries. Sharing a method for identifying authoritative versions can help another organization examine a similar workflow. The second organization still needs to assess its own conditions.
This distinction prevents a practice from becoming a universal prescription. An access rule suited to one service may not fit another. A test set can inspire evaluation without establishing another system’s performance.
Sharing should include limitations and unsuccessful approaches where appropriate. A record containing only successful demonstrations gives a distorted picture of the work required.
The practical benefit is a stronger common language for questions and evidence. Organizations can compare mechanisms, identify relevant differences and avoid repeating avoidable mistakes without pretending that their applications are identical.
The same evidence habit can inform auditable promises to affected communities; commitments need scope, ownership and observable delivery, while their legal and contractual requirements remain specific to the project.
Evaluate whether safety enables useful speed
The acceleration claim should be measured through the complete delivery process. Useful speed includes preparation, release, reliable operation, correction and recovery. A rapid launch followed by repeated repairs may not be the fastest route to sustained benefit.
A fictional workflow with clear permissions can reduce repeated uncertainty about which actions are allowed. A well-designed evidence register can shorten release review because relevant results are easy to inspect. A rehearsed recovery process can reduce disruption when a failure occurs.
These are conditional benefits of specific arrangements. Controls can also create unnecessary friction if they are poorly matched to the task. Evaluation should record that burden and consider a more precise design.
Compare the workflow with a reasonable alternative. Ask whether the safeguard changes the outcome, whether its cost is proportionate and whether it preserves legitimate capability. A control that exists only for appearance should not receive credit for a benefit it has not demonstrated.
The strongest relationship between safety and progress is maintained usefulness. A system can take on more work when the organization has credible evidence about where it belongs and can respond when that evidence changes.
The useful consequence is adoption justified by relevant operating evidence; assurance can support a broader rollout when the task, conditions and remaining uncertainties match the evidence.
Communicate findings without turning them into guarantees
An audit report should state its scope, method, result and limitations in language suited to its audience. Technical detail can remain available while the main conclusion explains the operating decision.
A fictional report might state that a document assistant met specified source-fidelity criteria on representative cases and that access restrictions rejected the tested outside-scope operations. It should also identify conditions not evaluated and any required user review.
Avoid absolute language that the evidence cannot support. Passing a test does not prove that failure is impossible. External review does not establish that every future configuration will behave identically.
The report should distinguish a finding from a management decision. Evaluators provide evidence; the authorized organization decides whether use proceeds within its responsibilities. Recording both makes accountability clearer.
Communication is part of the control system because users act on the claims they receive. A precise statement helps them use the application within its evidence, while an exaggerated statement can undermine the very safeguards the audit examined.
Examine the assumptions behind a test set
A test set contains selected cases. Its usefulness depends on how those cases connect with the intended use. A large set of easy examples can provide less relevant evidence than a smaller set that includes the failure conditions the application is likely to encounter.
A fictional comparison assistant should be tested on complete proposals, missing terms and conflicting definitions if those appear in actual work. The evaluation should explain how cases were selected and which important conditions remain absent.
Repeatedly using the same visible tests can also shape development toward those examples. Improvement on the test may reflect a useful repair, but reviewers should examine whether the mechanism transfers to new cases. Holding back representative cases or conducting an independent review can help investigate that question.
A test result should therefore include the population it describes and the uncertainty around generalization. A clean score does not reveal what was never tested.
This is an evidence-design issue, not a demand for endless testing. The organization should select tests that answer the decision at hand and expand them when a new use or observed failure exposes a meaningful gap.
Evaluate human review as a real control
Requiring human review does not establish that the review is effective. The process needs suitable information, time, expertise and authority to act on a finding.
A fictional agent presents a large set of record changes for approval. If the reviewer cannot compare them with the originals, the approval may become a superficial acknowledgment. If the interface groups harmless edits with consequential ones, attention may be directed poorly.
Evaluation can therefore examine representative review tasks. Can the reviewer identify an inserted error? Is the relevant source accessible? Does the process make uncertainty visible? What happens when the reviewer rejects part of the result?
The point is to assess the actual human-system arrangement. A capable reviewer can still be undermined by an unsuitable interface or unrealistic workload. Conversely, a clear comparison view can make review more useful without requiring the reviewer to reconstruct every step manually.
The audit should describe what review achieves and where it remains limited. This avoids using the phrase human oversight as a substitute for evidence about how the oversight operates.
Assess third-party components through their actual role
A deployed application may rely on a model provider, retrieval service, external dataset or tool integration. Each dependency contributes a different kind of evidence and uncertainty.
A fictional organization can receive a provider’s performance report, but the report may describe tasks unlike the organization’s workflow. The useful question is which parts support the local claim and which need evaluation in the configured application.
Dependencies also have operating assumptions. A service may change its response format, availability or features. The organization should identify which changes could affect the application and how it will learn about them.
The audit does not need to reproduce every provider’s entire evaluation. It needs a traceable account of reliance: what the component does, why it is suitable and what uncertainty remains.
This supports informed adoption. Trust in a provider can be relevant, but it should not erase the organization’s responsibility for the arrangement it builds. Local evidence connects the component’s capability with the actual task, permissions and review process.
Plan retirement as part of accountability
An application may need to be withdrawn, replaced or restricted. Responsible retirement considers the work and data that depend on it, rather than treating shutdown as a complete solution by itself.
A fictional assistant that supports a recurring reporting process may hold configuration, templates and records needed by another workflow. The organization should identify how those materials will be preserved or transferred appropriately if the service ends.
Users also need to understand what will change. A previously available action may no longer occur automatically. A new process may require a different review step. Clear communication helps prevent abandoned responsibilities.
Retirement can include a final assessment of unresolved incidents and continuing obligations. A problem does not disappear merely because the application is no longer active.
The ability to stop use is therefore an operational capability. It requires ownership, evidence and a transition plan. Including it in governance makes the organization less dependent on continuing a system whose benefits or operating conditions no longer justify its use.
Frequently asked questions about AI safety audits
Does an audit prove that an AI system is safe forever?
No. An audit provides evidence about a defined system, scope and set of conditions. Changes in configuration, inputs or use can affect the relevance of that evidence.
A useful report identifies its limits and the monitoring needed afterward. Safety is maintained through evaluation, operational controls and response, rather than established permanently by one assessment.
What is the difference between a benchmark and an audit?
A benchmark measures performance on a selected task or dataset. An audit examines a broader claim about a configured application and the process supporting its use.
A benchmark can contribute evidence to an audit, but it does not automatically cover permissions, data handling, user review, monitoring or recovery. The intended use determines which additional questions matter.
Is external review always better than internal review?
They offer different strengths. Internal review can examine the design closely and identify issues early. External review can challenge assumptions and reduce some conflicts of interest.
The value depends on scope, access, expertise and the response to findings. A limited external demonstration review may support less than a rigorous internal investigation. A dependable arrangement makes each role and its limitations explicit.
Can safety measures slow useful progress?
Yes, especially when they are poorly matched to the task or create effort without addressing a relevant failure. The complete workflow should be evaluated for both benefit and burden.
The response is to improve precision, not remove every boundary. A safeguard that supports dependable operation can enable useful deployment. The evidence should show how it affects actual work rather than assume the relationship.
What should leadership ask when reviewing a new application?
Ask what task the system performs, which evidence supports it and what authority it receives. Then examine monitoring, correction, ownership and the conditions that would require reconsideration.
The decision should connect benefits with operating limits and resources. A polished demonstration is a starting point for questions, not a substitute for the evidence needed to authorize a consequential use.
What completes the chain from capability to public benefit?
Useful capability must reach an appropriate task, operate within legitimate authority and produce results that can be trusted enough for their purpose. Institutions need evidence and a way to correct failures.
This connects the series’ physical foundations with its human outcomes. Energy and computing supply capacity. Applications turn capacity into work. Skills, controls and accountability determine whether that work delivers durable value.
Continue through the series
Explore the full collection in Super Intelligence: Energy, Work, Learning and Safety. The operational controls are developed in 0018 — Containment and Independent Monitoring for AI Agents, and policy interpretation in 0019 — Values Judgment and the Intent of the User. Return to 0001 — Computing Reinvented from the Ground Up to connect governance with the complete computing stack.
Previous: 0019 — Values, Judgment and the Intent of the User