Credential rotation is the controlled replacement of a secret, token, certificate or signing key used by an AI agent to call a tool. A successful rotation gives the intended workload valid new access, removes unnecessary old access, and proves that actual authorised work still functions. Simply generating a new key is not a successful rotation. A worker may continue using a cached old credential, a paused agent may wake up with expired authority, or a tool may accept the new token while rejecting its required scope. Rotate according to the credential type, verify each consumer, and revoke promptly when exposure is suspected.
For practical AI systems, a credential must never become part of a prompt simply because the assistant needs to use a tool. The tool runner should acquire and protect it outside model-visible text, enforce the right permission, handle expiry predictably and audit the outcome without logging secret values. This guide shows a safe lifecycle, a fictional hundred-call rotation exercise with checked arithmetic, incident handling, readback tests, a different-provider transfer problem and the circumstances where a scheduled overlap must be abandoned in favour of emergency revocation. These are controls for present-day systems, not claims that hypothetical superintelligence already exists.
This chapter extends the How Super Intelligence Works library. Read agent identity and delegation for authority and scope, durable workflows for sleeping tasks and incident response for containment after a suspected leak. Examples below use invented names and values; they are not vendor prices or observed reliability statistics.
Choose a reading route
- Building an assistant that calls APIs? Start with credential types, inventory and the normal migration sequence.
- Operating a long-running workflow? Focus on expiry, sleeping jobs, cached sessions and renewed authority.
- Responding to a leak? Compare emergency revocation against a planned overlap, and read the failure diagnosis.
- Studying systems engineering? Solve the hundred-call packet and then try the changed-provider exercise.
Contents
- 1. What rotation changes
- 2. Separate five kinds of credential
- 3. Inventories, leases and expiry
- 4. Map consumers and failure modes
- 5. A safe scheduled rotation
- 6. Compromise changes the sequence
- 7. Worked 100-call laboratory
- 8. Compute outcomes without false denominators
- 9. Prove old access is gone
- 10. Diagnose and repair failure
- 11. Transfer exercise and solution
- 12. Limits, trade-offs and governance
- Frequently asked questions
- Official standards and further reading
- Reader pathways
1. What rotation changes
An agent usually consists of a model that proposes a tool call, software that validates it, and an external service that decides whether the caller has permission. The model’s confidence cannot substitute for credentials or authorization. Rotation occurs along the service-to-service trust boundary: a credential stops being the one relied upon, a replacement becomes accepted, and all eligible consumers are migrated. The intended business capability should continue only where its new credential still has the necessary, approved scope.
Why rotate at all? Long-lived credentials can remain useful to an attacker after exposure, accumulate untracked users and survive staff or supplier changes. Rotation can reduce exposure windows, support incident containment and enforce lifecycle ownership. Yet a careless rotation can break important work, especially when tools have long-lived connections, servers validate old certificates against stale trust stores, or queues resume execution under a frozen credential. Secure design needs both a bound on old authority and a plan for uninterrupted legitimate work.
Separate normal change from suspected compromise. A routine scheduled change can sometimes use a short, monitored overlap between old and new credentials. If an old secret is likely stolen, preserving overlap may preserve the attacker’s access as well. In that case, faster revocation, temporarily disabled operations and explicit stakeholder communication may be safer than a graceful handover. No universal “rotate every X days” figure applies to every form of workload credential or risk model.
A rotated secret is also not proof of renewed consent. If a parent approved one booking change, a fresh service key must not grant the agent a new unrelated banking operation. The tool layer should apply least privilege, action-specific permission, audience restrictions and identity checks independently of whether the authentication token is new. That is the key difference between “can connect” and “may perform this user request.”
2. Separate five kinds of credential
A static API key is a secret string recognised by one provider. Its migration usually depends on whether that provider permits two active keys simultaneously, how keys are stored and how workers retrieve updates. An OAuth access token is typically limited by audience, scopes and expiry; obtaining a new one is not necessarily the same as revoking a compromised refresh token or changing the consent grant. A refresh token can be longer-lived and require its own rotation rules. Treat each token family according to its issuer’s documented behaviour.
A workload certificate and private key are different again. A new certificate may identify the same service while using a different cryptographic key pair, trust bundle and validity interval. Its consumers might load it from a workload identity interface rather than a static environment variable. A signing key authenticates artifacts or events rather than necessarily logging into an API; rotating it creates questions about how older signatures will continue to verify. Finally, an interactive human session belongs to an actual user and must not be silently impersonated by an unattended agent.
The SPIFFE documentation on SVIDs describes workload identities and short-lived X.509 documents obtained from a Workload API. The IETF OAuth 2.0 Security Best Current Practice, RFC 9700, addresses OAuth-specific threats and current security recommendations. These are not interchangeable specifications: choosing the right protocol depends on the system boundary, credential issuer, client type and provider capabilities.
List the protection rule beside each credential type. API keys must not appear in user-facing logs or prompts. Access tokens should be scoped and audience-bound where the provider supports it. Refresh tokens need controlled persistence and revocation. Private keys should be generated and handled using appropriate cryptographic facilities. Signing-key changes require a verification and trust-distribution plan. For every type, the operational owner should know who can issue, observe, suspend and terminate the relevant authority.
3. Inventories, leases and expiry
Start with an inventory rather than a rotate button. Each entry needs a credential identifier that is not the secret itself, owning team, issuer, purpose, intended workload, allowed scope, dependencies, creation and expiry policy, storage location, retrieval method, access log location, revocation process and recovery procedure. This list is the foundation for deciding whether the secret is still needed. An unknown consumer can turn a routine replacement into a surprising outage.
Record effective expiry, not just a number in a configuration file. Tokens can fail because an issuer changed policy, a client clock is wrong, a certificate chain is no longer trusted or a service cached outdated data. A thirty-minute token does not imply a thirty-minute safe job lease if an operation runs for forty minutes and cannot refresh. Derive a renewal margin from expected job duration, clock tolerance, propagation delay, queue delay and incident-response time; then test it against actual worst-case behaviour.
One useful state machine is issued, active, migration-in-progress, retiring, revoked or expired, with an exceptional compromised state that triggers containment. Track whether all authorised consumers have stopped using the retiring version. Do not infer successful migration because deployment configuration changed: a paused workflow, a stale worker pod, a forgotten scheduled job or a separate region may still use the old key. Consumer coverage should be observable, and the retirement gate should be explicit.
OWASP’s Secrets Management Cheat Sheet outlines creation, rotation, revocation and expiration as distinct lifecycle stages. For cryptographic keys, NIST SP 800-57 Part 1 Revision 5 describes key-management practices. As checked in October 2026, NIST also lists a Revision 6 initial public draft; it should not be misrepresented as a superseding final standard.
4. Map consumers and failure modes
A connector may be used by more than one agent, region, scheduled task or human-operated recovery script. Draw a dependency graph: secret issuer to secret manager, distribution process, individual consumers, target endpoints, and any downstream delegated calls. Mark how each workload refreshes its credential. One service may reread the vault every call; another may copy a value at startup; a third may hold an authenticated connection for hours. Rotating the vault entry alone does not update every consumer.
Classify each dependent operation by consequence. Read-only catalogue checks may be retried with little business risk after a 401 response. A billing transfer cannot simply be retried under a new key without reconciling whether the original write occurred. A queued action might remain unapproved after a policy change. A signing operation might produce a perfectly valid signature under a key the receiving system no longer trusts. These are separate operational hazards even though all look like “authentication trouble” on a dashboard.
A useful dependency review asks which failures are fail-closed and which can safely continue. Read-only work may queue for a bounded period. Sensitive writes should usually stop when valid authority cannot be proven. If the provider supplies only one active key and no overlapping handover, document a maintenance window or alternative trusted channel rather than inventing the ability to use two keys. Recovery instructions should not depend on the very broken credential they are meant to repair.
Teach the team to recognise status codes cautiously. A 401 can indicate missing or invalid authentication, but provider-specific semantics matter. A 403 may signal insufficient authorization rather than expiration. A timeout does not prove rejection. Clock or certificate-chain problems can appear before an API responds at all. Measure provider behaviour and record the exact error classification; do not launch a blanket key refresh for every failure, because that can mask outages and repeatedly invoke already-accepted operations.
5. A safe scheduled rotation
A planned rotation begins with a tested candidate credential of the correct audience and least privilege. Store it in the approved secret-management system, give it an internal version identifier and confirm its use through a harmless, authorised test. Stage rollout to a small set of stateless read-only consumers first, then to the remaining consumers according to their refresh behaviour. Observe acceptance, latency, permission denials and unexpected old-key traffic during the transition.
Only after every relevant worker has switched, sleeping and scheduled jobs have a safe refresh path, and provider evidence shows that the old version is no longer needed should the owner revoke it. Keep a bounded abort plan. If the new key fails for a subset of workloads, stop migration and diagnose the root cause. Returning temporarily to the old key may be acceptable in a routine change while it remains uncompromised and authorised, but should not be treated as a universal rollback in a security incident.
Instrument the handover without recording secret values. Use credential version IDs, signed or trusted issuer events, connector instance IDs, token expiration buckets and safe error categories. This allows an incident reviewer to ask which worker was stale without leaving a usable token in the evidence ledger. Audit logging should also record who approved the rotation and who can revoke the old authority; the model should never be the sole approver of a scope expansion.
Finish with a closure checklist: authorised calls continue, unsafe writes remained blocked during uncertainty, the former credential has been revoked or destroyed as intended, old sessions and paused tasks behave according to policy, and the next on-call operator knows the outcome. A configuration diff is useful, but final verification belongs at the identity provider and the real consuming services. The following invented laboratory makes this more concrete with denominators, retry limits and a deliberately imperfect result.
6. Compromise changes the sequence
Imagine the old static key appears in a public error log at 09:08. The provider supports simultaneous old and new keys, and a normal change window planned to retire the old one at 09:30. Following the normal overlap would leave twenty-two more minutes of potential access from an exposed key. Once exposure is credible, that graceful sequence is the wrong objective. The incident owner should assess impact, revoke compromised authority as promptly as feasible, interrupt sensitive writes, invalidate dependent sessions where the provider supports it, rotate affected secrets and investigate use since the earliest plausible exposure.
Revocation is often asynchronous across distributors, gateways and caches. After issuing a revoke command, test whether the old identity can still authenticate through representative endpoints, subject to service policy. If an issuer has propagation windows, describe those windows and the residual exposure. A provider response saying “revoke accepted” is not identical to proof that every edge has ceased to honour the old credential. Escalate any unexpected successful use of the old version rather than silently shortening the verification step.
One compromised token may reveal other risks: logs that included the secret, a container image with an embedded copy, a copy in a support ticket, or a developer machine still configured to use it. Replacement without locating the exposure mechanism sets up another leak. Review the source of compromise through authorised defensive processes, remove the credential from unsafe surfaces, scrub applicable log copies under policy, and check for actual misuse. Do not publish token strings in incident reports or examples.
A fresh key may also be insufficient if the underlying workload identity, trusted issuer or host has been compromised. In that case, recreate the workload from trusted infrastructure, review permissions and tokens across related systems, and seek independent security review. The decision is an incident-response judgment, not something a model should make by reasoning privately from a browser page. In uncertain circumstances, stop consequential tool actions and preserve evidence without extending the compromised authority.
7. Worked 100-call laboratory
Use a fictional Harbour Learning Agent connecting to four read-only services: a lesson timetable, a study-resource catalogue, a library inventory and a parent-message archive. It must make twenty-five authorised retrieval calls to each, for one hundred distinct request intentions. All values below are deliberately hypothetical. The owner performs a scheduled key replacement with a candidate introduced at 09:00, canary tests at 09:10, wider rollout at 09:15 and an intended retirement check at 09:30. The agents are not allowed to write or disclose personal messages beyond the approved access scope.
At the first verification window, eighty-eight of the hundred requests have validated source readbacks. Twelve do not. Six failures trace to workers that still hold the old version after it is no longer accepted by the target service. Four connect with the new credential but receive a scope mismatch. Two fail because local validation is affected by a fictional clock-offset problem. The labels identify different remedies. Repeating all one hundred calls under an ever more privileged secret would erase the diagnosis and violate the intended access boundary.
For the six stale-worker requests, operators reload the approved credential through the tool runner and confirm the provider accepts it. All six complete after bounded new attempts that retain their original intention IDs. For the four scope-mismatch cases, the security owner reviews original grants and provider settings. Three were supposed to have a read permission that was accidentally omitted from the new version, and complete once that previously authorised scope is restored. The fourth was genuinely outside the user’s authority and remains denied. Scope correction is never delegated to the model merely to improve throughput.
Of the two clock-related cases, one completes after the service clock and token-validity interpretation are corrected. The other is held because the verifier cannot establish whether its time source can be trusted. With ninety-eight requests confirmed and two safely stopped, the packet reaches an auditable closure. A held request is not described as successful; a justified denial is not described as a broken authenticator. This separation is more valuable than declaring rotation “seamless” because a green deployment indicator appeared.
- 100 read-only intentions = 4 connectors × 25 requests each.
- First independent readback: 88 confirmed, 6 stale-worker failures, 4 scope mismatches, 2 clock-related failures.
- After bounded repair: 6 stale workers succeed; 3 properly scoped requests succeed; 1 clock issue resolves.
- Unfulfilled intentions: 1 correctly denied for missing authority; 1 held due to untrusted time validation.
- Final confirmed outcomes: 88 + 6 + 3 + 1 = 98; all 100 intentions have a recorded disposition.
What should the event stream contain? The rotation event records issuer, version label, intended audience and operator authorisation. Each connector records its active credential version by an opaque identifier, not by secret value. Every request carries a unique intention ID, attempt ID, source endpoint and classification of result. For successful retrievals, a separate check confirms the source response belongs to the right request and has not been mistaken for another user’s record. For stopped requests, the ledger records the control or uncertainty that caused the hold. This is how a rotation teaches accountability rather than just secret replacement.
8. Compute outcomes without false denominators
The first-pass confirmed fraction is 88 ÷ 100 = 88%. That number is not the failure rate of a particular cloud provider: it is a made-up teaching result from a controlled packet with four deliberately introduced problem types. After authorised repair, the confirmed fraction rises to 98 ÷ 100 = 98%. The increase is ten confirmed intentions, or ten percentage points, and consists of six stale-worker recoveries, three restored authorised scopes and one time-validation repair. It is not a ten-percent relative increase; the relative increase is 10 ÷ 88 ≈ 11.36%.
Counting attempts rather than intentions gives a different denominator. Suppose each of the ten repaired intentions needed exactly one additional attempt and neither stopped intention was retried. The system processed 100 original attempts + 10 follow-up attempts = 110 attempts. Confirmed user outcomes remain 98, so dividing 98 by 110 yields about 89.1% confirmed intentions per attempt, a hybrid quantity that does not describe user-success probability. Report attempt volume and verified intent completion separately. A repeated read is not another learner request.
Check migration coverage by consumers as well. If the four services have two workers each, there are eight consumer instances. Suppose seven report loading the replacement while one remains stale. Instance migration coverage is 7 ÷ 8 = 87.5%. This does not mean 87.5% of requests are safe, because the stale instance might handle more traffic than the others. If it serves thirty of one hundred calls, exposure by call volume may be quite different. Good reporting names the population and never allows a neat percentage to hide uneven load.
The overlap duration also needs a precise definition. From initial candidate availability at 09:00 to planned retirement at 09:30 is thirty minutes of overlapping validity, assuming both credentials were actually accepted over that period. From wide deployment at 09:15 to planned retirement is fifteen minutes. These are two different intervals. A test performed only at 09:30 cannot prove that the retired key was never used after 09:15. The system needs version-tagged observations and a policy for verifying revocation after propagation.
9. Prove old access is gone
The final retirement gate has three parts. The issuance service reports the old credential revoked or expired; the inventory finds no authorised consumer configured to retrieve it; and representative old-identity authentication probes fail under a controlled test. A fourth check should verify that new work remains within its intended scope. This is a risk-based assurance process, not a mathematical proof that every unknown copy on earth has disappeared. An old key in a forgotten offline backup might exist even when it can no longer authenticate.
For an OAuth integration, check the right token family. Revoking a refresh token may prevent future access-token issuance but may not invalidate every already-issued access token instantly; the issuer and resource server decide token validity semantics. For a certificate, a new chain may be installed yet old sessions remain open, depending on protocol and implementation. For a static API key, a provider may show revocation centrally while regional verification caches catch up. Read provider documentation and measure actual behaviour; do not state universal instantaneous guarantees.
Independently observe a representative authorised tool operation after migration. A successful login alone is insufficient if required source reads now return filtered results or if the new scope accidentally includes more resources than intended. Validate identity, audience, action permission, data boundary and result. For operations that modify the world, maintain separate approval and readback steps. Secret rotation should not let a model convert a fresh technical login into a fresh user mandate.
Define closure evidence in terms an on-call colleague can inspect. Retain the change ticket or policy approval, opaque version IDs, consumer inventory, counts of failed and repaired intentions, source of revocation confirmation, time of final probes and the person who accepted residual risk. Keep secrets out of the record. If an exception remains, such as a provider with delayed revocation propagation, state it explicitly and assign a follow-up with a deadline rather than rounding the report to “all clear.”
10. Diagnose and repair failure
Problem A: new workloads authenticate, but an old scheduled job fails the next morning. Diagnose configuration caching and the paused workflow’s refresh policy before changing anything else. Repair by retrieving a fresh credential at resumption, rechecking the original authority and testing a paused job in staging. An agent checkpoint should store an intent and durable references, not an indefinitely reusable secret copied into serialized state.
Problem B: the tool returns permission denied after rotation, and the model repeatedly asks for administrator scope. The likely cause may be a migration mismatch, but the proposed remedy creates a new risk. Compare the new scope with the original approved permission map; check audience and API path; restore only permission demonstrably needed and already authorised. If the current request requires a new privilege, route it for human permission rather than teaching the agent that errors justify escalation.
Problem C: a user sees two completed requests after a timeout. Rotating the credential may have caused a retry, but the underlying error is treating unknown outcome as failed outcome. Stop duplicate write retries; query the destination using stable operation or idempotency identifiers; reconcile confirmed effects; compensate only when a separately approved and safe reversal exists. Retry engineering must remain independent of authentication repair.
Problem D: a leak is suspected, but operators preserve a long overlap to avoid inconvenience. The runbook incorrectly treats a security incident as a planned change. Revoke compromised authority according to severity, disable exposed workflows and use a known-clean issuance path. Notify relevant owners about partial service disruption. A temporary loss of automation can be a successful containment choice when the alternative is leaving an attacker a usable credential.
Problem E: no one can say whether a key is still used. The organisation lacks an inventory and version observability, so it cannot safely retire the old secret or demonstrate closure. Add opaque version labels to consumer telemetry, reconcile active jobs, decommission orphaned integrations and nominate owners. Do not solve the visibility problem by printing actual tokens into application logs. Test detection by deliberately leaving a harmless development consumer on an old version and confirming that the retirement gate refuses to close.
11. Transfer exercise and solution
Transfer the lesson to a fictional school payments provider that permits only one active API key at a time. Forty-eight separately authorised receipt updates are queued. Twelve have already been dispatched when the old key is revoked. Ten have trusted success receipts; two timed out and remain uncertain. Thirty-six have not been dispatched. The new key can be introduced only after the old one stops working. What must the operator do before issuing any remaining updates, and how many updates need new execution if reconciliation finds that one timed-out operation committed and the other did not?
Solution: stop the outbound queue, preserve all forty-eight intention IDs and original authorisations, and reconcile the two unknown operations against the payment system by transaction reference. The ten acknowledged effects plus one reconciled effect make eleven confirmed completions. The other timed-out attempt did not commit, so the remaining unperformed group is thirty-six never-dispatched updates plus that one absent effect, giving thirty-seven. Execute those thirty-seven through the new valid key only after permission recheck and duplicate-prevention controls. The accounting is 11 completed + 37 remaining = 48, with no second execution of an already committed payment.
Change one fact: the provider cannot query a receipt by stable reference. The numerical answer is no longer operationally sufficient. The two uncertain writes cannot be safely retried without independent reconciliation, so a human-controlled exception path must resolve them. The system may proceed with genuinely untouched queue items under the new key, if policy allows separation, while holding ambiguous ones. The lesson transfers from credential rotation into transaction consistency: changing access does not turn uncertainty about side effects into a safe retry.
A second change tests emergency logic. If the old key is confirmed leaked before those twelve dispatched updates finish, do not maintain a transition period merely to preserve throughput. Revoke, contain, determine whether the received receipts remain trustworthy, and carry forward only authorisations that remain valid. The exact number of successful operations will then depend on independent evidence, not the arithmetic from a planned maintenance window. This is why a runbook must contain decision branches instead of a single cheerful checklist.
12. Limits, trade-offs and governance
Shorter credential lifetimes can limit misuse but increase refresh traffic, coordination complexity and the chance of outages when the issuer fails. Long-lived keys reduce renewal overhead but increase the usefulness of an exposed copy. Dynamic workload identity can simplify rotation for some architectures, but federation, trust bundles and issuer availability create their own dependencies. The safe choice depends on workload duration, consequence of compromise, incident speed, provider behaviour and recovery capability.
No credential technology prevents a properly authenticated system from making an unauthorised business decision if the application layer ignores user permissions. Similarly, a valid certificate says something about identity within a trust framework, not about whether a generated answer is true. Treat authentication, authorisation, content verification, human approval and side-effect confirmation as separate tests. A strong design preserves each boundary even when they share an incident timeline.
Operations teams need a defined rotation owner, key-issuing authority, security reviewer, on-call responder and a tested way to stop actions. Training should include provider-specific runbooks and the ability to distinguish 401, 403, expiry, timeout and downstream business rejection. Finally, avoid using a futuristic label to excuse weak controls today. Current AI agents are limited software systems with fallible outputs and tools. Hypothetical artificial superintelligence is not a demonstrated credential-management product.
Frequently asked questions
Is rotation just changing an API key?
No. Replacement includes updating all intended consumers, confirming required access, retiring the old authority and testing that important work remains correct. The issuance of a new token is only one step in that process.
What is the difference between expiry and revocation?
Expiry follows a validity rule, while revocation is an explicit decision to terminate authority before or regardless of a planned expiry. Real enforcement and propagation behaviour depend on the issuer and consuming service. Neither should be assumed instantaneous in every distributed system.
Should an assistant read the credential to use a tool?
Normally no. Store and retrieve secrets in a controlled execution layer, pass only the narrowly scoped result to the model, and avoid including passwords or keys in prompts, examples, debugging output and user-facing messages.
Can we roll back to the old credential if the new one fails?
Only where the old authority remains legitimate and uncompromised, the issuer still permits it and the rollback is an approved part of the scheduled change. An exposed key is not a safe fallback merely because it restores service.
How frequently should keys rotate?
Use the issuer’s requirements and your risk assessment: type of credential, exposure consequences, revocation mechanism, job duration and recovery capability. Do not copy a single rotation interval from another product without checking these factors.
Why independently verify a rotation if the dashboard is green?
Because configuration changes and successful deployments do not prove that a forgotten worker migrated, a token has the right scope, or the retired key no longer authenticates. Verify real authorised calls and the issuer’s revocation state.
Official standards and further reading
- OWASP Secrets Management Cheat Sheet: secret lifecycle and defensive handling.
- NIST SP 800-57 Part 1 Revision 5 (final): cryptographic key-management principles.
- IETF RFC 9700: OAuth 2.0 security best current practice, published January 2025.
- SPIFFE Workload SVID documentation: workload identity and short-lived certificates.
- NIST AI RMF 1.0: documented AI risk governance and evaluation.
Reader pathways
For software engineering, link this chapter to idempotency and safe retries. For risk and operations, continue with disaster recovery and change control. For a wider learning map, explore the How X Works hub and Gold Standard of Education. In everyday work, the transferable lesson is to keep capability and authority distinct, then verify the real effect rather than celebrating the installation of a new mechanism.