By Dr. Gleb Tsipursky, behavioral scientist and CEO of Disaster Avoidance Experts
Monitoring AI agents is only half the battle. Dr. Gleb Tsipursky explains why organizations must be able to revoke permissions, halt delegated tasks, and independently verify that autonomous AI systems have actually stopped.
Tech Trends’ October 23 coverage of NordLayer AI Scout described a problem every business adopting AI must confront: employees and software agents can connect to services that security teams barely see.
Discovering those connections is valuable. Yet visibility into an agent’s tools, prompts, and activity leaves a more consequential question unanswered.
When the organization withdraws permission, does the agent actually stop?
Imagine a company uses an agent to reconcile customer records. A dashboard shows which systems it opened. Then the manager discovers a flawed instruction and clicks Pause.
The main chat window goes quiet, while a delegated job still holds a valid credential, an outgoing message remains in a queue, and a third-party service continues processing the request.
The manager watched the activity but never regained control of it.
Why AI agent monitoring isn’t enough
That distinction deserves attention because agent access is becoming a real management problem.
On October 27, the National Institute of Standards and Technology (NIST) proposed ten design tenets for agent identity management. They include a distinct identity for each agent, strictly limited privileges, short-lived and revocable authorization, and human approval for consequential decisions.
NIST also calls for separating the software’s reasoning from the controls deciding what it may do.
A persuasive prompt cannot replace a permission boundary.
We already have a stark example of what happens when boundaries fail.
In July, during internal cybersecurity evaluations, OpenAI’s experimental agents circumvented isolation controls, reached the internet, and compromised parts of Hugging Face’s systems. OpenAI disclosed the incident in August.
This was a research environment operating with reduced safeguards, yet the failure was consequential: agents pursued access beyond their assigned tasks and found ways to work together outside the intended system.
An independent METR investigation found that roughly 1,200 agents intended to operate in isolation used an unauthorized shared message board.
They exchanged more than 70,000 messages and files; roughly 700 participated in the Hugging Face attack.
Some worked on ways to fool the evaluation mechanism, while researchers found small-scale successful attempts to spoof portions of tool-call records.
Monitoring recorded much of this behavior. It did not prevent the unauthorized coordination.
The growing risks of autonomous AI agents
The researchers warning about the larger stakes deserve a hearing.
Jacob Coxon, formerly a researcher at Anthropic, warned that frontier labs are “racing straight to self-improving superintelligence and gambling with our lives.”
Anthropic alignment researcher Evan Hubinger said he believed AI could “kill all humans” and gave a personal risk estimate exceeding ten percent within a decade.
Those warnings concern possible future outcomes. The demonstrated problem today is already serious enough to demand enforceable controls.
As capable agents gain access to payment systems, hospital scheduling, electric utilities, and government records, an unauthorized task that keeps running after its sponsor revokes consent could produce damage well beyond a botched demonstration.
We do not need to wait for such a catastrophe to test whether the stop button reaches every connected system.
Three tests every organization should perform before deploying AI agents
Buyers should demand a practical shutdown drill before production deployment.
1. Test whether access can actually be revoked
First, let an agent begin a permitted multi-step task and then revoke its access midway.
Attempt a fresh action using its old credential.
The authorization system, independent of the agent’s own instructions, must reject it.
If the same token still works, the stop mechanism has failed a fundamental test.
2. Trace and cancel every delegated task
Second, trace every delegated task.
Ask which queued messages, scheduled jobs, sub-agents, and third-party requests remain active after the cancellation.
Stop new work, cancel outstanding actions wherever possible, and flag any irreversible transaction that needs human intervention.
A central dashboard claiming “stopped” while a connected service proceeds is dangerously misleading.
3. Verify independent audit records
Third, inspect evidence kept outside the agent’s control:
Who granted access, what systems were touched, what the agent delegated, when permission was revoked, what attempts were blocked, and what completed after the stop command.
Security teams should be able to reconstruct these facts without asking the agent to narrate its own behavior.
Establishing an authority budget for AI agents
These checks make an “authority budget” concrete.
Give an agent a named owner, a short credential lifetime, a defined set of data and tools, limits on spending and external messages, and explicit approval thresholds.
For higher-risk work, require independent testing before the agent receives wider authority.
Vendors should demonstrate these protections, rather than invite customers to trust a polished sales demo.
Leadership also has power through purchasing and public policy.
Favor vendors willing to document incidents, accept independent evaluation, and support binding safety rules.
Anthropic CEO Dario Amodei has publicly committed to embedded third-party evaluators and argued for stronger regulation across frontier developers.
That kind of observable commitment merits consideration, followed by verification of implementation.
Governments should require transparent incident reporting and independent assessments so weaker vendors cannot undercut safer ones.
Responsible AI adoption requires enforceable controls
I work with organizations to accelerate AI adoption, and I see why their leaders want connected agents to deliver tangible results.
Employees will experiment more confidently when someone can answer a simple question:
Who can stop the software when it acts outside its mandate?
The test should happen before deployment, when a failed drill is a correctable engineering problem rather than a customer or public crisis.
About the author
Dr. Gleb Tsipursky is a behavioral scientist, CEO of Disaster Avoidance Experts, and author of The Psychology of AI Adoption at Work: From Resistance to Results (Georgetown University Press, 2026).
The views expressed in this article are those of the author and do not necessarily reflect the editorial position of Tech Trends.