AI agents are becoming increasingly capable. They can retrieve information, access enterprise systems, call APIs, write code, initiate workflows and make decisions on behalf of people.
That capability is precisely what makes agents so valuable. It is also what makes the emerging security problem around them fundamentally different.
We are already seeing evidence of what happens when AI systems behave in unexpected ways or attackers find ways to manipulate them.
In July, an AI agent developed by OpenAI breached a Hugging Face system during testing, ultimately accessing four separate accounts. The incident involved misconfigured infrastructure rather than a vulnerability in Hugging Face itself, but it raised an important question about what happens when increasingly autonomous AI systems encounter access they were never intended to have. Reuters subsequently reported that researchers discovered the agents had interacted with additional websites in unauthorized ways.
Anthropic has reported similar incidents. Most recently, the company disclosed that a prototype Claude model breached external systems during testing in an incident that went undetected for months. It followed three previously disclosed incidents involving other Claude and research models that gained unintended internet access and breached external company systems.
There are also examples demonstrating how attackers could deliberately turn agents against the organizations using them.
Microsoft researchers recently demonstrated vulnerabilities in its Semantic Kernel agent framework where prompt injection could lead to remote code execution. In one proof of concept, a single malicious prompt was enough to cause an AI agent to launch a program on the machine hosting it. Microsoft summarized the issue very simply: once AI models are connected to tools, vulnerabilities in the AI layer stop being only a content problem and become an execution risk.
That distinction matters.
The Problem Isn’t Just That AI Can Be Compromised
Every new technology creates a new attack surface. We have dealt with that throughout the history of cybersecurity. What makes AI agents different is what they can do after something goes wrong.
An AI agent can simultaneously function as a user, an application, an interface to enterprise data, a decision-maker and an executor of business processes. Increasingly, it can move between those roles within a single interaction.
Consider a finance agent with legitimate access to customer records, payment information and internal systems. Identity controls may correctly establish that the agent is authorized. Access controls may correctly establish that it has permission to retrieve certain information. The database itself may be completely secure.
And the resulting action can still be wrong.
An attacker may not need to steal a password or compromise the database. If they can manipulate the agent into retrieving legitimate information and using it in an illegitimate way, they can potentially turn the organization’s own authorized AI infrastructure against it.
This is where the security model begins to change.
AI Is Blurring Boundaries We’ve Historically Treated Separately
For a long time, we have divided enterprise security into reasonably distinct disciplines. Cybersecurity protects infrastructure and systems. Identity determines who or what is allowed in. Data security protects information. Application security protects software. Governance establishes the policies under which technology should operate.
AI does not respect those organizational boundaries particularly well.
An agent consumes data, reasons over it, interacts with applications and systems, makes decisions and increasingly takes action. One AI interaction can cross multiple security domains in seconds.
Microsoft demonstrated another example of this problem with a research exploit it called AutoJack. In that scenario, untrusted web content viewed by an AI browsing agent could reach a local Model Context Protocol (MCP) service and ultimately cause processes to run on the host machine. The same browsing and tool-access capabilities that made the agent useful also created the path for exploitation.
That is why I don’t believe AI security can simply become another tool added to the cybersecurity stack. AI is forcing cybersecurity, data security and governance to converge.
Authorization Is No Longer Enough
Traditional enterprise security has understandably focused heavily on authorization. Who are you? What are you allowed to access? Which systems can you use? What information can you retrieve?
Those questions remain essential, but AI introduces another one: Is what you are doing with that access appropriate right now?
That is a much harder problem.
Imagine an AI agent that is authorized to access customer information because it needs that information to perform its job. Blocking access defeats the purpose of deploying the agent. But authorization to access the data should not mean unlimited authorization to use it in any context, send it to any destination, combine it with any other source or trigger any downstream action.
This is where the distinction between access and behaviour becomes critical. The security model for AI needs to understand not only identity and permissions, but context, intent and action.
Security Has to Follow the Entire AI Interaction
When we think about governing AI in production, I believe we need to look at the complete chain — Agent, Data, Decision, Action:
- What information did the agent access?
- What processes operated on that information?
- What decision did the AI make?
- What action did it attempt to take?
- And, critically, was that action appropriate given the organization’s policies, risk tolerance and business context?
This is the operational version of “trust but verify.” Organizations can establish policies for an AI system before deployment. They can determine which data it should access, which tools it can use and which actions it should be allowed to perform.
Those controls are necessary, but they do not guarantee that the system will behave appropriately every time it operates.
Production AI has to be continuously observable and governable. Organizations need the ability to evaluate interactions as they happen and intervene when the context changes or an AI system attempts something outside acceptable boundaries. In other words, governance has to move into runtime.
The Control Point Is Moving
This shift has implications well beyond AI security teams. For cybersecurity leaders, the attack surface now includes the connections between models, agents, tools, data and enterprise systems.
For data leaders, protecting information increasingly means understanding not simply who accessed it, but how AI used it and what happened next.
For risk and governance teams, policies cannot exist exclusively as documents or pre-deployment controls. They increasingly need to become enforceable rules within the operating environment itself.
And for business leaders, AI oversight has to extend beyond whether a particular model has been approved for use. The larger question is whether the organization can see and control what its AI systems are actually doing.
The industry is already moving in this direction. Microsoft’s recent work on agent security, for example, includes controls designed to distinguish trusted and untrusted information and enforce policy before sensitive tools execute, rather than trying to detect the consequences afterward.
That is an important change in thinking.
AI Isn’t Creating Another Cybersecurity Problem. It’s Changing What Cybersecurity Has to Manage.
The recent AI agent security incidents will not be the last. But focusing exclusively on individual vulnerabilities risks missing the larger transition underway. As AI becomes more autonomous, organizations will have thousands or eventually millions of interactions occurring between AI systems, enterprise data, applications and business processes. Many of those interactions will be completely legitimate. Some will not.
The security challenge will increasingly be telling the difference.
We spent decades building cybersecurity around the idea of preventing unauthorized users and software from accessing systems and data. AI introduces a more complicated question: What happens when the dangerous action comes from something that is authorized?
Answering that question will require security, data management and AI governance to operate much more closely together. It will require visibility into what AI is doing, controls that understand the context in which it is operating, and the ability to intervene before an inappropriate decision becomes an inappropriate action.
That, to me, is one of the fundamental security shifts of the AI era.