The AI Control Gap
Accountability arrived before the ability to see, constrain, and stop the agent.
An AI agent is not under control because a policy exists. It is under control when the business can prove what it did, why it was allowed, and how to stop it.
2 in 3
leaders report the gap
54
agent incidents, annual average
17%
of incidents are high severity
The AI control gap is what happens when someone owns the outcome, but nobody can reliably govern the system producing it.
That distinction matters because enterprise AI has moved beyond a model that drafts a paragraph or classifies a ticket. An agent can select tools, pass information between systems, trigger a workflow, modify a record, and continue after the first action. It can carry an assumption from one step into the next. A bad assumption is no longer just a bad answer. It can become a sequence of otherwise valid actions, each one making the earlier mistake harder to unwind.
IBM's June 2026 study of 2,000 C-level technology executives across 33 countries gives the problem a name. Two-thirds said their organizations face a growing gap between who is accountable for AI and who has control over it. Seventy percent said the business is deploying technology faster than IT can track. Only 11% reported being ready to scale AI. These are not objections to AI adoption. They are an operating warning: the organization has begun to delegate work before it has built the means to supervise delegated work.
A useful definition is deliberately plain. Control means the business can identify the agent, limit what it is allowed to access, reconstruct a material decision, test its behavior under pressure, and contain or stop it when the context changes. None of those capabilities is exotic. They are familiar in security, finance, and operations. What is new is the need to apply them to a system that can decide which action comes next.
That is why this is a runtime problem, not a policy-writing problem. A policy can say an agent may assist a support team. Runtime control determines whether the particular agent that opened a customer record had authority to do so, which tools it used, what information it drew on, what changed after it acted, and who could intervene before the error traveled farther. Governance defines the obligation. The control plane gives that obligation a working mechanism.
When an incident becomes a sequence.
The danger is not only that an agent produces a wrong answer. It is that the wrong answer becomes a trigger. An agent can misread a request, choose the wrong tool, act with more authority than the task needs, and then create a record that makes the next system treat the first error as fact. IBM reports an average of 54 agent incidents a year among organizations studied, with 17% classed as high severity. Of the severe incidents, the study identifies data exposure, cascading behavior, and compliance failures as recurring outcomes.
Consider a customer-service agent with access to account data, a refund tool, and an internal knowledge base. A malicious or misleading request can change its interpretation of the case. If the agent retrieves the wrong policy, proposes an exception, writes that exception back to the case record, and triggers a refund workflow, a reviewer now has to inspect a chain rather than a single response. The key question is not whether a human could have spotted the first mistake in a transcript. It is whether the system had enough controls to prevent that mistake from becoming the next action.
Microsoft's AI Red Team research frames the same issue through agent failure modes: an agent's identity, instructions, tools, memories, and surrounding environment all become part of the attack and failure surface. This is why a conventional application log is not enough. A timestamp proving that an API call happened does not tell an investigator whether the agent was allowed to make it, whether its authority had expired, or whether a person had a chance to stop the sequence.
What runtime control actually means.
A runtime control plane is not a dashboard with more charts. It is the set of controls that make an agent's authority inspectable while it is working. Start with identity. The company should be able to tell one agent from another, know which human or business process authorized it, and revoke it without guessing which workflow will break. Workday's Agent Passport is one current example of this direction: testing, verifying, and continuously monitoring agents rather than treating their identity as an implementation detail. Its Agent System of Record makes the operational claim explicit, agents need a place where identity, permissions, provenance, and activity can be managed together.
Next comes authority. A support agent does not need the same ability to write data as a finance agent. A finance agent should not inherit the ability to touch every ledger simply because it was assigned a broad business objective. Permission needs to be specific, scoped, and revocable. It also needs to travel with the agent when that agent uses a third-party service or another agent. Cisco's enterprise agent security guidance makes this practical: a platform needs controls for identities, tool calls, data movement, and policies, not a single security review before deployment.
Then comes observability with meaning. The evidence trail for a material action should preserve inputs, relevant state, selected tools, permissions used, action output, and the result. It should make replay and review possible without exposing every sensitive token or customer datum to everyone. This is the same discipline explored in the AI audit-trail liability problem. A trace is useful when it lets an accountable person answer a decision question, not merely when it records that an application emitted an event.
Finally, a company needs evaluation and containment that survive launch. Evaluation has to be ongoing because models, tools, data sources, and attack patterns change. Containment has to be more than an emergency note in a runbook. It needs a tested pause, an approval boundary for material actions, a safe fallback, and a named decision owner. The agent that cannot be paused without improvisation is not operating inside a meaningful control system.
The control plane does not have to be a single product. It can combine an identity provider, a policy engine, a gateway for tools, an event store, an evaluation service, and an incident process. What matters is that the chain remains coherent at review time. An operator should not have to search five disconnected systems to answer one basic question about a consequential action. The design should connect the agent's identity to its permission, the permission to the tool call, the tool call to the changed state, and the changed state to the owner responsible for the outcome.
This is also where vendor claims deserve restraint. A platform can make agent deployment faster without automatically making the enterprise safer. The Gravitee State of AI Agent Security report describes how much of the market still lacks consistent visibility and governance for agent activity. That is a useful market signal, not a reason to surrender judgment to another dashboard. The buyer still needs to ask where identity is verified, who grants and revokes authority, which evidence is preserved, and what happens when the control service itself is unavailable.
Build the stop button before the volume.
Teams often frame control as a concession to speed. IBM's study suggests the reverse. Organizations with embedded controls reported 25% fewer agent incidents, 16 times more agents in production, and 18% higher profit margins. The advantage is not that these teams add a compliance ceremony to every task. It is that they create a reliable path for more work to be delegated without making every exception a bespoke investigation.
The strongest question at an agent launch gate is simple: can one named owner explain what this agent may do, what it may not do, what evidence it leaves behind, and how the team can stop it? If the answer takes a cross-functional meeting to discover, the work is not ready for wider authority. That is also why financial control matters. IBM found that organizations with mature financial controls had more agents in production and were more likely to be ready for scale. They could connect the scope of the system to the cost and consequence of keeping it live.
This does not mean every agent needs a human to approve every keystroke. Low-risk work can be automated within narrow bounds. The point is to reserve human judgment for the actions where authority is material, consequences are harder to reverse, or the evidence is too incomplete to trust the model's confidence. A good control design makes that handoff explicit before the incident, rather than inventing it after a customer, regulator, or security team asks what happened.
A 90-day control plan.
Start with one consequential workflow, not the whole agent portfolio. Map the agent's identity, its instructions, data sources, tools, permissions, and named owner. Select a single action that would be costly if it were wrong, such as editing a customer record, issuing a credit, changing a price, or initiating a payment. That action becomes the test case for the entire control plane.
In the first 30 days, narrow permissions and make a decision record mandatory for that action. In the next 30, test failure modes that matter to the workflow: a misleading request, an untrusted tool output, stale data, an out-of-scope action, and a missing approval. In the final 30, rehearse containment. Pause the agent, revoke the credential, roll back or quarantine the affected state, and verify that the owner gets an alert with enough context to decide what happens next. The exercise should be uncomfortable. That is the point of testing a control before an incident tests it publicly.
The work can then compound. Add another workflow only after the first one produces evidence the business can use. Track incomplete traces, over-broad permissions, time to containment, and recurring exception patterns. Connect the program to the broader measurement drift problem and the agent fragmentation problem. Those are not separate governance projects. They are evidence of the same need: automation has to remain legible to the people responsible for its outcome.
Finance should have a view of the program too. IBM reports that AI spend is expected to rise from 15% to 25% of IT budgets between 2025 and 2027 while most organizations still lack live visibility into the spend and usage that produce it. A control system makes this more than a budgeting discussion. It can show what agents exist, what work they are authorized to perform, what failures they generate, and whether the additional authority is earning its cost. That gives an executive team a real decision: expand, narrow, remediate, or stop. Without it, the enterprise is left with adoption metrics and a growing exposure it cannot explain.
FAQs
What is the AI control gap?+
The AI control gap is the distance between being accountable for an AI agent's outcome and having enough operational control to see, constrain, test, trace, and stop that agent in production. It is not a model-quality problem alone. It is a runtime operating problem that becomes visible when an agent can take actions across systems.
Why are AI agents harder to control than a conventional application?+
A conventional application typically executes a bounded, prewritten path. An agent can select tools, form a multi-step plan, use changing context, and make a sequence of actions. A single weak assumption can travel through that sequence. The useful unit of review is therefore the decision path, the authority used, the tools called, the state changed, and the human or automated control that could have stopped it.
What controls should an enterprise require before scaling agents?+
Start with a verifiable agent identity, least-privilege permissions, a named owner, decision and tool-call traces, a tested rollback or kill path, protected approval boundaries, and regular adversarial evaluation. The exact stack differs by risk, but each material action should be attributable, reviewable, and reversible or containable.
Can a company rely on an AI governance policy to close the control gap?+
A policy is necessary, but it cannot act at runtime. It defines who may use an agent and for what purpose. Runtime controls enforce those boundaries when the agent encounters a tool, a data source, a changed condition, or a request outside its authority. Governance sets the rule; the control plane proves and enforces it while work is happening.
How should a company measure whether its controls are working?+
Measure the share of agent actions with complete decision records, the time to detect and contain abnormal behavior, the percentage of agents with scoped and revocable permissions, evaluation coverage for high-risk workflows, and the rate and severity of incidents. A dashboard is not evidence by itself. The measure has to connect to a decision an owner can take.
The agent you cannot stop is not ready to scale.