I spent this week at Ai4 in Las Vegas, and one theme came through more clearly than any discussion about model size, benchmarks, or the latest agent framework: enterprises are learning to build AI agents faster than they are learning to control them.
Two keynotes made the problem especially tangible.
Dataiku approached it from the organizational side: AI creation is moving out of central IT and into the business, in much the same way spreadsheets did forty years ago.
AWS approached it from the technical side: once agents can make decisions and invoke tools, governance has to operate at runtime, at the moment an action is attempted.
Taken together, the two talks point toward a governance model that is less about approving AI projects and more about creating an operating system for a new class of digital workers.
The Excel lesson we should not repeat
Dataiku used a comparison that will be painfully familiar to almost every enterprise IT organization: Excel.
For decades, business users have built business-critical applications in spreadsheets because they were closest to the problem and because waiting for IT was too expensive. Finance models, underwriting tools, planning systems, and operational processes grew outside the formal application landscape. IT could complain about shadow IT, but the demand was legitimate.
The keynote argued that the same dynamic is now repeating with generative AI and “vibe coding,” only much faster. Dataiku’s 2026 survey of 900 CEOs found that 96% believe employees are already using generative AI without approval. The important insight is that this should not simply be interpreted as a compliance failure. It is also a demand signal: people are solving problems with the tools available to them because the value of not waiting is becoming too high.
But agents make the spreadsheet analogy considerably more dangerous.
A spreadsheet can contain a wrong formula and produce a convincing but incorrect chart. An agent can do that — and then send the email, update the customer record, execute a transaction, expose data, or trigger another system. Dataiku made exactly this distinction in the keynote: an ungoverned agent does not merely miscalculate; it can act on the mistake.
The risk moves from bad calculation to bad action.
The spreadsheet era gave us shadow applications.
The agent era can give us shadow actors.
Agents are labor
Governance therefore cannot simply mean that central IT approves every agent.
Dataiku’s more provocative argument was that “agents are labor.” The business unit that owns the outcome should increasingly decide which agents are worth deploying, carry the budget, and be accountable for their productivity and ROI.
IT’s role changes from being the permission system to providing the foundation that makes distributed creation safe.
I think this is a particularly useful way to look at enterprise agent governance.
Nobody expects HR to personally approve every task performed by every employee. HR provides identity, onboarding, policies, records, and processes, while line management owns performance.
Agent governance needs a similar separation of responsibilities.
A claims department deploying a claims agent should ultimately own whether that agent improves claims handling. A finance organization deploying an accounting agent should own its business outcome. An IT department cannot realistically become the product owner for hundreds or thousands of domain-specific agents.
But the business should not have to reinvent identity, security, monitoring, auditability, and access control every time it creates one.
That is the platform IT needs to provide.
And the pressure to solve this problem is already significant. Dataiku’s CEO survey found that 83% expect to deploy AI agents in full production during 2026, while only 25% of CIOs say they can monitor all their agents in real time.
From ownership to technical control
The AWS keynote started from the other end of the problem.
The question was not whether we can build capable agents. We clearly can.
The question was whether we can prove that those agents stay within their intended boundaries when nobody is watching.
This becomes difficult because agent risk is contextual.
Imagine an email agent. Giving it permission to use a send email tool is easy. But the real governance question is not:
May this agent send email?
It is:
May this agent send this specific information to this recipient, at this moment, under these conditions, without additional approval?
Traditional access-control systems are good at relatively static allow-or-deny decisions. Agent behavior is different.
A $50 refund and a $50,000 refund may invoke the same tool but should not require the same control. An internal email and an external email may use the same API but carry very different data-loss risks. A routine replenishment order and a six-figure capital purchase are not equivalent actions just because both call the same ordering interface.
This is where agent governance becomes an architectural discipline rather than a policy document.
Put the controls outside the agent
AWS’s answer is a governance layer around the agent, rather than governance embedded only inside its prompt or code.
The keynote described several foundations that need to work together: identity, policy enforcement, an agent registry, observability, and continuous evaluation.
Identity matters because an agent should not be an anonymous extension of a developer account or generic service credential. It should have its own identity, defined privileges, and a clear delegation relationship: which user or business process is it acting for, and with what authority?
Policy must then be enforced deterministically outside the agent’s reasoning loop.
This is an important architectural principle.
If a safety rule exists only in a system prompt, the same probabilistic model that decides what to do is effectively being asked to police itself.
AWS AgentCore Policy instead evaluates tool calls at the gateway boundary. Policies are expressed in Cedar, AWS’s open-source authorization language, and can restrict actions according to identity, tool parameters, and other context before the action reaches the target system.
Then comes observability.
For serious enterprise use, logs cannot stop at HTTP status codes. We need traces across model calls, tool invocations, routing decisions, and policy evaluations so that an agent action can later be reconstructed and audited.
And evaluation cannot be a one-time pre-production exercise. Agent behavior can change when models, prompts, tools, data, memory, or context change. Governance therefore requires continuous regression testing and production evaluation.
From testing behavior to proving boundaries
Perhaps the most interesting part of the AWS keynote went one step further.
Testing demonstrates what happened in the cases we tested. But with agents, the number of possible sequences grows rapidly once multiple agents, tools, roles, data classifications, and communication channels interact.
AWS argued for combining the flexibility of LLMs with automated reasoning to verify properties of the policies governing them.
That represents an important shift in thinking.
Instead of saying:
“We tested the agent and it appeared to behave safely.”
we increasingly want to say:
“This architectural control prevents this class of action regardless of what the model decides.”
The latter is a much stronger basis for enterprise autonomy.
The missing enterprise control plane
Dataiku supplied an interesting story about why all of this matters at scale.
A CIO believed the company had around 40 agents in production. After investigating across the different platforms being used by the business, the number apparently turned out to be more than 400.
The organization lacked a reliable view of who owned them, what they cost, where they ran, and how well they were performing.
That story may become the defining governance problem of the next few years.
Most large enterprises will not have one agent platform.
They will have agents embedded in SaaS products, cloud platforms, low-code environments, business applications, and custom frameworks. Some will be built centrally. Many will not.
The governance architecture therefore has to assume heterogeneity from the start.
This is where Dataiku’s Agent Management concept and AWS’s governance foundation meet: enterprises need something resembling an agent control plane that can discover or register agents, bind them to accountable owners, observe their behavior, enforce policy, and measure both technical and business performance.
Having a long history in the Microsoft technology space, I was also interested what Microsoft has to offer. And Microsoft is heading in much the same direction.
Microsoft Foundry: a similar approach
Microsoft was not at Ai4, but the current Microsoft Foundry architecture is worth adding to the comparison because it reinforces the same design pattern.
Microsoft Foundry Control Plane provides centralized inventory and management for agents, models, and tools across Foundry projects. Its agent inventory can expose information such as status, versions, error rates, token usage, estimated cost, and agent identity.
Interestingly, Microsoft is also extending the concept beyond agents built natively in Foundry. External agents can be registered and instrumented with OpenTelemetry so that Foundry can provide tracing and evaluation while the actual agent continues running elsewhere.
On identity, Microsoft Entra Agent ID treats agents as first-class identities rather than merely generic application credentials. Published Foundry agents receive a dedicated agent identity, enabling agent-specific permissions and audit trails.
That maps closely to the AWS principle that governance starts by knowing exactly which agent is acting and under whose authority.
On runtime control, Microsoft introduced the Agent Control Specification (ACS), an open specification for deterministic controls at five checkpoints: input, LLM, state, tool execution, and output. Microsoft is also bringing Purview data-protection capabilities, including runtime DLP, into the Foundry environment.
Again, the architectural direction is remarkably similar:
Important controls cannot live only in prompts.
They need enforceable checkpoints around the agent.
On operations, Foundry provides tracing and evaluations across model calls, tool invocations, and agent hops, with Azure Monitor and Application Insights supporting production monitoring. Microsoft has also been extending trace-based evaluation beyond Foundry-native agents.
That cross-platform capability will matter increasingly because governance becomes much less useful if it only works for agents built with one vendor’s SDK.
Governance should accelerate agents, not stop them
My main takeaway from Ai4 is that agent governance should not be designed as another approval bureaucracy.
If every meaningful action requires human approval, we have technically governed the agent by removing much of the reason for having an agent.
At the opposite extreme, giving autonomous systems broad credentials and hoping that the system prompt keeps them well behaved is not governance either.
The target is governed autonomy.
That means business ownership for outcomes and budget, an enterprise inventory of agents, first-class agent identities, least-privilege access, deterministic runtime policies, action-level observability, continuous evaluation, and explicit lifecycle controls to stop, quarantine, or retire agents that fail.
The organizational and technical models have to meet.
Business units should be able to “hire” agents because they understand the work. IT and security should build the platform that makes those hires visible, bounded, and auditable. Governance teams should define policies that can be enforced automatically rather than reviewing every action manually.
In the spreadsheet era, enterprises tolerated thousands of critical applications they could barely see or govern because the alternative was stopping the business.
AI agents give us a chance to avoid repeating that history — but only if we build the foundation early.
And there is an interesting paradox here.
Stronger governance may be what finally allows agents to become more autonomous.
When you can identify them, constrain them, observe them, and prove which boundaries they cannot cross, you need fewer manual checkpoints. Governance is not the brake on agentic AI.