How Agentic AI Governance That Tries to Cover Everything Ends Up Governing Nothing

Every company talking about agentic AI governance faces the same trap: defining the category so broadly that the governance framework becomes impossible to implement, at which point teams start routing around it entirely. The umbrella definition of agentic AI covers anything with multiple steps and a degree of autonomy. That includes everything from conditional logic workflows to fully autonomous systems that can commit real-world consequences. Trying to govern the whole spectrum with one framework produces a structure that either blocks everything or governs nothing.

Matt Kelly is a Partner at Simpson Thacher & Bartlett LLP, where he leads the firm's AI practice and is a member of the Privacy and Cybersecurity group. He has spent more than 15 years advising clients on technology governance, regulatory investigations, cybersecurity incidents, and technology-driven transactions across financial services, healthcare, private equity, and Fortune 500 companies. He approaches agentic AI governance from the position that practical impact matters more than comprehensive coverage.

"Don't try to govern all of AI. Don't try to govern all of agentic AI," Kelly says. "Start by trying to govern the thing you're actually worried about and the risk you're trying to prevent. Because then you can actually identify the risks and address them."

Most agents are still macros

Kelly pushes back on the market narrative that enterprises are running thousands of unique autonomous agents. "I've yet to see a single company for whom I think that's really true," he says. "Or at least for whom that's true and it's efficient." Inventory what an organization has actually deployed, and the result is typically either a small number of complex agentic tools, or a large number of search-and-look-up features that apply conditional logic: check this, if this, then do that. "They're not yet really autonomously operating to make the type or volume of decisions one would expect based on the headlines," Kelly says.

The pressure to deploy anyway comes from an echo chamber where the loudest voices claim transformative scale, pushing companies forward faster than they can build the platforms to support it. Kelly sees the same expectation gap cascading through every layer of the organization.

"Boards expect faster AI transformation than management thinks is currently possible. Management expects the workforce to get more done with less, and to do it faster, too. And employees turn back and say that they just can't get everything done." Every level is operating against tech debt and unrealistic timelines.

Liability follows the company externally and the deployer internally

Externally, any effort to disclaim responsibility for what a company causes a computer to do will likely collapse. Kelly points to financial markets as precedent: autonomous systematic trading has operated without human intervention for decades, and both sides generally accept responsibility for the behavior of the systems they deploy.

Internally, agents that are user-triggered and attributable to a specific account should be owned by the user who created and launched them. Background autonomous processes that are not tied to any individual user are harder to govern, but a good starting point is to assume they should be owned by the team that deployed them. "The owner of the decision to release something in the wild and the owners of the setup really should be the ones accountable for the outcomes," Kelly says. Where those owners are unclear, governance has already failed.

Every agent should be designed to shut down safely

Kelly's advice to in-house practitioners starts with cross-functional collaboration. Legal, security, technology, and business teams need a shared operating understanding, not just email handoffs. He notes that security ops, CTO teams, and business leads are not always aligned on their expectations for agentic AI or their comfort with deploying agents in the enterprise environment. Legal teams should be aware of that gap and work to bridge it.

A key design principle for agents is safe shutdown. "If you have something so integral and critical to your business operations that shutting it down creates as many problems as it solves, it's probably a design problem," Kelly says. "Ideally, you should be able to pull the plug on any agent operating in your environment without causing additional harm as a result of that decision."

On the regulatory side, Kelly counsels restraint. Most real enforcement exposure today comes from long-standing, tech-neutral laws that companies would have seen coming regardless of AI. "You could run yourself ragged trying to look at every AI regulation that's been proposed or that will be proposed in the months ahead," he says. "Very few of those will ever wind up being law. And the ones that do will often be unrecognizable. Focus on the operational risk. If you have a clear sense of what that looks like, and if you make a good faith effort to limit the sources of operational risk, you will be very well positioned to manage any regulatory regimes that do come into play."