How Agentic AI Governance That Tries to Cover Everything and Ends Up Governing Nothing

Every company talking about agentic AI governance faces the same trap: defining the category so broadly that the governance framework becomes impossible to implement, at which point teams start routing around it entirely. The umbrella definition of agentic AI covers anything with multiple steps and a degree of autonomy. That includes everything from conditional logic workflows to fully autonomous systems that can commit real-world consequences. Trying to govern the whole spectrum with one framework produces a structure that either blocks everything or governs nothing.

Matt Kelly is a Partner at Simpson Thacher & Bartlett LLP, where he leads the firm's AI practice and is a member of the Privacy and Cybersecurity group. He has spent more than 15 years advising clients on technology governance, regulatory investigations, cybersecurity incidents, and technology-driven transactions across financial services, healthcare, private equity, and Fortune 500 companies. He approaches agentic AI governance from the position that precision matters more than comprehensiveness.

"Don't try to govern all of AI. Don't try to govern all of agentic AI," Kelly says. "Start by trying to govern the thing you're actually worried about and the risk you're trying to prevent. Because then you can actually identify the risks and address them."

Most agents are still macros

Kelly pushes back on the market narrative that enterprises are running thousands of autonomous agents. "I've yet to see a single company for whom I think that's really true," he says. "Or at least for whom that's true and it's efficient." Inventory what an organization has actually deployed, and the result is typically a small number of conditional logic tools: check this, if this, then do that. "They're not yet really autonomously operating in a space making decisions that are broader than one might expect," Kelly says.

The pressure to deploy anyway comes from an echo chamber where the loudest voices claim transformative scale, pushing companies forward faster than they can build the platforms to support it. Kelly sees the same expectation gap cascading through every layer of the organization.

"Boards expect faster AI transformation than management thinks is currently possible. Management turns to employees and says you should be getting everything done. The employees say, we can't get everything done." Every level is operating against tech debt and unrealistic timelines.

Liability follows the company externally and the deployer internally

Externally, any effort to disclaim responsibility for what a company causes a computer to do will likely collapse. Kelly points to financial markets as precedent: autonomous systematic trading has operated without human intervention for two decades, and both sides accept responsibility for the behavior of the systems they deploy.

Internally, agents that are user-triggered and attributable to a specific account should be owned by the user who created and launched them. Background autonomous processes that are not tied to any individual user should be owned by the team that deployed them. "The owner of that decision and the owner of the setup really should be the person who is accountable for that outcome," Kelly says. Where neither owner is clear, governance has already failed.

Every agent should be designed to shut down safely

Kelly's advice to in-house practitioners starts with cross-functional collaboration. Legal, security, and technology teams need a shared operating understanding, not email handoffs. He notes that security ops and CTO teams are not always aligned on agentic AI, and legal teams should be aware of that gap and work to bridge it.

The design principle that matters most is safe shutdown. "If you have something so integral and critical to your business operations that shutting it down causes just as much harm as it going wrong in the first place, it's probably a design problem," Kelly says. "Ideally, you should be able to pull the plug on any agent operating in your environment without causing more harm from that decision."

On the regulatory side, Kelly counsels restraint. Most real enforcement exposure today comes from long-standing, tech-neutral laws that companies would have seen coming regardless of AI. "You could run yourself ragged trying to look at every regulation that's been proposed," he says. "Very few wind up being law. Focus on the operational risk. If you have a clear sense of what that looks like, you will be very well positioned to manage the regulatory risk that does come into play."