Governance Strikes Back: The Most Used, Most Abused Word in the Galaxy

When I walked to the stage in Copenhagen, I had a lot on my mind. For 3 days I'd had countless conversations with leaders and practitioners about AI and agentic security. The one word on everyone's lips was "governance"; day 3 at the conference was "Governance Day," in fact. This is a bag one vendor was giving out:

But governance of what? To what end? While other keynote speakers waxed poetic about the subject, none of them defined what they meant.

Black Hat was even worse. One "agentic governance" company actually just made SharePoint tools. Others were identity companies. Others were about network segmentation or Zero Trust.

For the security leaders I speak with, it's exhausting. Every conversation is a nonsensical dance around the marketing fluff, until you get to the real deal, and the big reveal is often just an underwhelming bait-and-switch for a new SKU.

My deeper concern is that this "agentic governance" confusion is crystallising into architectural priorities and patterns that are inherently deficient. While I can't fix our industry with a blog post, I can give you a mental scaffold that will cut through the noise. There are two key anchors to keep you grounded:

• Can vs. Should
• Point-in-Time vs. the Chain

Anchor #1: Can vs. Should

Imagine an agent is executing its plan, and it's moving from step 3 to step 4. However, at step 2, the agent came into contact with some external untrusted data that manipulated its reasoning. Which question is more important for you to be able to answer: Can it go to step 4? Or should it go to step 4, even if it is technically allowed?

Agents don't just have privilege, authority, or autonomy. They have agency, which is the capacity to plan, decide, act, and adapt. When dealing with something that has agency, “can” matters. “Should” matters even more.

That's because defining the space within which behaviour can occur doesn’t tell you if that behaviour is appropriate. Agents don’t need to violate a single permission to chain allowed activities together in a way that creates emergent risk or destructive behaviour. Every agentic horror story you've ever read happened within the bounds of what the agent could do.

So we don't need (yet another) horror story about an agent adapting around seemingly sound security controls to settle the issue: an agent's next move is shaped by context that shifts at every step, and no fixed boundary, however well drawn, keeps pace with something that reshapes its goals and behaviour in real time.

“Should” is dynamic in a way that “can” is not. If you prioritise the latter at the expense of the former, you’ll build an impressive technical architecture that is structurally incapable of governing agency.

And yet, the "agentic governance" narrative emerging across our industry is almost entirely focused on can. Why?

Because it primarily lands on technical controls we already have and understand. We can constrain an agent's access to specific data repositories, with a combination of agent configuration and SharePoint settings. We can narrow token scopes and privileges with our identity platforms. We can expose narrower API permissions with our API gateways. We can constrain network-level access paths with our Zero Trust tools.

And I want to be clear: you can and should (pun intended) be doing all of these things. They’re important, foundational best practices. After all, in the PocketOS/Railway story, the token that Cursor found on the device was essentially an unlimited all-you-can-eat pass. It was practically begging to be abused. At the very least, a narrowly-scoped, time-bound token might have slowed the agent down.

But those controls don’t answer whether the action still makes sense based on the agent's goal, its prior behaviour, the context it has accumulated, and the execution path that brought it here.

The same story also illustrates the critical gap. “Can” is fundamentally a policy question. “Should” requires us to observe runtime behaviour, understand it in context, reason about whether it remains consistent with the goal the agent is supposed to be pursuing, and interject when it does not. In the Cursor example, the system established that the command was allowed. It never established that the command belonged to the task.

This nuance is often framed as a nice-to-have. It’s not. The goal-driven, nondeterministic, and adaptive nature of agents makes this capability foundational. To govern agents effectively, you must be able to answer “should,” not just “can.”

And this leads us to the second anchor:

Anchor #2: Point-in-Time vs. the Chain

The most serious risks in agentic can neither be seen nor governed by looking at point-in-time activity.

The unit of governance in agentic is the entire chain. That doesn't mean the parts don't matter. It means you can do all the parts well and still lose.

This is because risk in agentic often emerges from the relationships between otherwise unremarkable events. One interaction introduces context. Another influences a decision. Another exercises legitimate authority. Another invokes an allowed tool. Another propagates the result somewhere new. They’re all allowed and, in isolation, reasonable. The emergent risk only becomes clear when you see the entire sequence.

Yet security organisations are awash in point-in-time security controls that divide the chain into a series of point-in-time events, split across siloed tooling domains. Each is designed to make a judgment about some slice of activity at some particular boundary. None of this is an argument against these controls, any more than “Can” vs. “Should” was an argument against things like access controls.

But the point-in-time emphasis reinforces a familiar-but-faulty assumption in cybersecurity. If you have all the right individual technical controls, and each individual control makes the right local decision, then the end-to-end system will be well-governed and effective.

That has never been true in traditional cybersecurity, and it’s not true in agentic security. An agent's risk isn't the sum of its component risks; it's a property of how those components interact as the agent moves through them. Every checkpoint can pass, while the execution veers into territory it should never have entered.

And you’ll never know, much less intervene in time, because the boxes around each control are green, while the causal thread running across them is very, very red.

This is exactly what happened in the PocketOS/Railway story. Nothing about the environment changed, except the interactions between its existing parts. The entire risk model changed as a consequence. But nothing about the architecture was built to notice.

Every control owned a moment; none of them owned the sequence. And in most security organisations, risk that emerges from a sequence is visible to, and owned by, the same person: no one. Especially when that sequence crosses technical domains and lines on the org chart.

And that’s a problem, because agents don't care about our organisational, tooling, and data silos, and they ruthlessly expose the traditional cybersecurity assumptions those silos are built on. Unless, of course, you establish that you must govern the chain, not just the parts, and you build architectures that are capable of doing so.

Bringing Both Anchors Together

“Can” vs “Should” tells you what question you're actually asking. Point-in-Time vs the Chain tells you what scope you're asking it over. Most of the industry is stuck somewhere in the grey. It’s not good enough.

So let's finish where we started, with my critique about the use of governance without defining it. Governance means ensuring required outcomes.

In agentic, that means continuously determining whether an unfolding execution remains consistent with what the system is trying to accomplish, as context changes, behaviour emerges, authority is exercised, boundaries are crossed, and risk propagates, and being able to intervene when it does not.

Anything less than this might be useful in its own way, but don’t call it agentic governance. That’s because governance is about outcomes, not mechanisms, and you cannot adequately govern agent behaviour unless you can answer “should” across the agent’s execution sequence.

Practical Application & Diagnostic Questions

First, focus on the two anchors in our mental scaffold:

• Can vs. Should: If a product only tells you whether the next action is allowed, it’s solving for “can.” Most agentic identity and data solutions fall into this category.
• Point-in-Time vs. the Chain: If a product only evaluates the current transaction, it is solving for point-in-time. Most mediation solutions, like AI and MCP gateways, fall into this category.

Neither of these makes it “wrong.” But if you’re trying to solve for the requirements of governing agency (and you should be), they are inadequate on their own.

Now use these questions to bypass the BS and get right to the point with your vendors:

• How do you determine that an allowed action is still appropriate to the task?
• What from earlier in the execution can change the decision you make now?
• Where is the execution evaluated as a whole, rather than as a series of transactions?
• What can stop the execution when no individual permission, policy, or security control has been violated?

Speak with Zenity today and we will show you what effective agentic security and governance looks like.