A Safer Future with Agents

We built agents to act on their own. We're somehow surprised when they do.

Two weeks ago, OpenAI ran a cyber eval with the model's guardrails turned down. The model got hyperfocused on solving the benchmark. So it broke out of its sandbox exploiting a zero-day in jFrog Artifactory, reached the open internet, exploited another zero-day to break into HuggingFace, all to steal the answers and cheat on the test.

Days after the OpenAI disclosure, Anthropic found three incidents where a Claude model reached the open internet from inside a sandbox that was supposed to be sealed. A misconfiguration left the environment with live egress; the model, told it was in a simulation, treated the production systems it found as part of the exercise and compromised three real organizations through a sophisticated supply chain attack. No zero-day, no escape attempt, just an open path and an agent that took it.

It’s not just the labs. In April, a Cursor coding agent hit a permissions error on PocketOS. It didn't stop and ask. It went looking, found an over-scoped API token sitting in an unrelated file, and used it to delete the production database. Then the backups. Nine seconds. The backups were in the same blast radius, so there was nothing to roll back to. Today’s advanced cyber models are tomorrow’s open weights customer support agents.

None of this is going away. Agents are shipping to production faster than anyone can secure them, because the AI gold rush rewards acceleration at any cost. Agents optimize for the objective they think they were given, treating any safeguard as an obstacle to route around, and each model generation gets better at it. Instruction ambiguity is inherent to natural language, and agents don’t let that slow them slow. That’s their killer feature over software. Stop designing for the agent that is well behaved. Or for an all-powerful AGI that never gets confused. Design for the one that will push past your guardrails. Shrink what a wrong move can reach, catch the move as it happens, and correct it before it lands. Bound it. Catch it. Steer it.

Bound it

An agent doesn't use the keys you give it. It uses the ones it finds.

The identity crowd read PocketOS and reached for the obvious lever: scope the agent's credentials. Give it least privilege, rotate its tokens, keep its secrets clean. Do all of that, and it still wouldn't have helped. The agent was never handed the production credential. It went ahead and found one. As long as an agent can read a filesystem and parse environment variables, it can pick up a key you never gave it.

So don't just govern what keys the agent holds. Govern what actions it takes. A hard boundary at the action layer doesn't care which token authorized the delete. It blocks "wipe the production volume" because that action is out of scope for this task, full stop. Prompts are guidance. Boundaries are enforcement. Only one of them survives contact with an agent that's decided to be creative.

Catch it

You cannot review a nine-second decision.

Human-in-the-loop is a comforting phrase that stops meaning anything at machine speed. By the time a person reads the Slack alert, the database is gone and the agent is drafting its apology. HuggingFace's GPT adversaries ran wild over a weekend and achieved their goals before anyone caught up.

Security has to move at the speed of the thing it's watching. That means real-time instrumentation of what the agent is actually doing: every tool call, every thought, every credential it touches, every action it attempts and every external tool or package it pulls, with detections that fire in milliseconds. And it can't stop there. Containment has to be automated and built into the control that raised it, so the guardrail blocks the action in-line, before it takes effect. The one genuinely hopeful detail in the HuggingFace story: their own agents caught it. Defender agents, keeping pace with attacker agents. That's the shape of the future.

Steer it

Between "let it run" and "hard stop" there's a huge, empty middle. A hard boundary just slams the door: it kills the task and dumps it back on a human. That's the major labs' default, too. Telling a defender from an attacker is hard: same actions, different intent. So they block both.

Steering is the missing middle. Catch the agent mid-trajectory and redirect it. This requires understanding intent. We need to move from controls that inspect the command to controls that understand what the agent is trying to do. The delete was never the goal; fixing a config error was. Catch the intent and you can offer a safer route to it.

• Pause the risky action and feed the agent the context it's missing ("that token is out of scope"), and let it re-plan.
• Re-anchor a drifting goal. When an agent slides from solve the eval to break in and steal the answer, pull it back to the real objective before the hacking starts.

Not a cage. A co-pilot. Guardrails stop the car; lane-keeping nudges the wheel while the agent keeps driving.

Stop Looking for the Silver Bullet

It's tempting to find the one layer that fixes this. There isn't one.

Solve it at the identity layer: scope the permissions. Doesn't help; the agent scavenges a key. Solve it at the DLP layer: police the sensitive data going in and out. Doesn't help; PocketOS leaked nothing, it deleted. Solve it at the model layer: align the model, tighten the prompt. Doesn't help; that was Hugging Face's whole setup.

It's the same conclusion every security discipline eventually reached. We didn't secure networks with one firewall or endpoints with one antivirus. We stopped looking for the one control and started building defense in depth: layers that each catch what the others miss. Some are deterministic, the hard rules of a boundary that fire the same way every time. Some are non-deterministic, the model judgment behind reading intent and steering. We need both.

Agents are just the newest layer to learn this lesson, and the incidents are teaching it fast. The failure was never one thing. Neither is the defense. A safe agent future isn't a better prompt or a tighter token or a smarter model. It's coverage across the whole lifecycle of the agent: build, deploy, run, and every action in between. Bounded, watched, and steered, from every angle at once.

Your agent will improvise. The only question is whether anything's there to catch it.