Your Voice AI Agent Takes Calls From Strangers. Who’s Checking?

Last week, US prosecutors indicted three people who allegedly used AI to impersonate both a CEO and a law-firm attorney on live calls to steal about $129 million from companies worldwide. Employees first heard a familiar executive’s voice, then an authoritative “attorney” telling them where to wire the money. Victims allegedly included hospital systems, a bank, a fintech, and a mutual fund company.

Earlier this year, the ShinyHunters group allegedly used a vishing call to reach an employee’s Microsoft Entra account at Charter Communications and extract 4.9 million sensitive records.

The most worrying aspect of these attacks is that neither case needed sophisticated malware: both used readily available AI to fake voices, and both depended on a person who believed what they heard.

Now consider your own voice AI agent. It answers at any hour, can’t see a face, and is built to be helpful. It can look up accounts, reset access, and maybe even issue refunds. An attacker doesn’t need an exploit to get through it. They only need to set up an AI-driven operation that probes the agent call after call until it finds weaknesses. Within minutes, the agent hands over, or does something else it shouldn’t.

Why Voice Agents Are Everywhere

Phone calls were the last large customer channel left mostly unautomated, and real-time speech models now handle business-specific context as well. Meticulous Research sizes the voice AI agent market at $5.1 billion in 2026, growing to $43.8 billion by 2036. McKinsey’s 2026 survey found 40% of large organizations are now scaling AI agents, up from 27% a year earlier.

For security, what matters is that voice agents can now take actions with consequences: book, refund, order, verify, and disclose.

Why Voice Agents Are Hard To Protect

Securing voice agents presents a unique security challenge compared to traditional text-based AI or web interfaces. Because voice conversations rely on real-time human interaction, defenders must balance sub-second responsiveness with continuous context tracking to prevent sophisticated exploitation. Below are the core technical and operational challenges we prioritized with security teams when safeguarding AI voice agents:

1. Latency. A phone conversation has almost no room for delay. Every extra moment of silence makes the agent sound like a machine, and a security layer that slows each turn blocks the business.
2. Urgency is the business case. Most often, voice agents exist to meet immediate customer needs: a locked account, a missed delivery, a claim, a payment. The agent is built to resolve things fast, and urgency is the oldest lever in social engineering.
3. Attacks unfold over the whole call. Each sentence can look harmless while the conversation as a whole steers the agent somewhere it shouldn’t go.
4. Three strikes, one call. In a classic kill chain, the stages are spread over hours or days, and leave traces. A voice call compresses everything into minutes: untrusted input, access to private data, and an outbound channel arrive as three coordinated strikes. Each looks routine on its own, and no single log source sees all three together.
5. Agents are non-deterministic and keep changing. A correctly permissioned agent can still misuse that permission, so a defense can’t rely only on spotting known attack patterns. The agent also doesn’t stay the same. Teams add new use cases and grant new permissions over time, so a control that fit the agent at launch may not fit what it can do six months later.
6. No single call shows the attack. An attacker can dispatch an operation across many calls, several agents, and different workflows, each one a seemingly unrelated, ordinary request. Together they form a coordinated attack, closer to a swarm than a single intruder. A basic security tool that looks at one conversation at a time sees nothing wrong, and it may even read the repeating pattern as customers adopting the agent and a usage pattern that works.

Where Zenity Fits

Start from an uncomfortable premise: a voice can’t prove who is calling. Cloning a voice takes seconds of audio, and Sam Altman (OpenAI CEO) warned financial regulators in July 2025 that AI had “fully defeated voiceprint authentication”. So we cannot depend on individuals being able to tell a real caller from a synthetic one.

We build on the one signal that still matters: intent. Even a genuine, verified caller can be socially engineered, so it’s not about figuring out who’s on the other end of the call, it’s about understanding the situation. If it is an agent, given everything that’s happened on this call, should the agent be doing this?

How it works, and why the caller never feels it

From the outset, it was evident to us that Zenity would never sit in the audio path of our customer’s AI-powered service platform. In the flow we designed with their security architects, Zenity listens out-of-band, receiving the call transcripts, events, and metadata as the conversation unfolds, directly from the platform.

As the call develops, we continuously evaluate it against the agent’s intended purpose and raise a flag when something looks wrong: an attempt to extract data, a steer toward out-of-scope actions, or a gradual drift from the task. Raising a flag changes nothing the caller experiences, because Zenity intervenes only when it really matters.

We designed the customer’s agent to consult Zenity before any risky operation: sending a payment, disclosing sensitive information, changing a credential, or issuing a refund. The customer’s application knows which operations count as risky, and at that point the agent asks Zenity one question: has this call already raised a flag, or does what the caller just said raise one now?

Usually the answer is immediate, because we’ve been listening all along. In the worst case, where the latest audio has to be assessed, a decision comes back within milliseconds at most, and the caller doesn’t notice, because the conversation never stops.

The decision is one of two answers: Allow lets the flow continue. Flag orders call routing to replace the AI agent with a human operator, who validates the caller’s identity or runs other checks. We flag only when we’re confident that a security drift has happened, so legitimate callers aren’t interrupted by false alarms.

Figure 1. Zenity alongside voice AI customer service platform

What each challenge looks like with Zenity

To address these complex security hurdles, Zenity provides a comprehensive design specifically for real-time voice interaction. Below is a brief overview of how our platform directly tackles each of these operational and technical challenges.

• Latency: out-of-band listening on every turn, and within millisecond decisions only for risky operations, covered by the agent’s own confirmation step.
• Urgency: the agent stays fast for every ordinary request, and only the high-stakes step gets a second look, so pressure can’t push a consequential action straight through.
• Whole-call attacks: Zenity evaluates the call as a session, carrying state forward, rather than judging each sentence on its own.
• Three strikes, one call: we see the transcript and the agent’s actions together, as one session, so the combination of untrusted input, private data and an outbound channel is treated as a known, sensitive pattern to validate.
• Non-determinism: the check at the risky step doesn’t depend on the agent behaving as expected, and it applies to whatever the agent can do today, including use cases and permissions added after launch.
• No single call shows the attack: Zenity doesn’t judge each conversation in isolation. It links the findings from many calls, agents, and workflows over time, and builds them into a single incident with the full chain of events, so a swarm of ordinary-looking requests is recognized as one operation.