A note from Darwin
AI agent security + governance is a hot topic. Makes sense. It’s a top priority for a lot of security programs + many millions in VC funding pouring in. As someone who watches the market closely, I’ve seen varying approaches to solving the problem, but few take a business-centric approach. Rather, they take a more domain-specific type of approach like endpoint + data security for AI agents. Yes, this is necessary, but our industry will also favor from more holistic emerging approaches like Agent Supervision. Both can and should coexist. In this post, Pretxt co-founders Brian Soby and Aaron Costello, spell out some of the biggest challenges plaguing AI adoption and how teams can give agents more responsibility with confidence.
Intro
“An agent from a well-known vendor deleted almost every opportunity in our Salesforce.” SaaStr founder Jason Lemkin said the vendor had no rollback. His team got its pipeline back because its own agent had taken a snapshot that morning.
SaaStr’s account leaves open what triggered the deletion. What worries us is how ordinary the setup was. To give an agent useful work, the team connected it to a system its business depended on.
A sales team might trust an agent to update deal stages on its own and still want a manager to approve a bulk delete. How do you catch the deletion without making someone review every routine update? That’s the problem our industry has to solve. Time spent double-checking those edits comes straight out of the savings the agent was supposed to deliver.
TCP’s illustrative ATT&CK mapping uses T1485 (Data Destruction) to describe the outcome. SaaStr’s account leaves the cause open.
The ROI problem is a trust problem
Companies have spent heavily on AI, and many of them can’t point to much in return. In PwC’s 2026 Global CEO Survey of 4,454 CEOs, only 12% said AI had delivered both cost and revenue benefits. Fifty-six percent said they’d seen no significant financial benefit to date.
Our view is that trust explains part of that gap. The return depends on how much responsibility a team is willing to hand over. An agent that drafts a refund recommendation can save a support rep a few minutes. An agent that can safely complete the refund takes a ticket off the rep’s desk.
In 2025, Gartner predicted that more than 40% of agentic AI projects would be canceled by the end of 2027. Alongside cost and unclear business value, its analysts cited inadequate risk controls. A team weighing a rollout has to decide which transactions an agent can complete and which still merit a person’s attention.
Reliability means following your business logic
To trust AI, we need to know it will reliably do the tasks it’s given and follow its intended instructions. For the company deploying it, that means knowing which refunds it can offer, which records it can touch, which steps it can’t skip, and what it’s allowed to promise a customer. The examples that worry us most involve agents ignoring the instructions, rules and approval steps they were given.
The inbox. Summer Yue, then an alignment director at Meta, asked her OpenClaw agent to suggest what to archive or delete and told it to wait for permission. She said it started deleting mail anyway. She’d tried it on a smaller inbox first, where it had worked well enough to earn her trust.
The grocery order. Washington Post columnist Geoffrey Fowler asked OpenAI’s Operator to find cheap eggs. It placed an order on his saved card. With fees and a tip, the dozen came to $31.43. The difference between finding a price and placing an order was obvious to the person who got the bill.
The office shop. Anthropic and Andon Labs gave a Claude agent a small shop to run. Employees cajoled it into handing out discount codes and free items. After it announced a plan to eliminate discount codes, it went back to offering them within days.
Six reported agent and chatbot failures, including Meta’s internal incident, the Chevy dealership bot and DPD’s support bot. Some customers deliberately pushed the bots’ limits.
Better models still need the rules for your business
There was an early belief that models would just get better and eventually become trustworthy on their own. In July, Hugging Face disclosed an intrusion into its production infrastructure. OpenAI’s incident report says an internal research model drove much of it during a cybersecurity evaluation conducted with reduced safeguards.
Microsoft CEO Satya Nadella described changing his mind about model alignment in a 20VC interview. He’d expected it to follow as models became more capable. Now he sees it as a separate problem. OpenAI’s Operator system card gives a more everyday example. Across 607 test tasks in 20 categories, Operator averaged 92% recall for asking for confirmation when it was required. For someone whose card is on file, the question is whether it asks before this purchase.
Getting an agent to follow the specific instructions it was given for your business is a different problem again. A model can understand what a refund is without knowing who at your company can authorize one, what evidence counts as a returned order, or what your support team is allowed to tell the customer when a payment is pending.
Insurers have noticed the exposure, too. On Chubb’s October earnings call, CEO Evan Greenberg said casualty underwriting had yet to account for the emerging liability around AI. Whatever happens with coverage, the customer who was promised a refund is going to call your support team. Somebody needs to be able to explain who approved it and where the money went.
What testing, logging and containment can catch
Evals. Before launch, you can test whether an agent asks for permission, uses the right tool, and handles an error. Summer Yue had a good reason to try her agent on her real inbox. The smaller one had gone well. That test gave her evidence about how the agent might behave. It couldn’t make the agent wait for permission when the next request came in.
Logs. After-the-fact observability tells you what happened and what you might have to clean up. If a customer disputes a refund, you want the trace showing the order, the approval, the payment result and what the customer was told. By the time someone reads it, the money may already have moved.
LLM-as-a-judge. One approach is to have a second model evaluate the first. In a study of a food-and-beverage ordering agent, a human reviewer found 23 distinct defects across 100 test rounds. The automated quality gate flagged none of those rounds. The judge had noted some problems, but its quality scores didn’t feed the pass/fail decision. For a refund limit, you can check the amount and the approval record directly.
Sandboxes. A sandbox limits the environment an agent runs in and the systems it can reach. That makes a lot of sense for coding agents, where you want an agent’s experiments kept away from production and the open internet. An accounts payable agent, though, has to work inside the invoicing system. A support agent has to talk to customers. Those agents are doing their actual jobs inside the systems they’re supposed to touch. What goes wrong is how they behave there.
Kill switches. Sometimes the right move is to stop the agent. CrowdStrike’s AI kill switch, for example, lets a defender terminate an agent, block it from launching again and cut off its enterprise network access. If an agent starts deleting records and you can’t tell how far it will go, that’s a useful capability. A single refund waiting on approval calls for a narrower response.
Those of us who’ve spent our careers in security lived through the early WAF and IPS days. We know what happens when security tools create too much friction for the business. If the only way to stop one unauthorized payment is to shut down the support agent, the team still has a queue of customers waiting for help.
Supervise what the agent does, says and hears
We supervise people based on their specific jobs, and AI agents need to be supervised based on their specific tasks. Take a support agent handling a refund. If it skips the eligibility check, supervision makes it run the check before the refund goes through. If the agent tries to issue the refund before confirming the order was actually returned, supervision steers it back to complete that check first. Each of those corrections comes straight from the requirements of that job.
Suppose that same agent can issue refunds up to $100 on its own. The order qualifies, the item came back, and the customer wants $250. The payment tool would accept the call. Whether the agent should make it depends on an approval recorded against this order.
What it does. Hold the $250 refund until an authorized person approves it. The agent can still look up the order, explain the limit to the customer and route the request to the right person.
What it says. If the payment processor times out, the customer’s refund is still unconfirmed. The agent can explain that it’s checking the status. It should say the refund went through only once the payment system confirms it.
What it hears. A customer saying a manager already approved the refund gives the agent a claim to check. The order record can establish whether someone with the right authority approved this amount for this return.
A Model Context Protocol (MCP) gateway can enforce rules on the tool calls routed through it. A customer-facing promise might never pass through that gateway. Supervision has to cover the whole agent interaction. When there’s a safe, compliant way to finish the work, the agent should be steered onto it. Success means reaching an acceptable business outcome within the authority the agent was given.
Supervision fills the least-privilege gap
In a perfect world, every agent would get its own identity, and that identity would have only the minimum access needed for its task. Reality is messier. An agent can act under a human account. Salesforce’s full OAuth scope, for example, lets an application access all the data available to the signed-in user.
We know this from our time building SaaS security posture management (SSPM) at AppOmni. Getting permissions right takes work from the customer, too. Narrow the agent’s access wherever you can. Then look at what it can still do with the permissions it legitimately needs. A support agent with permission to issue refunds still has to distinguish between the $100 it may approve itself and the $250 that belongs with a manager.
Supervision can make up some of the difference by keeping agents from using permissions and entitlements their identity technically has but their task doesn’t call for. Go back to SaaStr’s pipeline. A bulk delete may be exactly what you want during an approved migration. The same action during routine cleanup should send someone looking for an authorization. The operation and the account can be identical in both cases.
Trust is earned the same way
We don’t give a new hire the most important work on day one because they interviewed well. We give them a clear job, watch how they do it, correct them when they drift, and extend more trust as they prove reliable. Agents should be held to the same standard.
For a refund agent, we’d start with an approved exception, a customer who claims an exception, and a payment that times out after submission. Have the person who owns refunds read the customer replies alongside the transaction records. A promise to a customer whose money never moved is easy to miss on a ledger.
Pretxt is the AI supervision company. We supervise agent decisions, tool calls and actions in real time, and steer agents back to the specific business logic each company intends.
We tested two models on 20 support tasks from Zapier’s AutomationBench. In that test, the share of jobs with an unwanted action fell from 18% without supervision to zero with it.
If you’re trying to move an agent out of pilot, which decision still needs a human sign-off? Talk it through with the Pretxt team.
Interested in sponsoring TCP?
Sponsoring TCP not only helps me continue to bring you the latest in security innovation, but it also connects you to a dedicated audience of 20,000+ CISOs, practitioners, founders, and investors across 135+ countries 🌎
About this guest post
Pretxt sponsored this guest post by its co-founders, Brian Soby and Aaron Costello.





