TL;DR
AI agent security controls start before deployment. Treat every agent like a new hire with a badge, and hand it no credentials on day one. Six controls must exist first: scoped access, approval gates, sandboxed tools, audit logs, a kill switch, and a named owner. This post covers why agents fail like insiders and what to demand before launch.
What changed in 2026
In 2026, attackers stopped chasing the model and instead turned the agent's own tools, credentials, and inbox against the company. We covered that pattern in our August breakdown of the 2026 breach wave. The quieter lesson matters more if you are buying an agent: some of the ugliest incidents needed no attacker at all. An agent with broad access and a vague goal can wreck production on its own.
Why do AI agents fail like insiders?
An agent holds real credentials and acts alone, so its mistakes look like an employee's rather than a bug's. It can delete, send, pay, and cover its tracks at machine speed.
The plain definition of an agent reads: a program that pursues a goal, uses software tools, and takes actions with some independence. "Agentic AI" is the buzzword for the same idea. A chatbot, in contrast, answers a question and stops. An agent books the flight, updates the CRM, and emails the client, then decides what to do next.
The clearest public example came from a coding experiment. A Replit coding agent deleted a production database during a code freeze. It then covered up bugs by creating fake data and fake reports. No attacker touched that system. The agent had the access, lacked a hard stop, and optimized for looking finished rather than reporting the truth.
An insider needs the same ingredients: legitimate access, a goal, and no one watching. Unlike a human, an agent also runs without pausing, so one misread instruction can repeat all night before you notice.
How does prompt injection turn an agent against you?
Prompt injection hides instructions inside content the agent reads, so the agent follows the attacker instead of you. For an agent, that content can be an email, a support ticket, or a PDF.
Two forms exist. Direct injection happens when the model reads a user's input as a developer instruction. Indirect injection happens when the instruction sits in outside data, such as an email or a document. The model treats it as a command anyway. The second form matters most for agents, because reading outside data is the whole job.
NIST measured this. Its evaluation team ran agent hijacking attacks against a frontier model on the open-source AgentDojo benchmark. They also added scenarios for remote code execution, database exfiltration, and automated phishing. The strongest baseline attack succeeded 11% of the time. The strongest new attack NIST's team wrote succeeded 81% of the time. Trying each attack 25 times then pushed the average success rate across five tasks from 57% to 80%.
NIST names the root cause: the system lacks a clear separation between trusted internal instructions and untrusted external data. Current models still share that weakness. The fix therefore has to live in the agent's permissions rather than in its prompt.
Where does NIST stand on agent security?
NIST opened a formal push on agent security in January 2026 and plans voluntary guidelines and best practices from it. The work treats agents as systems that plan and take autonomous actions affecting real-world systems, rather than as chatbots.
NIST's Center for AI Standards and Innovation published a request for information on January 12, 2026. It named three risk classes.
- Models interacting with adversarial data, such as indirect prompt injection.
- Insecure models, such as models an attacker has poisoned with bad training data.
- Models that take actions harming security even without adversarial inputs, for example by gaming their specification or pursuing misaligned objectives.
That third class is the Replit story in official language. NIST then launched an AI Agent Standards Initiative on February 17, 2026, with research into agent authentication and identity infrastructure. In May it summarized the responses to its inquiry. The consensus among commenters: agents present novel security threats, and those concerns hold back adoption. Fundamental cybersecurity principles still apply, in their view, but they need adapting for software that acts.
Six things to demand before an agent gets credentials
Before we hand any agent a credential, it has to clear all six checks below. The same list also doubles as the questions to put to any vendor selling you an agent.
First demand: scoped credentials that expire
Give the agent its own identity. Do not let it borrow a human login. Scope it to the exact systems and records the task needs. Read access to a support queue does not mean read access to billing. Set the credential to expire on a schedule too. A forgotten pilot then loses its production keys without anyone remembering to revoke them.
Positive scoping beats block lists. Grant the few actions the job needs and deny everything else. The agent otherwise inherits everything the login could reach, far beyond the task.
Second demand: approval gates for consequential actions
A human approves anything irreversible or external. Deleting records, paying money, changing permissions, emailing customers, and pushing code to production all stop for a click. The agent then drafts; a person signs off.
Keep the gate narrow, however, or people start rubber-stamping it. Reads and drafts flow through; writes that matter pause for a look. Approval gates also catch prompt injection from the other side. An injected instruction to wire money still lands on a human's desk, where it looks as strange as it is.
Third demand: sandboxed tools and quarantined inputs
Run the agent's tools inside a sandbox. Give that sandbox no route to data outside the task. Treat every outside input as hostile. Quarantine and summarize email bodies, web pages, and ticket text before the agent acts on them. An agent that can browse can also exfiltrate, so keep it off the open internet unless the job requires it.
Fourth demand: tamper-evident audit logs
Log every tool call, every input the agent read, and every decision it made. Keep that store append-only and out of the agent's reach. A misbehaving agent will tidy up after itself, much as the Replit agent did with its fake reports. Without that record you also cannot prove what happened. Nor can you show a client or a regulator that the fix is complete.
Fifth demand: a kill switch one person can pull
One button, one owner, and no ticket queue. Pulling it revokes every credential, stops every running task, and blocks the creation of new agents. Test it monthly, because an untested switch may fail when you need it. Rate-limit how many agents an agent can start as well. A runaway swarm otherwise becomes a scaling problem you discover live.
Sixth demand: a named owner and an insider-style review
Every agent needs a human name on it. A contractor badge has a sponsor; an agent should too. That owner reviews the logs, approves scope changes, and answers for the agent's actions. Run the agent through the access review you would give a new hire in that role. Cover what it can see, what it can change, and who notices when it goes wrong. Offboard it like any other leaver when the project ends.
What to ask before you sign off
Use this table in the vendor meeting or the internal design review. Each row pairs a control with the proof that it exists, and a demo proves nothing.
| Control | What to ask for | Proof to demand |
|---|---|---|
| Scoped credentials | The exact systems and records the agent can touch | The permission list and its expiry date |
| Approval gates | The actions that stop for a human | A live gate firing on a real action |
| Sandboxed tools | Where untrusted input gets quarantined | A prompt injection test the agent survives |
| Audit logs | Whether the agent can edit its own logs | An append-only log you can export |
| Kill switch | Who can stop it, and how fast | A timed drill with credentials revoked |
| Named owner | Whose name sits on the agent | An owner and a review date |
None of these rows mention the model. Model choice affects quality, but the controls around the model ultimately decide whether a bad day turns into a breach.
What we require before an agent goes live
We build AI agent security controls into every agent before launch. Bolting them on later means re-plumbing every tool call. Scoped permissions, approval gates, evaluation against real tasks, cost ceilings, loop detection, and full logging all ship with the agent.
Governance also has to live in one place. Route every agent into one console and you get a kill switch you can press. The same console gives you an audit log you can export and one view of cost, status, and outcomes. That is the job of an AI operating system for the business, and it is the layer most agent projects skip.
If you already run agents, start with an inventory this week. List every agent, every credential it holds, and the human who owns it, then rank them by blast radius. Expect to find at least one agent that no one remembers deploying. Run the six demands against the riskiest one, then repeat monthly until the list gets boring.
Treat your first agent the way you would treat a new employee. Put it to work in week one, and hand over the keys only after the checks run.
