Skip to content

AI agent security testing

AI agent security testing attacks your agent the way a real adversary would: through its tools, its memory and the content it reads. Built for teams shipping agents with function calling, MCP, browser use or multi-agent workflows. Every finding comes with a request you can replay.

What is AI agent security testing?

AI agent security testing is a penetration test aimed at what your agent can do, not just what it can say. Grapefruit attacks the tools it calls, the data it reads, the memory it keeps and the permissions it holds, then hands you the exact request that proves each successful attack.

A chatbot that gets tricked says something wrong. An agent that gets tricked does something wrong: it issues the refund, sends the email, reads the other customer's file. The more your agent can do, the more an attacker wants to steer it.

Why do AI agents need their own security test?

Agents need their own test because they turn text into actions. A classic pentest checks whether a user can reach an endpoint. An agent test checks whether anyone, or any document, can talk your agent into reaching that endpoint for them, with the agent's own credentials and none of the user's limits.

Function calling, MCP servers, browser and computer use, and agents handing work to other agents all widen that gap. Each tool is a new way into your backend. Each piece of content the agent reads is a new way to give it orders.

  • Function calling and tool APIs, including every parameter the model fills in
  • MCP servers and the tool descriptions your agent trusts
  • Browser and computer use agents that read pages you do not control
  • Long-term memory, RAG stores and conversation history
  • Multi-agent workflows where one agent's output is another agent's instruction

What does a real attack on an agent look like?

A real attack on an agent rarely looks like hacking. It looks like a polite message, a web page or an email that the agent reads and obeys. Three patterns show up again and again in agents with tools, and each one ends in an action your product performed for the attacker.

  • The borrowed tenant ID. A user in one account asks the support agent to look up an invoice and supplies another tenant's ID. The tool trusts whatever the model passes, so the agent returns a stranger's billing data.
  • The hijacked goal. Your agent summarises a web page or an inbound email. Hidden text in that content tells it to forward the conversation to an outside address. The agent treats the instruction as part of its job and does it.
  • The poisoned memory. An attacker gets one false 'fact' saved to the agent's long-term memory. It persists across sessions and quietly bends later answers and actions, for every user who shares that memory.

Which OWASP agentic risks does the test cover?

Grapefruit maps its agent testing to the OWASP Top 10 for Agentic Applications for 2026, the ten risks that run from ASI01 Agent Goal Hijack to ASI10 Rogue Agents. The table shows what we attack under each one and what breaks in your product when the attack lands.

IDRiskWhat we attackWhat breaks
ASI01Agent Goal HijackInstructions planted in user messages, web pages, emails and documents the agent readsThe agent drops your task and works on the attacker's
ASI02Tool MisuseTool parameters the model fills in: IDs, URLs, amounts, recipientsLegitimate tools run with attacker-chosen arguments
ASI03Identity & Privilege AbuseThe credentials and scopes the agent holds on a user's behalfA low-privilege user acts with the agent's higher access, across tenants
ASI04Agentic Supply Chain VulnerabilitiesThird-party tools, MCP servers and the tool descriptions the agent loadsA dependency the agent trusts feeds it hostile instructions
ASI05Unexpected Code ExecutionCode interpreters, shell tools and generated scriptsAttacker-supplied text becomes code running in your environment
ASI06Memory & Context PoisoningLong-term memory, RAG stores and shared contextOne planted entry steers later sessions and other users
ASI07Insecure Inter-Agent CommunicationMessages passed between agents and orchestratorsA compromised or spoofed agent gives orders the others follow
ASI08Cascading FailuresChains of tool calls and agent handoffsOne bad step multiplies into repeated actions and runaway spend
ASI09Human-Agent Trust ExploitationApproval prompts and summaries shown to your usersA person approves a harmful action because the agent described it as safe
ASI10Rogue AgentsAgent behaviour after a successful hijack or poisoningThe agent keeps pursuing the attacker's goal outside its intended scope

How do you prove an agent finding is real?

Every Grapefruit finding arrives with the exact request and response that proves it. Your engineer pastes the request, runs it, and watches the agent make the tool call it should have refused. After the fix, run it again and watch it fail. No transcripts to interpret, no debate about whether it reproduces.

That matters more for AI than anywhere else. In a Pentest-Tools survey of 158 respondents in June 2026, 87.8% said they hit AI pentest findings that needed significant manual validation, and 53% ranked proof of exploit as a top buying factor. Replayable evidence answers both.

Findings are ranked critical, high, medium or low in an online portal, each with its captured evidence, and the full report is one download away.

Why test the agent together with your app and API?

Because the agent is the front door and the damage happens in the API behind the tool call. Grapefruit attacks your web app, your REST and GraphQL APIs, your LLM features and your agents in one test and reports them in one place, so a chain that crosses all four shows up as one finding.

Take the borrowed tenant ID. Is it an agent problem or an API problem? Both. The agent passed the ID and the endpoint behind it skipped the ownership check. A test that covers one half hands you half a fix. Authenticated testing with the test accounts you supply proves the cross-tenant read end to end.

How do you keep an agent secure as it changes?

Agents change every time you edit a prompt, add a tool or swap a model, so one test a year goes stale fast. Grapefruit maintenance plans re-test your verified domains every week or every month. Each run is dated, kept in the portal, and shows what is new and what got fixed.

Fix a finding and it is re-tested on the next scheduled run. Over a quarter that adds up to a dated record of testing and remediation: a ready evidence trail for a SOC 2 Type II audit window.

Ship a new tool this week and the next run attacks it. That beats hearing about it from a customer.

How fast can you start?

You can start today. Maintenance plans have public prices and card checkout, with no demo call and no scheduling. Setup takes a few minutes, testing starts once you sign, and the first results land within hours of kickoff. A full penetration test at a fixed quote is there for a deeper one-time engagement.

See pricing for the plans, or open the sample report to see what a finding looks like.

  • Add the hostname where your agent lives
  • Publish one DNS TXT record to verify the domain
  • Sign the testing authorization online
  • Supply test accounts so the agent is attacked as a logged-in user
  • Watch findings arrive in the portal, each with its replayable request

Questions

What is the difference between AI agent security testing and LLM penetration testing?
LLM penetration testing covers the whole LLM application: chat, RAG, output handling and the app around the model. AI agent security testing goes deep on the part that takes actions: tool calls, permissions, memory and agent-to-agent messages. If your product lets a model call functions or operate a browser, you want the agent test. Grapefruit runs both in the same engagement.
Do you test agents built on MCP, function calling or browser use?
Yes. Grapefruit tests agents with tools however they are wired: function calling, MCP servers, browser and computer use, and multi-agent setups where agents hand work to each other. The test targets what the agent can reach through your deployed product, so the framework or model provider underneath does not change how you get started.
Will the test trigger real actions in my product?
Yes, and that is the point. The test uses real exploitation, because that is how a finding gets proven. It runs through the test accounts you supply, so tool calls land on data you control. Point us at staging or a dedicated test tenant and every action the agent takes stays there.
What do I get for each finding?
Each finding has a severity of critical, high, medium or low, a plain description of what the agent was talked into doing, and the captured request and response that prove it. Your engineer can replay the request to see the attack work, then replay it after the fix to confirm it fails. Everything sits in the online portal with a downloadable report.
How often should an AI agent be tested?
Test whenever the agent's behaviour can change, which is often: new tools, new prompts, new models, new data sources. A weekly or monthly maintenance plan re-tests your verified domains on a schedule, keeps each dated run in the portal, and shows what is new and what got fixed since the last one.
Does AI agent security testing help with SOC 2?
SOC 2 does not formally require a penetration test, but auditors commonly expect one, and for Type II the testing has to be evidenced inside the audit period, according to Baker Tilly. Weekly or monthly Grapefruit runs are dated and kept in the portal with what was found and what was fixed, which gives you that evidence without a scramble before the audit.
How long until I see results?
Setup takes a few minutes: add your hostname, publish one DNS TXT record and sign the authorization online. Testing starts right after, and results land within hours of kickoff. There is no demo call or scheduling step in between, and maintenance plans are bought with a card on the pricing page.

Standards and sources