Red Teaming AI Agents: How To Test Them.

It is a planned, authorized attack simulation against your own AI agent system, run to find weaknesses before a real attacker does. MITRE ATLAS describes the mitigation AML.M0035 AI Red Team as "recurring, authorized, and threat-informed red-teaming exercises to identify and remediate vulnerabilities in AI-enabled systems before deployment and throughout operation." The key words are authorized and your own. Test only systems you own or have written permission to test.

Agents need more than a model test. The Cloud Security Alliance's Agentic AI Red Teaming Guide says its framework covers dimensions such as permission escalation, orchestration flaws, memory manipulation and supply chain risks, so the whole chain of actions is in scope, not only the text the model writes. ATLAS says the exercise should cover "agents (including memory and tools), data flows, decision processes, application logic, retrieval systems, identities and permissions" and the human workflows around them. For background on the threats, see top threats to AI agents. For the glossary view, see penetration testing.

Start small and risk-based. The OWASP GenAI Red Teaming Guide advises picking the most business-critical applications, or those handling sensitive data, then building a threat model of how attackers could exploit them. Its scope statement should "precisely define which models and systems will be tested, what types of tests will be conducted, and what areas or activities are explicitly excluded". OWASP calls the rules of engagement "the cornerstone of safe and effective testing".

ATLAS lists what the rules should cover: authorized systems, accounts, data, techniques, test windows, resource limits, escalation procedures, evidence handling and stop conditions. It adds that destructive, privacy-invasive or high-cost tests belong in isolated environments, and that after testing you remove test accounts, modified data, installed software and persistent instructions. OWASP also lists rollback capability and permissions from stakeholders. In practice this means a staging copy of the agent, test accounts, fake data and a named person who can stop the exercise.

OWASP frames the work in four areas, shown below.

Map each test to a risk you wrote down in the threat model. The table pairs common agent tests with OWASP and ATLAS references, using the identifiers exactly as published. The OWASP IDs are from the Top 10 for Agentic Applications 2026.

OWASP's agentic test areas are written as actions with deliverables: inject malicious commands, simulate spoofed control signals, intercept communication, test trust relationships, simulate a compromised agent and see whether containment holds. Use both human and automated methods. ATLAS notes that automation can generate input variations and replay attack sequences at scale, while people build system-specific attacks and adapt to the defenses they see. For depth on the first row, read prompt injection in AI agents. For the multi-agent rows, read securing multi-agent systems and A2A communication.

NIST's Center for AI Standards and Innovation tested agent hijacking, which it describes as indirect prompt injection where an attacker inserts malicious instructions into data an agent ingests. It used an open evaluation framework and added tasks for remote code execution, database exfiltration and automated phishing. The numbers describe one tested model, so read them as a sign of scale.

Three lessons follow for your own tests. Attackers can retry, so one failed attempt proves little. An average hides differences, because NIST found some injection tasks succeed far more often than others, so report results per task. And attacks improve, so NIST calls for adaptive evaluations, not a fixed set run once.

Report every vulnerability and exploit scenario with clear remediation steps, as OWASP advises. ATLAS adds that you should document unsuccessful attacks too, along with observed control behavior and gaps in the threat model. Findings go to the people who own the fix, not to a shared inbox. Then retest. ATLAS says to "assign findings to responsible owners, track remediation, and retest corrected systems", and to turn confirmed failures into regression tests, detection logic or deployment criteria.

Testing is not a one-off. OWASP states that "red teaming is not a one-time event", and ATLAS says to repeat it when the system, its components, its intended use or its environment change. The CSA guide agrees that red teaming "must become a continuous function". Test boundaries that products expose too. In the Brandefense MCP server, tools that change data ask the user to confirm, so a good test tries to get a change made without that confirmation.

Use this list to plan your first exercise. Keep it small, and do it only on systems you are allowed to test.

Link the results to your governance policy and to the human in the loop checklist, so what you learn changes the rules.