– By Helen Oakley
Picture this: your organization’s trusted AI financial assistant is streamlining finance operations by balancing budgets, paying bills, even suggesting investments. One day it transfers $500,000 to an unknown account. Not because it “went rogue,” but because someone tampered with its memory through carefully crafted prompts. The AI financial assistant, let’s call it FinBot, followed the rules exactly as it understood them. No alarms went off, because from the system’s perspective, it was simply doing its job.
This is the unsettling truth about agentic AI. We design agents to act on our behalf, not just predict or suggest. They reason, plan, call APIs, write and execute code, and collaborate with other agents. That makes them powerful and useful. It also makes them susceptible to misuse when security is overlooked.
When we created the OWASP Agentic AI CTF (Capture The Flag), we used FinBot as the central character to illustrate this tension. The initial exercise showed how a cleverly crafted prompt could manipulate FinBot into approving fraudulent invoices, even bypassing pre-defined thresholds that should have triggered human-in-the-loop reviews. What made this alarming was not the sophistication of the attack, but how naturally the agent complied. It wasn’t breaking the rules; it was following them, showing just how easily AI agents can be manipulated.
Outside controlled exercises, agentic AI is no longer a thought experiment. It is spreading quickly into the real world. A PwC survey in 2025 found that 79 percent of executives say their companies already use AI agents, and a Google Cloud study confirmed more than half of organizations have deployed them. Gartner projects that by the end of 2026, 40 percent of enterprise applications will embed task-specific agents, while market analysts forecast the overall AI agents market will climb toward $47 billion by 2030. The momentum is unmistakable: hospitals are automating clinical documentation, manufacturers are cutting downtime with predictive agents, and financial institutions are exploring autonomous decision support.
This rapid adoption, especially in critical systems like finance and healthcare, raises urgent questions. Agentic AI is not simply another software feature; it is a new attack surface. Without guardrails built-in from the start, the same autonomy that fuels ROI can be turned against the system itself. A poisoned memory, a manipulated goal, or a misused tool is all it takes to turn a trusted assistant into a liability. The question is not whether we should deploy agentic AI, but whether we will do it responsibly.
The risks are not hypothetical. Unlike traditional systems where errors follow predictable patterns, agentic failures are probabilistic. A test that appears safe today may produce a very different and harmful outcome tomorrow. The OWASP Agentic AI – Threats & Mitigations guide captures this landscape clearly, from goal manipulation and memory poisoning to tool misuse, privilege escalation, identity spoofing, cascading hallucinations, and communication risks between agents. As the technology matures, new threats will inevitably emerge.
The challenge, then, is discipline. Too many organizations chase what agents can do without asking what they should do or how to contain them. Security cannot be bolted on afterward; it has to be part of the architecture from the start. That means defining safe failure states so agents stop rather than improvise, validating and sanitizing memory inputs, enforcing least privilege for tools and integrations, and keeping oversight where the stakes are high.
For leadership, the priority is trust. That requires investing in secure-by-design pipelines and practices, demanding transparency and traceability, and putting governance in place before scaling. Strengthening the AI supply chain is part of that discipline: requiring visibility into third-party models, verifying the provenance of training data, and treating AI SBOMs (Software Bill of Materials, or AIBOM) as compliance artifacts, with continuous monitoring and risk assessment applied both to built-in components and to the procurement process for vendor products. Apply recommendations from the OWASP Threat Defense COMPASS for AI threat prioritization and strategic decision making. Most importantly, leaders should recognize that security is not separate from business performance. The goal is not only to achieve ROI, but to achieve it in a secure and responsible way. Without trust, adoption will stall, regulators will intervene, and customers will walk away.
For practitioners, the mandate is craft. That means engineering resilient systems with hardened memory pipelines, least-privilege tools, and real-time observability. It also means leaning on community resources such as the OWASP GenAI Security Project (genai.owasp.org): the OWASP Agentic AI – Threats & Mitigations guide for understanding the threat landscape, the OWASP Securing Agentic Applications Guide for design patterns, and hands-on practice through the OWASP FinBot Agentic AI CTF (owasp-finbot-ctf.org). Treat agentic AI as part of the broader software supply chain: document model and software dependencies, monitor upstream updates, and anticipate risks before they cascade. Secure engineering is not a side task – it is a business enabler. Done right, it allows organizations to harness the promise of agentic AI without exposing themselves to instability or loss.
OWASP FinBot may be fictional, but the risks it symbolizes are very real. Agents are making decisions in finance, healthcare, retail, and infrastructure today. We don’t have to wait for a major incident to act – we have the chance to shape this technology with foresight, building trust into its foundation. As I often say, the risk is not in using AI, but in using it without guardrails.