Why AI Expands Your Attack Surface
Security teams typically set clear boundaries. When a request arrives, the application follows its rules, and the system responds as expected. This predictability makes traditional controls seem reliable because you can verify and validate inputs. That changes when AI enters the picture, and AI prompt injection security becomes a concern most teams are not prepared for.
AI blurs the line between instructions and content. In systems with large language models, language is more than a user request; it guides the system. Prompts, documents, chat history, tool outputs, and system guidance all share the same context. This makes AI powerful but also easier to manipulate than most teams expect. Recognizing this change is the first step to building secure systems.
AI Prompt Injection Security Starts with Text
If you have spent years blocking attacks that sneak commands through inputs, prompt injection will feel familiar, even if the method is new. The pattern is the same: something the system sees as text ends up acting like an instruction. This can make the model ignore rules, reveal information it should not, or act in ways the product never intended (OWASP, n.d.).
Many AI-powered apps are built in ways that invite this risk. They are designed to follow instructions and be helpful. They also generalize, so if the app relies only on the model to do the right thing, it can be easy to steer it in the wrong direction. Unlike traditional systems, there is no parsing step to separate commands from content. Everything comes in as language, and the model decides how to handle it. This flexibility is useful in normal use, but risky when under attack.
The Hidden Risk Inside Your Own Content
Prompt injection is even more dangerous when your system uses retrieval. Once a model can read documents, browse internal tickets, or pull content from a knowledge base, the attack surface expands to anything it might read. Attackers do not need a direct chat window. They can hide harmful instructions in a document the model might see later, hoping the model treats them as important context.
This type of indirect prompt injection has occurred in real-world cases, showing that AI-powered apps can be compromised if they trust content simply because it originates within the system (Greshake et al., 2023). The lesson is clear: retrieval is not just a feature. It is a new way for attackers to get input into your system, and most teams are not monitoring it as closely as a login form or API endpoint.
The Data Leak That Quietly Scales
AI makes data leaks easier to cause and harder to spot. Sometimes the leak is obvious, such as when a user pastes sensitive information into a chat, and it is exposed. More often, it is subtle and happens throughout the system. A model might summarize a restricted document if access controls are not enforced during retrieval. An agent might use internal tools and then share results with someone who should not see them. Logs and traces can also store prompt context and, inadvertently, become places where sensitive data is stored.
The scale makes this especially tricky. A traditional data leak usually has a clear failure point you can trace and fix. With AI, the same problem can happen across hundreds of interactions before anyone notices. This is why risk frameworks focus on managing AI in real-world settings, rather than assuming that intent, policy, or well-crafted prompts will always work (National Institute of Standards and Technology, 2023, 2024). A model can behave well in normal use but still leak data when tested by an attacker.
How Retrieval Changes Your Trust Boundary
Retrieval-augmented generation, or RAG, is often used to improve accuracy and reduce mistakes, but it also changes what trust means in your system. Your model’s output now depends not only on the prompt and user message but also on the documents it retrieves, the ranking logic, and the quality of the content your system indexes.
If an attacker can add misleading documents, manipulate rankings, or hide instructions in seemingly harmless content, they can influence the model without a traditional bug. Often, the system works as designed but simply consumes harmful content (National Institute of Standards and Technology, 2024). This means you can have clean code, a good prompt, and a well-configured model, yet still produce compromised output if you do not carefully control the content fed into the retrieval pipeline.
Supply Chain Risk Moves Upstream to Models and Data
Software supply chain risk used to focus on packages and build pipelines. With AI, the supply chain now includes models, embeddings, fine-tuning data, evaluation sets, prompt templates, tool definitions, and retrieval content. If you do not control where these come from or track changes over time, you are taking on risks you might not even be measuring.
This is why AI security guidance treats model- and data-supply chain problems as core risks, not rare edge cases. When behavior can change with a model update or a poisoned content library, the risk is beyond the code. It affects what the system says and does, and sometimes it happens quietly, without triggering any alerts your team would normally rely on (OWASP, n.d.).
Supply Chain Risk Now Includes Models and Data
Denial-of-service is no longer just about being offline. Even with AI, your system can be running but still unavailable due to high costs. Long prompts, agent loops, repeated tool calls, and heavy retrieval can quickly turn a useful feature into a budget problem. This can happen by accident or through abuse, which is why model denial-of-service is now its own risk category in AI security discussions (OWASP, n.d.). Teams that focus only on uptime are missing part of the problem.
When Cost Becomes the Outage
The main change in thinking is recognizing that you should not rely on the model as your primary security layer. Large language models can support policy, but they cannot be the only gatekeepers because they are designed to be flexible and responsive. The most important controls are outside the model: authorization checks at retrieval, strict limits on tool and data access, careful handling of secrets before they enter a prompt, and monitoring that treats prompts, retrieval calls, and tool use as security events worth logging and reviewing.
Threat modeling also needs to cover AI-specific risks. Traditional threat modeling remains useful, but AI introduces new attack patterns that require special attention. That is why dedicated knowledge bases exist to catalog adversarial behaviors unique to AI systems and their applications (MITRE, n.d.). If your threat model does not include prompt injection, indirect injection through retrieval, and data exposure through agent behavior, it has important gaps.
AI Prompt Injection Security Needs Outside Controls
AI does not replace your current security program. It pushes it in new directions. The attack surface shifts from strict parsers to natural language, from fixed logic to more flexible, unpredictable behavior, and from narrow input channels to blended context windows and content pipelines. If you treat AI as just another feature, you will secure the framework but miss what is actually guiding the system.
The safest way forward is easy to describe but hard to do. Treat AI as a new interface to your data and capabilities, and wrap it in the same strict controls you would use for any powerful interface. This is now the newest attack surface your system exposes, and it should be treated that way from the beginning.
References
Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023). Not what you’ve signed up for, Compromising real-world LLM-integrated applications with indirect prompt injection. https://arxiv.org/abs/2302.12173
MITRE. (n.d.). ATLAS™, Adversarial Threat Landscape for Artificial-Intelligence Systems. https://atlas.mitre.org/
National Institute of Standards and Technology. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf
National Institute of Standards and Technology. (2024). Artificial Intelligence Risk Management Framework, Generative AI Profile (NIST AI 600-1). https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
OWASP. (n.d.). OWASP Top 10 for Large Language Model Applications. https://owasp.org/www-project-top-10-for-large-language-model-applications/
