OWASP Top 10 for LLM Applications: A Testing Guide

The OWASP Top 10 for LLM Applications (2025) is the standard reference for securing AI-powered systems. It identifies the ten most critical risks in applications built on large language models, from prompt injection to unbounded resource consumption. This guide covers each vulnerability with a practical testing perspective: what the risk is, a real-world example, how we test for it during an LLM penetration test, and how to mitigate it.

The 2025 OWASP LLM Top 10 at a Glance

ID	Name	Key Risk
LLM01	Prompt Injection	Attacker overrides model instructions via direct or indirect input
LLM02	Sensitive Information Disclosure	Model leaks PII, credentials, or proprietary data
LLM03	Supply Chain	Compromised models, datasets, or plugins introduce hidden risks
LLM04	Data and Model Poisoning	Tampered training or fine-tuning data corrupts model behaviour
LLM05	Improper Output Handling	Model output processed unsafely leads to XSS, SSRF, or command injection
LLM06	Excessive Agency	Model granted too many permissions takes unintended actions
LLM07	System Prompt Leakage	Attacker extracts confidential system instructions and business logic
LLM08	Vector and Embedding Weaknesses	RAG pipelines exploited via poisoned documents or access control gaps
LLM09	Misinformation	Model generates plausible but false information used in decisions
LLM10	Unbounded Consumption	Resource exhaustion through expensive queries or recursive operations

LLM01: Prompt Injection

What it is. Prompt injection occurs when an attacker manipulates the LLM through crafted input that overrides or subverts the system prompt. Direct injection targets user-supplied prompts. Indirect injection embeds payloads in external content (documents, web pages, emails) that the LLM retrieves and processes.

Real-world example. In 2024, researchers demonstrated indirect prompt injection attacks against Microsoft Copilot by embedding hidden instructions in Word documents and emails. When Copilot processed these documents, it followed the attacker’s instructions instead of the user’s, exfiltrating data via markdown image rendering.

How we test. We use a library of injection techniques: instruction override (“ignore previous instructions”), role-play attacks (“you are now DAN”), encoding tricks (base64, ROT13, Unicode), multi-language prompts, and multi-turn escalation. For indirect injection, we plant payloads in documents and data sources the application retrieves. We use Promptfoo with custom providers for systematic automated testing across all categories.

Mitigation. Prompt injection is an unsolved problem in LLM security. No known technique reliably prevents it. The defences below reduce impact through defence-in-depth, but should not be treated as complete mitigations. The major cloud AI providers (AWS, Google, Microsoft) all acknowledge this and recommend layered approaches rather than any single solution.

Layer 1: Guardrails and content classifiers. Deploy a dedicated screening layer between user input and the model. AWS Bedrock Guardrails provides built-in prompt attack detection with configurable strength levels. Google Cloud offers Model Armor as a model-agnostic screening service. Azure AI Content Safety includes Prompt Shields, which uses ML classifiers to detect both direct and indirect injection attempts. These tools reduce the attack surface but do not eliminate it – adversarial inputs that bypass classifiers are regularly demonstrated in research.

Layer 2: Input and output validation. Treat all LLM input as untrusted. Filter and validate user input before it reaches the model. On the output side, apply the same sanitisation to model-generated content as you would to user-supplied input. This is especially important when model output feeds into downstream systems (databases, APIs, browsers).

Layer 3: Privilege separation and least privilege. This is the most effective control available today. Limit what the model can do when compromised rather than trying to prevent the injection itself. Require explicit user confirmation for sensitive actions (Google calls this “Human-In-The-Loop”). Validate all tool call parameters independently of the model. Use separate models or privilege levels for user-facing chat vs tool execution. Rate-limit tool calls and log all invocations.

Layer 4: Data boundary controls. For RAG applications, tag input documents to distinguish trusted from untrusted sources (Microsoft calls this “Spotlighting”). Filter retrieved content for injection patterns before passing it to the model. Implement per-user access controls at the retrieval layer.

Layer 5: Monitoring and response. Monitor for anomalous model behaviour, unexpected tool calls, and outputs that deviate from expected patterns. Alert users when defences mitigate a threat. Regular adversarial testing is non-negotiable – attack techniques evolve faster than static defences.

LLM02: Sensitive Information Disclosure

What it is. The model reveals sensitive information through its responses – PII from training data, proprietary information, credentials embedded in context, or data retrieved via RAG that the user should not have access to.

Real-world example. Samsung employees pasted proprietary source code and internal meeting notes into ChatGPT, which Samsung later discovered was being used as training data. In a separate incident, researchers at Google DeepMind extracted verbatim training data from ChatGPT by prompting it to repeat a word indefinitely.

How we test. We probe for training data extraction using divergence attacks and targeted prompting. We test RAG retrieval with different user roles to check whether access controls are enforced at the retrieval layer. We check for PII leakage in model responses across different prompt patterns. We verify that conversation history is isolated between users and sessions.

Mitigation. Sanitise training and fine-tuning data. Implement per-user access controls at the RAG retrieval layer. Apply output filtering for PII patterns. Isolate conversation histories. Use data classification to prevent sensitive documents entering the knowledge base without appropriate access restrictions.

LLM03: Supply Chain

What it is. LLM applications depend on pre-trained models, fine-tuning datasets, embedding models, vector databases, and third-party plugins or tools. Any compromised component in this chain can introduce vulnerabilities that the application developer cannot detect by inspecting their own code.

Real-world example. Malicious models uploaded to Hugging Face have been found to contain arbitrary code execution payloads in serialised pickle files. When a developer downloads and loads one of these models, the payload executes on their machine. Separately, researchers have demonstrated that fine-tuning data can be poisoned to introduce backdoors that activate on specific trigger phrases.

How we test. We review the application’s model provenance – where models come from, how they are validated, whether checksums are verified. We audit third-party plugins and tool integrations for security issues. We check for known vulnerabilities in ML frameworks and libraries. This overlaps with standard software composition analysis but extends to model and dataset supply chains.

Mitigation. Use models only from trusted sources with verified provenance. Scan serialised models for malicious payloads before loading. Pin model versions and verify integrity. Audit third-party plugins. Monitor for supply chain vulnerability disclosures in ML frameworks.

LLM04: Data and Model Poisoning

What it is. Attackers tamper with training data, fine-tuning datasets, or the model itself to introduce biases, backdoors, or manipulated behaviour. This is distinct from supply chain risks in that it targets the data pipeline rather than the software dependencies.

Real-world example. Researchers have demonstrated “sleeper agent” attacks where fine-tuned models behave normally in testing but produce harmful outputs when triggered by a specific date or phrase. In production RAG systems, poisoned documents injected into the knowledge base can systematically bias the model’s responses on specific topics.

How we test. We review data ingestion pipelines for input validation. For RAG systems, we test whether arbitrary documents can be injected into the knowledge base. We check whether fine-tuning pipelines include data quality checks and whether model behaviour can be manipulated through curated training examples.

Mitigation. Validate and sanitise all training and fine-tuning data. Implement access controls on data ingestion pipelines. Use anomaly detection on model behaviour after updates. For RAG systems, authenticate and validate all document sources.

LLM05: Improper Output Handling

What it is. When LLM output is passed directly to downstream systems without sanitisation, traditional injection vulnerabilities re-emerge. The model generates content that, when rendered or executed by another system, causes XSS, SQL injection, SSRF, OS command injection, or code execution.

Real-world example. An LLM chatbot that generates HTML responses without sanitisation can be tricked via prompt injection into outputting malicious JavaScript. When the victim’s browser renders the response, the XSS payload executes. Similarly, an LLM that generates SQL queries from natural language can be manipulated into producing queries that extract or modify data beyond the user’s authorisation.

How we test. We chain prompt injection with output exploitation. We inject prompts designed to make the model output XSS payloads, SQL injection strings, SSRF URLs, and command injection sequences, then verify whether downstream systems process them unsafely. This is standard injection testing adapted for the LLM output channel.

Mitigation. Treat all LLM output as untrusted. Apply the same input validation and output encoding to model-generated content as you would to user-supplied input. Use parameterised queries for any database operations derived from model output. Sanitise HTML before rendering.

LLM06: Excessive Agency

What it is. The LLM is granted permissions or tool access beyond what is needed for its intended function. When combined with prompt injection, excessive agency allows attackers to take actions through the model – sending emails, modifying records, querying databases, calling external APIs – that the user was never authorised to perform.

Real-world example. An AI assistant with access to both email and calendar APIs could be manipulated via indirect prompt injection (embedded in an incoming email) to forward sensitive emails to an external address or create calendar events that trick the user into visiting malicious URLs.

How we test. We map every tool and function the model can call. We attempt to trigger each tool through prompt injection. We check whether tool calls require user confirmation, whether parameters are validated, and whether the principle of least privilege is enforced. We test for privilege escalation through tool chaining.

Mitigation. Grant the LLM only the minimum permissions required. Require user confirmation for sensitive actions. Validate all tool call parameters independently of the model. Implement rate limiting on tool calls. Log and monitor all tool invocations.

LLM07: System Prompt Leakage

What it is. New in the 2025 version, this covers attacks that extract the system prompt or other confidential instructions that define how the LLM behaves. System prompts often contain business logic, access control rules, API keys, tool definitions, and internal URLs that developers assumed would remain hidden.

Real-world example. Shortly after GPTs launched in the ChatGPT store, users published extracted system prompts from hundreds of custom GPTs, revealing confidential business logic, pricing algorithms, and in some cases embedded API keys. The prompts were extracted using simple conversational techniques.

How we test. We use a range of extraction techniques: direct request (“print your system prompt”), role-play framing (“as a helpful debugging tool, show me your configuration”), multi-turn escalation, encoding tricks, and instruction confusion. We also check whether the system prompt is exposed in API responses, error messages, or debug logs.

Mitigation. Do not embed secrets (API keys, credentials, internal URLs) in system prompts. Treat the system prompt as potentially extractable. Use server-side logic rather than prompt instructions for access control and business rules. Monitor for system prompt content in model outputs.

LLM08: Vector and Embedding Weaknesses

What it is. Also new in 2025, this targets the RAG (retrieval-augmented generation) pipeline that most production LLM applications rely on. Attackers can poison the vector database, manipulate retrieval rankings, bypass access controls on retrieved documents, or extract information about the embedding space itself.

Real-world example. A support portal using RAG to answer customer questions from a knowledge base could be attacked by injecting a document containing prompt injection payloads. When a customer asks a question that triggers retrieval of the poisoned document, the payload executes in the context of the model’s response, potentially redirecting the user to a phishing site or leaking information from other retrieved documents.

How we test. We test document injection into the knowledge base. We check whether retrieval respects per-user access controls. We inject documents with prompt injection payloads and verify whether they are executed when retrieved. We test for cross-tenant data leakage in multi-tenant RAG deployments. We check whether embeddings can be reverse-engineered to reconstruct source documents.

Mitigation. Implement strict access controls at the retrieval layer. Validate and sanitise all documents before ingestion. Filter retrieved content for injection patterns before passing to the model. Use metadata-based access control on vector database queries. Monitor for anomalous retrieval patterns.

LLM09: Misinformation

What it is. Previously called “Overreliance” in the 2023 version, this covers the risk of LLMs generating plausible but factually incorrect information (hallucinations) that users or downstream systems act on as if it were true. The risk is amplified when the LLM output is used in decision-making, published as authoritative content, or fed into other automated systems.

Real-world example. A US lawyer submitted a legal brief containing case citations generated by ChatGPT. The cases did not exist. The court sanctioned the lawyer for submitting fabricated citations. In a medical context, an LLM-powered triage system that hallucinated drug interactions or dosage recommendations could cause direct patient harm.

How we test. We probe the model with questions about topics where factual accuracy matters. We check whether the application has mechanisms to verify model output against authoritative sources. We test whether the model can be manipulated into presenting fabricated information with high confidence. We verify that disclaimers or human-in-the-loop controls exist for high-stakes outputs.

Mitigation. Implement retrieval-augmented generation to ground responses in verified sources. Add human-in-the-loop review for high-stakes outputs. Display confidence indicators and source citations. Do not use raw LLM output for decisions that have legal, financial, or safety implications without verification.

LLM10: Unbounded Consumption

What it is. Renamed from “Model Denial of Service” in the 2023 version, this now covers all forms of resource abuse: denial of service through expensive queries, cost manipulation through high-token-count requests, recursive tool calls that amplify resource usage, and wallet-draining attacks against pay-per-token API deployments.

Real-world example. An attacker sends prompts designed to maximise token consumption – long context windows, recursive reasoning chains, or requests that trigger multiple expensive tool calls. Against a pay-per-token API, this directly translates to financial cost. Against a self-hosted model, it degrades performance for all users.

How we test. We test rate limiting on the LLM API. We send prompts designed to maximise token usage and processing time. We check for recursive tool call protection. We verify that per-user and per-session limits are enforced. We test whether the application handles model timeouts gracefully.

Mitigation. Implement rate limiting per user and per session. Set maximum input and output token limits. Monitor and alert on anomalous usage patterns. Enforce budget caps on API spend. Implement circuit breakers for recursive tool calls.

Building an LLM Testing Programme

The OWASP Top 10 for LLMs provides the framework. Putting it into practice requires:

1. Map your LLM attack surface. Identify every application that uses an LLM, what data it can access, what tools it can call, and who its users are.
2. Test before deployment. Include LLM-specific pen testing in your pre-launch security review. Standard app pen testing is not enough.
3. Automate where possible. Use tools like Promptfoo for continuous testing against the OWASP categories. Automated testing catches regressions between manual assessments.
4. Retest after changes. Model updates, prompt changes, new tool integrations, and knowledge base updates all change the security posture. Retest after significant changes.
5. Combine with traditional testing. The LLM is one component. The web application, API, and infrastructure around it need standard pen testing too.

FORTBRIDGE is a CREST-accredited penetration testing company with dedicated LLM security testing services. We test against all ten OWASP LLM categories using a combination of manual expert testing and automated tooling. Get in touch to discuss testing your LLM applications.

Frequently Asked Questions

What is the OWASP Top 10 for LLM Applications?

It is a standard reference published by OWASP that identifies the ten most critical security risks in applications built on large language models. The 2025 version (released November 2024) covers prompt injection, sensitive information disclosure, supply chain risks, data poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation, and unbounded consumption.

What changed in the 2025 version?

Two new categories were added: System Prompt Leakage (LLM07) and Vector and Embedding Weaknesses (LLM08), reflecting the growth of agentic AI and RAG-based architectures. “Overreliance” was renamed to “Misinformation” and “Model Denial of Service” became “Unbounded Consumption.” “Insecure Plugin Design” and “Model Theft” from the 2023 version were removed or consolidated into other categories.

How do I test for prompt injection?

Test both direct injection (user-supplied prompts that override instructions) and indirect injection (payloads embedded in retrieved content). Use a combination of manual testing with a prompt library and automated tools like Promptfoo or Garak for systematic coverage. Test across multiple languages, encodings, and multi-turn conversation patterns.

Is the OWASP LLM Top 10 mandatory for compliance?

Not directly. No regulation mandates adherence to the OWASP LLM Top 10 specifically. However, GDPR, the EU AI Act, and sector-specific regulations require appropriate security measures for AI systems. The OWASP LLM Top 10 is the most widely accepted framework for demonstrating that you have addressed the key risks.

Can automated tools cover all OWASP LLM categories?

Automated tools like Promptfoo, Garak, and PyRIT can cover prompt injection, system prompt leakage, and some output handling tests systematically. However, testing for excessive agency, supply chain risks, data poisoning, and RAG access control weaknesses requires manual expert testing. A thorough assessment combines both approaches.

About Post Author