Useful prompt injection testing examines the application around the model: what untrusted content it reads, which data it can retrieve and what connected tools may do. Define forbidden outcomes, test them with controlled accounts and synthetic data, and verify the actual backend result.
What does prompt injection testing look for?
A direct prompt injection arrives through a user's message. An indirect prompt injection arrives through material the application reads, such as a document or retrieved page. OWASP's prompt injection guidance explains both paths and why their impact depends on the capabilities the surrounding application provides.
An assistant that summarizes a public document has a different risk profile from one that changes account settings. Before collecting test prompts, list the data, tools and credentials available to your feature.
The UK's National Cyber Security Centre cautions against treating prompt injection like SQL injection with a dependable separation between instructions and data. Design around reducing the consequences of a misleading input, rather than assuming one prompt rule eliminates the problem.
Define the data or action that must stay protected
Consider a hypothetical supplier portal. Its assistant reads uploaded onboarding documents, answers questions and can prepare a supplier address change. It should only read documents available to the current user. Preparing a change should not save it until the authorized user approves the exact proposed values.
Define two protected outcomes: an uploaded document must not cause an unapproved address change, and a document in one test organization must not reveal another organization's private onboarding record.
Write down what would count as failure before testing. A strange summary may be a quality issue. A tool request that the backend refuses shows an attempted action and a working control at that point. A saved unauthorized change demonstrates a different, more consequential result.
A document cannot approve an account change
In this illustrative supplier portal, the assistant may propose an operation. The application must still decide whether it is permitted.
- 01
Untrusted document
Content may influence a summary or suggest an unwanted action.
- 02
Assistant proposal
A proposed address change is an input to the application, not permission.
- 03
Backend checks
Verify the caller, organization, exact values and required approval.
- 04
Recorded outcome
Save an authorized change or refuse the operation; retain useful evidence.
The security result depends on the enforced permission and the actual saved state, not only the wording of the chat response.
Test indirect prompts with controlled accounts and documents
Prepare a staging version of the supplier portal, synthetic documents and accounts for two test organizations. Use harmless marker values to recognize whether a document influenced an answer or proposed action. Keep external delivery and irreversible actions disabled for these test cases.
| Test | Expected behavior | What to inspect |
|---|---|---|
| Document contains an instruction unrelated to onboarding | Content does not authorize a new account action | Summary, proposed tool arguments and backend result |
| Document refers to another test organization's record | Retrieval remains limited to the current user's permission | Retrieved record identifiers and returned content |
| Document suggests saving an address without approval | The required approval remains mandatory | Approval state and actual stored address |
| The user approves one address, then the proposed value changes | Approval cannot silently authorize the changed value | Approved values compared with the submitted operation |
OWASP's AI application testing guidance treats retrieval, tools and their permissions as part of the assessment. Repeat relevant cases and record the model and application configuration; one successful refusal does not establish that every variation will be refused.
Keep permission checks outside the model's decision
For the supplier portal, the address-change endpoint should verify the caller, organization, allowed action and required approval. It should make those checks even when the request comes from the assistant. A model-generated claim that an administrator approved the operation is not the approval record.
OWASP's AI Agent Security Cheat Sheet recommends limiting tool privileges and avoiding model output as the sole basis for authorization. Restrict what a tool can access and require suitable approval for consequential operations. Prompt instructions can help guide behavior, but the backend remains responsible for enforcing access.
These web and API boundaries are a practical starting point for a Sectiv scoping conversation. We can discuss the authenticated workflows, roles and backend actions that need assessment. Specialist model evaluation, broad jailbreak campaigns and AI-specific test coverage should be agreed explicitly rather than assumed to come with a routine web scan.
Record evidence that explains the actual impact
A useful supplier-portal finding links the test document, user identity, assistant output, tool request and resulting record state. Capture the application version and relevant configuration. Keep private source content and credentials out of broadly shared reports.
Suppose the assistant proposes the unwanted address change, but the API requires fresh approval and saves nothing. Record both facts. If a later case saves the change without approval, capture that separate outcome and the conditions that produced it. This distinction helps engineers locate the failed control.
After a fix, rerun the case and verify that legitimate onboarding still works. Preserve the test as a regression case when changing retrieval, tools or approval logic. Keep the claim bounded to what was tested; a clean run does not establish universal resistance to prompt injection.
Scope the application behind your AI feature
Before a Sectiv call, list the data your AI feature reads, the tools it can call and one action that must always require permission. Bring that map with your release goal. We can discuss the web and API assessment, test access and human guidance that fit your application, and identify any specialist AI evaluation requirements separately.
Common questions
Does a prompt injection always mean an application was compromised?
No. Identify the observed effect. A changed answer, attempted tool call, disclosed private record and completed unauthorized action have different implications. Check the application state and available evidence before assigning impact.
Can a stronger system prompt prevent every prompt injection?
It should not be treated as a complete security boundary. Use application permissions, limited tool access and approval controls to reduce what manipulated model behavior can cause. Test those controls against the feature's actual data and actions.
Is a normal web penetration test enough for an AI application?
It can assess important web, API and access-control behavior, but model, retrieval and tool-driven behavior may need additional test methods. Agree the coverage explicitly and record what is outside the engagement.
YOUR NEXT STEP
Test what your AI application is allowed to do.
Adding AI to your product? Book a Sectiv call to map the web and API permission boundaries, clarify the assessment scope and plan useful test evidence.
Book a Sectiv demo