Prompt-injection defense starts with permission boundaries, not instruction detection

Even in chat without tools, there is a risk that commands entered during input will overwrite the original instructions. In agents that read emails, search results, shared documents, and web pages, attackers can write commands into acquired data without directly conversing with the user. If the model treats this as an instruction rather than data, it may lead to data exfiltration or unintended operations.

AgentDojo prepared 97 normal tasks and 629 security test cases in environments simulating email, online banking, and travel booking. It reports that state-of-the-art models fail on some tasks even without attacks, and while existing attacks and defenses break or protect parts of safety, they are not universal. [1]

First, we take stock of assets and paths rather than attack strings. We map onto a single flow which model inputs external data enters, which secrets that model can read, and which tools can change its state. We distinguish whether a normal task simply failed or if an attack input led to unauthorized reading or operation, and exclude paths where the latter reproduces from the scope of public functions. [1]

Blacklists of prohibited words, fixed warnings, and attack examples are weak against commands with varied expressions or new data sources. If external documents are concatenated directly into the system prompt, the trust boundary disappears in the text. First, developer commands, user input, search results, and tool outputs must be handled with different types and paths.

StruQ separated prompts and user data into two channels and combined them with a model specially trained to ignore commands on the data side. It reports that in evaluations including at least 15 types of prompt injection, it significantly improved resilience while keeping the impact on practicality low. [2] However, it requires dedicated learning and is not a countermeasure that simply adds delimiters to general APIs.

The effect of structural separation is confirmed through controlled experiments that assign the same content to trusted commands and untrusted data. Passing requires satisfying both conditions: not following commands on the data side and answering legitimate data questions. Since a rise in rejection rate alone allows for the alternative explanation of functionality loss rather than safety improvement, we treat attack prevention and normal business success as separate metrics. [2]

Even if the model reads attack text, the impact can be minimized if it cannot reach secrets in read-only mode and cannot execute external sends or purchases. We whitelist the user, target, operation, amount, and destination for each tool, granting short-lived permissions only when necessary. If acquired text requests new tool permissions, we do not increase them automatically.

NIST AI 100-2e2025 organizes attack classifications and mitigations for generative AI, including evasion, poisoning, privacy, and misuse, and addresses the limitations of existing mitigations. [3] Following this approach, prompt injection can be threat-modeled not as a flaw of a single model, but as an attack path passing through data acquisition, inference, tools, output, and monitoring.

In permission testing, we verify not what the model answered, but whether unauthorized operations were rejected at the execution layer. If any of the user, target, operation, or destination is outside the permitted scope, it must be rejected even if the model explains it as legitimate. We make it a porting condition when moving the execution base to a different product that the damage operation does not hold under controlled conditions without model defense. [3]

StruQ evaluates command-data separation including dedicated learning, and the NIST document provides classifications of attacks and mitigations, but neither certifies that multiple tool chains in individual enterprises are safe. [2][3] Therefore, we investigate not only the permission tables of each tool but also the destinations reached when permissions are linked.

The following is a hypothetical inspection example. If one operation can read customer names, another can create drafts for external parties, and yet another can reuse approved sends, the impact after chaining changes even if each is limited individually. In real environments, before giving attack text to the model, we enumerate established paths through tests that simulate only state transitions.

Case records chronologically log the initial untrusted input, data obtained along the way, added permissions, and the final changed external state with timestamps. Even if rejections at each stage are correct, if the same endpoint is reached via retry or switching to another tool, it is a residual path. We use the sequence from input to asset as the unit of reproduction, not the model name.

The operational recommendation is to create a path diagram from the assumed entry point to the protected asset and choose a stop point that definitely halts at least one location. Stop points are not placed only on model rejections but are chosen from manageable boundaries where changes cannot be made from model output and can be mechanically determined, such as credential issuance, target selection, external sending, and state change.

We do not defend all stages with the same strength but compare the business loss when stopped versus the damage when passed. For example, we test hypotheses of degradation such as allowing browsing but stopping external sends, or allowing reservation creation but stopping confirmation. This is not a universal configuration but a procedure to determine residual paths acceptable for each business.

In the publication decision, we verify not how many known attacks were stopped, but whether there are independent stop points on all known paths to important assets. If new paths are added, permissions changed, or tool integrations added, previous passing status is not inherited as is. We do not assume untested paths are safe but invalidate the corresponding state changes before testing.

• PATH: Uniquely record the sequence of operations and intermediate states from untrusted input to the protected asset
• COMPOSITION: Enumerate reading, sending, and modification capabilities that arise only when combining individual permissions
• CUT: Match stop points enforceable outside the model with business functions that can remain when stopped
• RESIDUAL: Return to publication conditions the reachable paths remaining via detour, retry, or switching to another tool

Evaluations measure both normal task success and attack prevention simultaneously. Even if defense is strong, it is unusable if it rejects all legitimate requests. We separate entry paths such as direct input, acquired documents, tool return values, and in-image text, and test even under conditions where the attacker can paraphrase after knowing the defense.

For high-impact operations, we show the target and differences to the user before execution and obtain explicit approval from the person themselves, not reasons derived from external data. Logs retain raw input, trust classification, model judgment, tool calls, rejections, and approvals. The completion condition for defense includes invalidating credentials upon anomaly, stopping processing, and being able to search for affected operations.

Post-change attack testing does not end with replaying known text but proceeds by varying entry paths and expressions. If high-impact operations hold without approval, or if the impact scope cannot be identified from audit logs, we invalidate the credentials of the relevant tool and revert to read-only. We separate the person responsible for resumption from the person responsible for preserving evidence, and do not allow the modifier to declare passing alone.

• Trust boundary: Record developer commands, user input, acquired data, and tool return values as separate paths
• Least privilege: Restrict targets, operations, and destinations at the execution layer, and do not increase permissions from external text
• Dual evaluation: Independently judge attack prevention and normal task success in the same test
• Incident response: Test stopping, credential invalidation, searching for affected operations, and reverting to read-only