Indirect Prompt Injection Hijacks Tool-Using Agents

Description

The paper evaluates a reproducible indirect prompt injection issue in ReAct-style LLM agents: untrusted retrieved content can be interpreted as instructions and redirect the agent toward unauthorized tool calls. The authors report that successful attacks correlate with concentrated attention on injected content and evaluate defenses using InjectAgent, AgentDojo, TrojanTools, and a visual prompt-injection benchmark. These are paper-reported findings, not independently verified facts.

Examples

See the primary source for the paper's evaluation and defensive reproduction context: https://arxiv.org/abs/2602.20708 (opens in a new tab)

Impact

Successful indirect prompt injection can compromise agent integrity by overriding the user’s intended workflow, causing unauthorized tool selection or actions. The paper also identifies reliability harm: strict filtering or refusal-based defenses may terminate legitimate multi-step workflows.

Affected Systems

• Tool-augmented ReAct-style LLM agents that ingest untrusted retrieved content
• Agents connected to external tools, retrieval corpora, web pages, emails, or MCP-like servers
• Vision-language agents processing untrusted visual inputs

Mitigation Steps

• Treat retrieved text, tool output, and visual content as untrusted data; isolate it from trusted instructions with clear provenance and privilege boundaries.
• Require authorization and policy checks before consequential tool calls, independent of the model’s generated reasoning.
• Use prompt-injection detection and action filtering, while measuring both attack success rate and legitimate task completion to detect over-refusal regressions.
• For locally hosted white-box models, consider inference-time monitoring of attention anomalies and targeted attention steering such as ICON; validate thresholds and utility on deployment-specific data.
• Evaluate defenses against adaptive and out-of-distribution indirect prompt injection benchmarks, including multimodal cases where applicable.
• Apply least privilege to tools and require confirmation for sensitive or irreversible actions.

Evidence

• Primary source, abstract and §1: https://arxiv.org/abs/2602.20708 (opens in a new tab) — the paper states that malicious instructions embedded in retrieved content can hijack agent execution and cause unauthorized tool calls.
• Primary source, §3 Problem Formulation & Threat Model: https://arxiv.org/html/2602.20708 (opens in a new tab) — the assumed attacker injects through external observations such as retrieval corpora or compromised MCP servers and attempts malicious redirection to an unauthorized tool call; the defense assumes white-box access to model internals.
• Primary source, §5.1 Experimental Setup: https://arxiv.org/html/2602.20708 (opens in a new tab) — the authors evaluate ReAct agents with Qwen, LLaMA, and Mistral, use InjectAgent and AgentDojo, train on TrojanTools, and evaluate visual tasks with a Visual Prompt Injection Benchmark.
• Primary source, Table 2 and §5.2: https://arxiv.org/html/2602.20708 (opens in a new tab) — paper-reported results show undefended average attack success rates of 32.7% for Qwen-3-8B, 35.7% for LLaMA-3.1-8B, and 30.9% for Mistral-8B; the authors report ICON reducing average ASR to 0.4%, 0.3%, and 0.3% respectively. These measurements were not independently verified.
• Primary source, Table 3 and §5.3: https://arxiv.org/html/2602.20708 (opens in a new tab) — the authors report out-of-distribution evaluation on InjectAgent and AgentDojo, with detection rates above 80% on InjectAgent and above 97% on AgentDojo for the tested Qwen and LLaMA backbones; these are paper-reported measurements.
• Primary source, Table 5 and §5.5: https://arxiv.org/html/2602.20708 (opens in a new tab) — the paper reports multimodal evaluation on Qwen3VL, InternVL, and MiniCPM, with an undefended average ASR of 59.0% and ICON average ASR of 2.9%; these results were not independently verified.
• Primary source, §§4.5–4.7: https://arxiv.org/html/2602.20708 (opens in a new tab) — the proposed defense computes attention-entropy features, detects latent traces with a lightweight prober, and suppresses anomalous attention dependencies at inference time rather than terminating the workflow.