Persistent Agent Context Injection

Description

Conventional LLM agent architectures suffer from a working memory contamination vulnerability due to indiscriminate memory accumulation. When these agents retrieve external data via tools (e.g., web search, reading emails), the entire raw output is appended directly to their continuous context window. If the external data contains an Indirect Prompt Injection (IPI) payload, the malicious instruction persists in the agent's working memory across its entire multi-step reasoning workflow. This "Attack Persistence" forces the backend LLM to re-process the adversarial instruction at every subsequent decision node, granting the attacker continuous opportunities to hijack the agent's control flow and data flow, overriding the original user intent.

Examples

An agent retains an untrusted tool response in its working context, exposing subsequent decisions to the same injected instruction. AgentSys tests isolating tool work and restricting the structured data returned to the main agent; see the primary study (opens in a new tab).

Impact

• Control-Flow Manipulation: The persistent malicious context alters the agent's execution path, forcing it to invoke unintended tools, modify local files, or send unauthorized requests.
• Data-Flow Manipulation: Injected text poisons the data the agent relies on, corrupting subsequent tool arguments and tricking the agent into leaking sensitive conversation history or environment data.
• Heightened Attack Success Rate (ASR): Because the injection persists in the context window, the probability of a successful attack scales dramatically with the length of the agent's workflow, as the persistent instruction is repeatedly evaluated against the agent's safety alignment.

Affected Systems

• LLM-based agentic frameworks (e.g., standard ReAct implementations) that maintain state by continuously appending raw tool outputs, intermediate reasoning artifacts, and external observations directly into the main planner's context window.

Mitigation Steps

• Hierarchical Memory Isolation: Spawn short-lived, isolated "worker" agents to handle tool invocations. Never append raw tool outputs or subtask reasoning traces directly to the main agent’s working memory.
• Schema-Validated Communication: Require the main agent to declare a strictly typed "intent" schema (e.g., expected JSON fields) before invoking a tool. Have the worker agent extract only the requested data into this schema, and apply a deterministic syntactic gate (e.g., rule-based JSON validation) before admitting the output into the main context.
• Event-Triggered Validation: Implement an independent LLM-based validator to mediate recursive or multi-step tool calls initiated within the worker agent. Restrict the validator’s input to the original user query and compact call metadata, explicitly excluding the raw, untrusted tool outputs to prevent the validator itself from being poisoned.
• Bounded Sanitization and Recovery: If a tool call is denied by the validator, trigger a sanitize-restart loop using a dedicated LLM prompt to strip instruction-like spans (imperatives, role directives) from the untrusted observation before attempting data extraction again. Enforce strict retry limits to prevent denial-of-service via infinite loops.

Evidence

The arXiv submission history (opens in a new tab) dates the first submission to February 7, 2026. Section 6 and Tables 5–7 (opens in a new tab) identify GPT-4o-mini, GPT-4o, GPT-5.1, Claude-3.7-Sonnet, Gemini-2.5-Pro, and Qwen2.5-7B-Instruct. GPT-4o-mini is distinct from o4-mini. Results are paper-reported, not independently reproduced here.