Attack Surface
Security research involving prompt handling and processing
39 matching entries out of 610 in this category
Untrusted document images can redirect vision-language agents across instruction and tool-authorization boundaries. Repeat-After-Me evaluates six victim models on constructed document tasks.
Source: arXiv
Memory-augmented LLM web agents utilizing raw trajectory memory are vulnerable to Environment-injected Trajectory-based Agent Memory Poisoning (eTAMP). Attackers can embed malicious instructions within user-generated web content (e.g., product pages, forum posts). When the agent processes this content during a routine task, the instructions are passively ingested into its raw trajectory memory. During subsequent, entirely separate tasks on different websites, semantic retrieval mechanisms pull…
Source: arXiv
Vision-Language-Action (VLA) models are vulnerable to targeted, low-budget textual perturbations in their natural-language instruction inputs, which can maliciously alter sequential decision-making and downstream physical robotic behavior. Because VLA policies tightly couple language, perception, and control, bounded edits—such as character-level typos, token attribute swaps, or prompt-level uncertainty clauses—propagate through the model's execution trajectory. This allows a black-box…
Source: arXiv
Speech-driven Large Language Models (LLMs) and end-to-end Large Audio-Language Models (LALMs) are vulnerable to inaudible near-ultrasonic prompt injections, a framework dubbed Sirens' Whisper (SWhisper). By exploiting the non-linear response of commodity microphones, attackers can encode structured, phonetically optimized adversarial prompts into the 17–22 kHz near-ultrasonic band. Using regularized channel-inversion pre-compensation, the attacker shapes the waveform to account for microphone…
Source: arXiv
The paper evaluates a reproducible indirect prompt injection issue in ReAct-style LLM agents: untrusted retrieved content can be interpreted as instructions and redirect the agent toward unauthorized tool calls. The authors report that successful attacks correlate with concentrated attention on injected content and evaluate defenses using InjectAgent, AgentDojo, TrojanTools, and a visual prompt-injection benchmark. These are paper-reported findings, not independently verified facts.
Source: arXiv
Large Image Editing Models (LIEMs) supporting vision-prompt editing are vulnerable to Vision-Centric Jailbreak Attacks (VJA). This vulnerability arises from a modality mismatch in safety alignment: while safeguards primarily analyze textual instructions for policy violations, the underlying models are capable of interpreting and executing instructions embedded directly within the visual input (e.g., typographic text drawn on the image, arrows, symbols, or specific markings). An attacker can…
Source: arXiv
Multimodal LLM-based phishing detection systems are vulnerable to indirect prompt injection via "perceptual asymmetry." Attackers can embed hidden instructions within a phishing site's HTML, CSS, URLs, or rendered images that remain imperceptible to human victims but are parsed and executed by the evaluating LLM. This vulnerability allows threat actors to manipulate the LLM's contextual understanding, forcing it to misclassify malicious sites as benign (Legitimate Pretexting), trigger safety…
Source: arXiv
Mobile Large Language Model (LLM) agents operating under the "Screen-as-Interface" paradigm are vulnerable to visual indirect prompt injection and state desynchronization. Agents that rely on unstructured visual data (screenshots) and Accessibility Service APIs to perceive the environment lack a mechanism to distinguish between trusted system UI elements and untrusted content (e.g., web pages, emails, or malicious overlays). An attacker can inject visual cues, fake notifications, or hidden…
Source: arXiv
Multi-modal Large Language Models (MLLMs) are vulnerable to a multi-turn jailbreaking attack that leverages typographic visual prompts combined with conversational context drifting. The vulnerability exists because MLLMs establish trust and context during initial benign interactions, shifting the model's latent representation toward helpfulness and compromising its ability to detect malicious intent in subsequent turns. The attack vector utilizes an image where a harmful request is…
Source: arXiv
Vision-Language Models (VLMs) exhibit a vulnerability to moral judgment flipping, where the model's safety alignment can be bypassed through lightweight, model-agnostic multimodal perturbations. By introducing conflicting textual or visual cues that do not alter the underlying moral context of a scenario, an attacker can coerce the model into reversing its ethical stance (e.g., reclassifying a harmful action from "morally wrong" to "not morally wrong"). This vulnerability exploits the model's…
Source: arXiv