Impact

Data security Research

Issues affecting data confidentiality and integrity

9 matching entries out of 171 in this category

Data security research results

9 entries

Visual Confused Deputy Attack

Published 3/1/2026

Analyzed 4/11/2026

A "Visual Confused Deputy" vulnerability exists in Computer-Using Agents (CUAs) that rely on visual perception to execute coordinate-based GUI actions (e.g., click(x,y)). Because the agent's understanding of the system state is entirely dependent on the screenshot provided by the runtime, a compromised runtime or tool can intercept and alter the screenshot pixels before forwarding them to the LLM. By visually swapping the locations of benign and privileged UI elements, an attacker can trick…

Visual Confused Deputy: Exploiting and Defending Perception Failures in Computer-Using Agents

Evaluated models: Claude 3.7 Sonnet

Source: arXiv

Voice Agent Behavioral Bypass

Published 2/1/2026

Analyzed 2/21/2026

Audio Large Language Models (ALLMs) integrated into voice agent systems for high-stakes domains (banking, IT support, logistics) are vulnerable to multimodal adversarial attacks via spoken interaction. Adversaries can exploit the model's inherent compliance and contextual awareness through multi-turn dialogue to bypass authentication safeguards, escalate privileges (e.g., unauthorized credit limit increases), exfiltrate sensitive Personally Identifiable Information (PII), and poison…

Aegis: Towards Governance, Integrity, and Security of AI Voice Agents

Evaluated models: GPT-4o, GPT-4o Mini, Gemini 1.5 Pro +4 more

Source: arXiv

AI Browser Indirect Injection

Published 10/1/2025

Analyzed 12/30/2025

Agentic AI browsers and LLM-powered browser extensions are vulnerable to indirect prompt injection via the processing of untrusted web content. The vulnerability arises when the AI agent ingests the Document Object Model (DOM), including hidden elements, HTML comments, metadata, and accessibility labels, into its context window to perform tasks such as page summarization or autonomous navigation. Because the LLM cannot distinguish between system instructions and untrusted external data, an…

In-browser llm-guided fuzzing for real-time prompt injection testing in agentic AI browsers

Evaluated models: GPT-4, Llama 3.1 70B, Llama 3.3 70B

Source: arXiv

LLM Multi-Agent IP Leakage

Published 5/1/2025

Analyzed 5/31/2025

Large Language Model (LLM)-based Multi-Agent Systems (MAS) are vulnerable to intellectual property (IP) leakage attacks. An attacker with black-box access (only interacting via the public API) can craft adversarial queries that propagate through the MAS, extracting sensitive information such as system prompts, task instructions, tool specifications, number of agents, and system topology.

IP Leakage Attacks Targeting LLM-Based Multi-Agent Systems

Evaluated models: GPT-4o, GPT-4o Mini, Llama 3.1 70B +2 more

Source: arXiv

Embodied LLM Misaligned Actions

Published 7/1/2024

Analyzed 12/29/2024

Embodied Large Language Models (LLMs) are vulnerable to manipulation via voice-based interactions, leading to the execution of harmful physical actions. Attacks exploit three vulnerabilities: (1) cascading LLM jailbreaks resulting in malicious robotic commands; (2) misalignment between linguistic outputs (verbal refusal) and physical actions (command execution); and (3) conceptual deception, where seemingly benign instructions lead to harmful outcomes due to incomplete world knowledge within…

BadRobot: Manipulating Embodied LLMs in the Physical World

Evaluated models: BERT, GPT-3.5 Turbo, GPT-4 Turbo +2 more

Source: arXiv

LLM Robot Bias & Violence

Published 6/1/2024

Analyzed 4/12/2025

Large Language Models (LLMs) used to control robots exhibit biases leading to discriminatory and unsafe behaviors. When provided with personal characteristics (e.g., race, gender, disability), LLMs generate biased outputs resulting in discriminatory actions (e.g., assigning lower rescue priority to certain groups) and accept or deem feasible dangerous or unlawful instructions (e.g., removing a person's mobility aid).

Llm-driven robots risk enacting discrimination, violence, and unlawful actions

Evaluated models: GPT-3.5, GPT-3.5 Turbo, GPT-4 +1 more

Source: arXiv

Adversarial Speech Jailbreak

Published 5/1/2024

Analyzed 12/29/2024

Multimodal Large Language Models (LLMs) processing speech input are vulnerable to adversarial attacks. Imperceptible perturbations added to audio input can cause the model to generate unsafe or harmful text responses, overriding built-in safety mechanisms. The attacks are effective even with limited knowledge of the model's internal workings, demonstrating transferability across different models.

SpeechGuard: Exploring the adversarial robustness of multimodal large language models

Evaluated models: Flan-T5 XL, Llama 7B, Llama 2 13B Chat +2 more

Source: arXiv

MedMMLM Cross-Modality Jailbreak

Published 5/1/2024

Analyzed 1/26/2025

Medical Multimodal Large Language Models (MedMLLMs) are vulnerable to cross-modality attacks. Attackers can craft "mismatched malicious attacks" (2M-attacks) by providing MedMLLMs with image-text pairs where the image modality and/or anatomical region do not match the textual query, causing the model to generate incorrect or harmful responses. These attacks can be further optimized ("optimized mismatched malicious attacks"—O2M-attacks) using multimodal cross-optimization (MCM) techniques to…

Cross-Modality Jailbreak and Mismatched Attacks on Medical Multimodal Large Language Models

Evaluated models: CheXagent, LLaVA Med, Med-Flamingo +2 more

Source: arXiv

Visual Adversarial Tool Abuse

Published 10/1/2023

Analyzed 12/28/2024

A vulnerability exists in multimodal Large Language Models (LLMs) integrated with external tools. Adversarial images, visually indistinguishable from benign images, can manipulate the LLM to execute unintended tool commands, compromising the confidentiality and integrity of user resources. The attack is effective across diverse prompts, remaining stealthy both in the image itself and in the generated text response.

Misusing tools in large language models with visual adversarial examples

Evaluated models: Not reported

Source: arXiv

Related research categories