Attack Type

Extraction Vulnerabilities

Techniques for extracting sensitive information from models

97 matching entries

Extraction research results

97 entries

EchoLeak Zero-Click Data Exfiltration

Updated 9/30/2025

A zero-click indirect prompt injection vulnerability, CVE-2025-32711, existed in Microsoft 365 Copilot. A remote, unauthenticated attacker could exfiltrate sensitive data from a victim's session by sending a crafted email. When Copilot later processed this email as part of a user's query, hidden instructions caused it to retrieve sensitive data from the user's context (e.g., other emails, documents) and embed it into a URL. The attack chain involved bypassing Microsoft's XPIA prompt injection…

EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM System

Source: arXiv

Logit Leakage Model Clone

Updated 12/9/2025

Large Language Model (LLM) inference APIs that expose top-k logits or log-probabilities are vulnerable to model extraction and cloning. An attacker can execute a two-stage attack to replicate the proprietary model without access to weights, gradients, or training data. First, by submitting fewer than 10,000 random queries and aggregating the returned unrounded logits, the attacker recovers the model's output projection matrix using Singular Value Decomposition (SVD). Second, the attacker…

Clone What You Can't Steal: Black-Box LLM Replication via Logit Leakage and Distillation

Affects: GPT-3.5, Mistral 7B

Source: arXiv

Multi-Agent Compositional Leak

Updated 1/14/2026

Multi-agent Large Language Model (LLM) systems are vulnerable to compositional privacy leakage, a flaw where sensitive information is exposed through the aggregation of individually benign responses from distinct agents. In distributed architectures where data is siloed (e.g., distinct agents handling HR, Finance, and IT logs), individual agents lack a global view of the user’s accumulated knowledge or the sensitive attributes derivable from cross-agent data combinations. An attacker can…

The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration

Affects: Qwen 3 32B, Gemini 2.5 Pro, GPT-5

Source: arXiv

Prompt Injection Alignment Bypass

Updated 12/8/2025

Large Language Models (LLMs) integrated with external retrieval mechanisms (e.g., Retrieval-Augmented Generation (RAG), web search, or email processing) are vulnerable to Indirect Prompt Injection. This vulnerability occurs when an LLM consumes input from untrusted external sources—such as websites, code repositories, or incoming emails—that contain embedded adversarial prompts. Unlike direct injection, where the user attacks the model, here the "poisoned" data is retrieved by the system…

Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs

Source: arXiv

KV-Cache Sharing Timing Side-channel

Updated 1/14/2026

Multi-tenant Large Language Model (LLM) inference systems utilizing global Key-Value (KV) cache sharing are vulnerable to a timing side-channel attack. By measuring the Time-To-First-Token (TTFT) latency of crafted API requests, an unprivileged remote attacker can determine if specific token sequences have been previously processed and cached by the system for other users. This observable timing difference between cache hits (low TTFT) and cache misses (high TTFT) allows for the token-by-token…

Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference

Affects: Phi-4 14B, Qwen 3 30B-A3B, Qwen 3 32B +3 more

Source: arXiv

Activation Steering Leaks PII

Updated 7/14/2025

Large Language Models (LLMs) are vulnerable to activation steering attacks that bypass safety and privacy mechanisms. By manipulating internal attention head activations using lightweight linear probes trained on refusal/disclosure behavior, an attacker can induce the model to reveal Personally Identifiable Information (PII) memorized during training, including sensitive attributes like sexual orientation, relationships, and life events. The attack does not require adversarial prompts or…

PII Jailbreaking in LLMs via Activation Steering Reveals Personal Information Leakage

Affects: Gemma 2 9B, GLM 9B, GPT-4 +4 more

Source: arXiv

Agent Policy Hacking

Updated 9/7/2025

LLM-powered agentic systems that use external tools are vulnerable to prompt injection attacks that cause them to bypass their explicit policy instructions. The vulnerability can be exploited through both direct user interaction and indirect injection, where malicious instructions are embedded in external data sources processed by the agent (e.g., documents, API responses, webpages). These attacks cause agents to perform prohibited actions, leak confidential data, and adopt unauthorized…

Security challenges in ai agent deployment: Insights from a large scale public competition

Affects: Claude 3.5 Sonnet, Claude 3.7 Sonnet, Command R +11 more

Source: arXiv

Enterprise Multi-Turn Data Exfiltration

Updated 7/28/2025

Large Language Model (LLM) systems integrated with private enterprise data, such as those using Retrieval-Augmented Generation (RAG), are vulnerable to multi-stage prompt inference attacks. An attacker can use a sequence of individually benign-looking queries to incrementally extract confidential information from the LLM's context. Each query appears innocuous in isolation, bypassing safety filters designed to block single malicious prompts. By chaining these queries, the attacker can…

Multi-Stage Prompt Inference Attacks on Enterprise LLM Systems

Affects: GPT-2, GPT-3, GPT-4 +1 more

Source: arXiv

DNA Model Pathogen Synthesis

Updated 6/11/2025

DNA language models, such as the Evo series, are vulnerable to jailbreak attacks that coerce the generation of DNA sequences with high homology to known human pathogens. The GeneBreaker framework demonstrates this by using a combination of carefully crafted prompts leveraging high-homology non-pathogenic sequences and a beam search guided by pathogenicity prediction models (e.g., PathoLM) and log-probability heuristics. This allows bypassing safety mechanisms and generating sequences exceeding…

GeneBreaker: Jailbreak Attacks against DNA Language Models with Pathogenicity Guidance

Affects: Evo1 7B, Evo2 1B, Evo2 7B +2 more

Source: arXiv

Gradient-Based Privacy Jailbreak

Updated 5/31/2025

Large Language Models (LLMs) are vulnerable to a novel privacy jailbreak attack, dubbed PIG (Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization). PIG leverages in-context learning and gradient-based iterative optimization to extract Personally Identifiable Information (PII) from LLMs, bypassing built-in safety mechanisms. The attack iteratively refines a crafted prompt based on gradient information, focusing on tokens related to PII entities, thereby…

PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization

Affects: Claude 3.5 Sonnet, GPT-4o, Llama 2 7B Chat +3 more

Source: arXiv

Related research categories