Attack Type

Extraction Research

Techniques for extracting sensitive information from models

98 matching entries

Extraction research results

98 entries

Production LLM Copyright Extraction

Published 1/1/2026

Analyzed 2/21/2026

Production Large Language Models (LLMs) are vulnerable to long-form training data extraction via a two-phase prompt injection attack. This vulnerability allows an attacker to recover substantial portions of memorized, copyrighted text (such as novels) by exploiting the model's autoregressive text completion capabilities. The attack methodology involves two distinct phases: 1. Prefix Completion Probe: The attacker provides a short "seed" sequence (e.g., the first sentence of a book) coupled…

Extracting Books from Production Language Models

Evaluated models: Claude 3.7 Sonnet 20250219, GPT-4.1 2025-04-14, Gemini 2.5 Pro +1 more

Source: arXiv

Cross-Environment Agent Jailbreak

Published 12/1/2025

Analyzed 2/21/2026

Large Language Model (LLM) agents operating in tool-augmented environments are susceptible to "Contextual Fragility" and multi-turn "long-chain" exploitation. Existing safety mechanisms predominantly function on a stateless, atomic paradigm, evaluating individual input-output pairs in isolation. This allows an adversary to orchestrate complex attack trajectories where malicious intent is distributed across multiple, individually benign steps (a "Domino Effect"). Consequently, an attacker can…

DREAM: Dynamic Red-teaming for Evaluating Agentic Multi-Environment Security

Evaluated models: o4-mini, Gemini 2.5 Flash, GPT-5 +8 more

Source: arXiv

Guardrail Policy Extraction

Published 11/1/2025

Analyzed 2/21/2026

A black-box guardrail reverse-engineering vulnerability exists in Large Language Model (LLM) serving systems that employ output filtering mechanisms. The vulnerability allows remote attackers to replicate the proprietary decision-making policy and rule sets of the target's safety guardrail without direct access to model parameters. This is achieved through a technique termed Guardrail Reverse-engineering Attack (GRA), which utilizes a reinforcement learning framework combined with genetic…

Black-Box Guardrail Reverse-engineering Attack

Evaluated models: GPT-4o, Llama 3.1 8B

Source: arXiv

Latent Space Discontinuity Exploitation

Published 11/1/2025

Analyzed 12/5/2025

A vulnerability exists in certain Large Language Models and diffusion models due to discontinuities in their latent space, which arise from data sparsity during training. An attacker can craft inputs containing lexically rare or semantically ambiguous constructs to guide the model's inference process toward these unstable, poorly-conditioned regions. This technique, termed "Alignment Degradation Induction," can degrade or bypass safety alignment mechanisms. Through iterative, multi-turn…

Exploiting Latent Space Discontinuities for Building Universal LLM Jailbreaks and Data Extraction Attacks

Evaluated models: Not reported

Source: arXiv

Pervasive Multi-turn Jailbreaks

Published 11/1/2025

Analyzed 12/8/2025

Multiple open-weight Large Language Models (LLMs)—specifically those prioritizing capability over safety alignment—exhibit a critical vulnerability to adaptive multi-turn prompt injection and jailbreak attacks. While these models effectively reject isolated, single-turn adversarial inputs (averaging ~13.11% Attack Success Rate), they fail to maintain safety guardrails and policy enforcement across extended conversational contexts. By leveraging iterative strategies such as "Crescendo" (gradual…

Death by a Thousand Prompts: Open Model Vulnerability Analysis

Evaluated models: GPT-oss 20B, Llama 3.3 70B Instruct, Mistral Large 2 +5 more

Source: arXiv

Controlled-Release Guard Bypass

Published 10/1/2025

Analyzed 12/30/2025

A vulnerability termed "Controlled-Release Prompting" allows attackers to bypass lightweight input filters (prompt guards) deployed in front of Large Language Models (LLMs). The attack exploits the computational resource asymmetry between the resource-constrained guard model and the highly capable target model. Attackers encode malicious instructions using obfuscation techniques—such as substitution ciphers (Timed-Release) or verbose character descriptions (Spaced-Release)—that require…

Bypassing Prompt Guards in Production with Controlled-Release Prompting

Evaluated models: Gemini 2.5 Flash, Gemini 2.5 Pro, DeepSeek R1 +2 more

Source: arXiv

Distributed Backdoor in Multi-Agent Systems

Published 10/1/2025

Analyzed 10/31/2025

A distributed backdoor vulnerability, named "Collaborative Shadows", exists in LLM-based Multi-Agent Systems (MAS) that rely on external or modifiable tools. An attacker can poison multiple agent tools by embedding inert, encrypted "attack primitives" within them. These primitives are fragments of a larger malicious payload. A carefully crafted user instruction acts as both a trigger and a decryption key. The instruction steers the agents to collaborate in a specific sequence, causing them to…

Collaborative Shadows: Distributed Backdoor Attacks in LLM-Based Multi-Agent Systems

Evaluated models: Gemini 2.5 Pro, GLM 4.5, GPT-4.1 +2 more

Source: arXiv

Leaked Bits Collapse Attack Queries

Published 10/1/2025

Analyzed 12/9/2025

Large Language Models (LLMs), specifically variants of GPT-4o, DeepSeek-R1, OLMo-2, and Llama-4, are vulnerable to accelerated adaptive adversarial attacks due to excessive information leakage in observable output signals. When these models expose "thinking processes" (Chain-of-Thought traces) or token-level log-probabilities (logits) to the end user, they leak significant mutual information $I(Z;T)$ regarding the model's safety state or hidden instructions. This leakage allows adaptive attack…

Bits Leaked per Query: Information-Theoretic Bounds on Adversarial Attacks against LLMs

Evaluated models: DeepSeek R1, GPT-4o Mini 2024-07-18, Llama 4 Maverick 17B +4 more

Source: arXiv

Mobile Agent Channel Subversion

Published 10/1/2025

Analyzed 12/30/2025

Mobile LLM-based agents (including Mobile-Agent-E, AppAgent, AutoDroid, and others) are vulnerable to indirect prompt injection attacks delivered via untrusted third-party mobile channels, such as in-app advertisements, system notifications, and embedded webviews. These agents utilize Multimodal Large Language Models (MLLMs) to perceive the device state via screenshots or accessibility trees. The vulnerability exists because the agents concatenate the user's prompt ($p$) with the environmental…

Measuring the Security of Mobile LLM Agents under Adversarial Prompts from Untrusted Third-Party Channels

Evaluated models: GPT-3.5 Turbo, GPT-4 Turbo, GPT-4o +1 more

Source: arXiv

Trust Paradox Agent Over-Exposure

Published 10/1/2025

Analyzed 12/9/2025

A privilege escalation and information disclosure vulnerability exists in Large Language Model (LLM) based Multi-Agent Systems (MAS) utilizing explicit inter-agent trust modeling. When the internal trust coefficient ($\tau$) between a Custodian-Agent (holding sensitive data) and a Seeker-Agent is set to a high level (e.g., $\tau=0.9$) to optimize coordination efficiency, the Custodian-Agent systematically fails to enforce Minimum Necessary Information (MNI) principles. This…

The Trust Paradox in LLM-Based Multi-Agent Systems: When Collaboration Becomes a Security Vulnerability

Evaluated models: Llama 3 8B

Source: arXiv

Related research categories