Attack Type

Extraction Vulnerabilities

Techniques for extracting sensitive information from models

9 matching entries out of 97 in this category

Extraction research results

9 entries

LLM Prefix Cache Reconstruction

Updated 4/10/2026

Automatic Prefix Caching (APC) in multi-tenant LLM serving systems introduces a timing side-channel vulnerability that permits cross-tenant data leakage. APC shares computed Key-Value (KV) tensors across different users when their requests share identical initial tokens. Because reusing cached tensors is significantly faster than recomputing them, a measurable difference in Time-To-First-Token (TTFT) exists between cache hits and misses. An attacker can exploit this shared cache by sending…

PrefixWall: Mitigating Prefix Caching Side Channels in Shared LLM Systems

Affects: Gemma 3 4B IT, Llama 2 7B Chat, Llama 2 13B Chat +6 more

Source: arXiv

LLM Edit Fingerprint Leak

Updated 2/21/2026

A side-channel information leakage vulnerability exists in the "locate-then-edit" paradigm of Large Language Model (LLM) knowledge editing, specifically affecting algorithms such as ROME, MEMIT, and AlphaEdit. The parameter update matrix ($\Delta W$) generated during the editing process preserves the algebraic structure of the edited data. Specifically, the row space of the parameter difference matrix encodes a mathematical fingerprint of the key vectors associated with the edited subjects. An…

Reverse-Engineering Model Editing on Language Models

Affects: Llama 3 8B, Qwen 2.5 7B

Source: arXiv

Logit Leakage Model Clone

Updated 12/9/2025

Large Language Model (LLM) inference APIs that expose top-k logits or log-probabilities are vulnerable to model extraction and cloning. An attacker can execute a two-stage attack to replicate the proprietary model without access to weights, gradients, or training data. First, by submitting fewer than 10,000 random queries and aggregating the returned unrounded logits, the attacker recovers the model's output projection matrix using Singular Value Decomposition (SVD). Second, the attacker…

Clone What You Can't Steal: Black-Box LLM Replication via Logit Leakage and Distillation

Affects: GPT-3.5, Mistral 7B

Source: arXiv

KV-Cache Sharing Timing Side-channel

Updated 1/14/2026

Multi-tenant Large Language Model (LLM) inference systems utilizing global Key-Value (KV) cache sharing are vulnerable to a timing side-channel attack. By measuring the Time-To-First-Token (TTFT) latency of crafted API requests, an unprivileged remote attacker can determine if specific token sequences have been previously processed and cached by the system for other users. This observable timing difference between cache hits (low TTFT) and cache misses (high TTFT) allows for the token-by-token…

Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference

Affects: Phi-4 14B, Qwen 3 30B-A3B, Qwen 3 32B +3 more

Source: arXiv

Activation Steering Leaks PII

Updated 7/14/2025

Large Language Models (LLMs) are vulnerable to activation steering attacks that bypass safety and privacy mechanisms. By manipulating internal attention head activations using lightweight linear probes trained on refusal/disclosure behavior, an attacker can induce the model to reveal Personally Identifiable Information (PII) memorized during training, including sensitive attributes like sexual orientation, relationships, and life events. The attack does not require adversarial prompts or…

PII Jailbreaking in LLMs via Activation Steering Reveals Personal Information Leakage

Affects: Gemma 2 9B, GLM 9B, GPT-4 +4 more

Source: arXiv

LLM Censorship Vector Control

Updated 5/4/2025

Large Language Models (LLMs) employing safety mechanisms based on supervised fine-tuning and preference alignment exhibit a vulnerability to "steering" attacks. Maliciously crafted prompts or input manipulations can exploit representation vectors within the model to either bypass censorship ("refusal-compliance vector") or suppress the model's reasoning process ("thought suppression vector"), resulting in the generation of unintended or harmful outputs. This vulnerability is demonstrated…

Steering the CensorShip: Uncovering Representation Vectors for LLM" Thought" Control

Affects: DeepSeek R1 Distill Qwen 1.5B, DeepSeek R1 Distill Qwen 32B, DeepSeek R1 Distill Qwen 7B +8 more

Source: arXiv

Obfuscated Activations Jailbreak

Updated 12/29/2024

Large Language Models (LLMs) are vulnerable to attacks that generate obfuscated activations, bypassing latent-space defenses such as sparse autoencoders, representation probing, and latent out-of-distribution (OOD) detection. Attackers can manipulate model inputs or training data to produce outputs exhibiting malicious behavior while remaining undetected by these defenses. This occurs because the models can represent harmful behavior through diverse activation patterns, allowing attackers to…

Obfuscated Activations Bypass LLM Latent-Space Defenses

Affects: Gemma 2 2B, Llama 3 8B Instruct

Source: arXiv

LLM Version Fingerprinting

Updated 12/28/2024

Large Language Models (LLMs) integrated into applications reveal unique behavioral fingerprints through responses to crafted queries. LLMmap exploits this by sending carefully constructed prompts and analyzing the responses to identify the specific LLM version with high accuracy (over 95% in testing against 42 LLMs). This allows attackers to tailor attacks exploiting known vulnerabilities specific to the identified LLM version.

Llmmap: Fingerprinting for large language models

Affects: Aya-23-8B, Cohere-35B, GPT-4 +9 more

Source: arXiv

LLM Causal Neuron Attack

Updated 12/28/2024

Large Language Models (LLMs) such as Llama 2 and Vicuna exhibit a vulnerability where specific layers (e.g., layer 3 in Llama2-13B, layer 1 in Llama2-7B and Vicuna-13B) overfit to harmful prompts, resulting in a disproportionate influence on the model's output for such prompts. This overfitting creates a narrow "safety" mechanism easily bypassed by adversarial prompts designed to avoid triggering these specific layers. Additionally, a single neuron (e.g., neuron 2100 in Llama2 and Vicuna)…

Causality analysis for evaluating the security of large language models

Affects: GPT-3.5 Turbo, GPT-NeoX, Llama 2-13B-chat-hf +2 more

Source: arXiv

Related research categories