Attack Type

Side channel Vulnerabilities

Attacks exploiting indirect information leakage

10 matching entries out of 20 in this category

Side channel research results

10 entries

Activation Steering Leaks PII

Updated 7/14/2025

Large Language Models (LLMs) are vulnerable to activation steering attacks that bypass safety and privacy mechanisms. By manipulating internal attention head activations using lightweight linear probes trained on refusal/disclosure behavior, an attacker can induce the model to reveal Personally Identifiable Information (PII) memorized during training, including sensitive attributes like sexual orientation, relationships, and life events. The attack does not require adversarial prompts or…

PII Jailbreaking in LLMs via Activation Steering Reveals Personal Information Leakage

Affects: Gemma 2 9B, GLM 9B, GPT-4 +4 more

Source: arXiv

Asynchronous Audio Jailbreak

Updated 5/31/2025

End-to-end Large Audio-Language Models (LALMs) are vulnerable to AudioJailbreak, a novel attack that appends adversarial audio perturbations ("jailbreak audios") to user prompts. These perturbations, even when applied asynchronously and without alignment to the user's speech, can manipulate the LALM's response to generate adversary-desired outputs that bypass safety mechanisms. The attack achieves universality by employing a single perturbation effective across different prompts and robustness…

AudioJailbreak: Jailbreak Attacks against End-to-End Large Audio-Language Models

Affects: BLSP, FunAudioLLM, GPT-4o +9 more

Source: arXiv

LLM Censorship Vector Control

Updated 5/4/2025

Large Language Models (LLMs) employing safety mechanisms based on supervised fine-tuning and preference alignment exhibit a vulnerability to "steering" attacks. Maliciously crafted prompts or input manipulations can exploit representation vectors within the model to either bypass censorship ("refusal-compliance vector") or suppress the model's reasoning process ("thought suppression vector"), resulting in the generation of unintended or harmful outputs. This vulnerability is demonstrated…

Steering the CensorShip: Uncovering Representation Vectors for LLM" Thought" Control

Affects: DeepSeek R1 Distill Qwen 1.5B, DeepSeek R1 Distill Qwen 32B, DeepSeek R1 Distill Qwen 7B +8 more

Source: arXiv

Audio Adversarial Jailbreak

Updated 12/29/2024

Large Audio-Language Models (LALMs) are vulnerable to a stealthy adversarial jailbreak attack, AdvWave, which leverages a dual-phase optimization to overcome gradient shattering caused by audio discretization. The attack crafts adversarial audio by adding perceptually realistic environmental noise, making it difficult to detect. The attack also dynamically adapts the adversarial target based on the LALM's response patterns.

AdvWave: Stealthy Adversarial Jailbreak Attack against Large Audio-Language Models

Affects: GPT-4o, Llama Omni, Qwen 2 Audio +1 more

Source: arXiv

Bimodal Black-Box Jailbreak

Updated 12/29/2024

A bimodal adversarial attack, PBI-Attack, can manipulate Large Vision-Language Models (LVLMs) into generating toxic or harmful content by iteratively optimizing both textual and visual inputs in a black-box setting. The attack leverages a surrogate LVLM to inject malicious features from a harmful corpus into a benign image, then iteratively refines both image and text perturbations to maximize the toxicity of the model’s output as measured by a toxicity detection model (Perspective API or…

BAMBA: A Bimodal Adversarial Multi-Round Black-Box Jailbreak Attacker for LVLMs

Affects: GPT-4, InstructBLIP, MiniGPT-4 +1 more

Source: arXiv

Obfuscated Activations Jailbreak

Updated 12/29/2024

Large Language Models (LLMs) are vulnerable to attacks that generate obfuscated activations, bypassing latent-space defenses such as sparse autoencoders, representation probing, and latent out-of-distribution (OOD) detection. Attackers can manipulate model inputs or training data to produce outputs exhibiting malicious behavior while remaining undetected by these defenses. This occurs because the models can represent harmful behavior through diverse activation patterns, allowing attackers to…

Obfuscated Activations Bypass LLM Latent-Space Defenses

Affects: Gemma 2 2B, Llama 3 8B Instruct

Source: arXiv

Targeted Bit-Flip Jailbreak

Updated 2/2/2025

A vulnerability exists in large language models (LLMs) where targeted bitwise corruptions in model parameters can induce a "jailbroken" state, causing the model to generate harmful responses without input modification. Fewer than 25 bit-flips are sufficient to achieve this in many cases. The vulnerability stems from the susceptibility of the model's memory representation to fault injection attacks.

PrisonBreak: Jailbreaking Large Language Models with Fewer Than Twenty-Five Targeted Bit-flips

Affects: Llama 2 13B, Llama 2 7B, Llama 3 8B +4 more

Source: arXiv

Embodied LLM Misaligned Actions

Updated 12/29/2024

Embodied Large Language Models (LLMs) are vulnerable to manipulation via voice-based interactions, leading to the execution of harmful physical actions. Attacks exploit three vulnerabilities: (1) cascading LLM jailbreaks resulting in malicious robotic commands; (2) misalignment between linguistic outputs (verbal refusal) and physical actions (command execution); and (3) conceptual deception, where seemingly benign instructions lead to harmful outcomes due to incomplete world knowledge within…

BadRobot: Manipulating Embodied LLMs in the Physical World

Affects: BERT, GPT-3.5 Turbo, GPT-4 Turbo +2 more

Source: arXiv

Fast Projected Gradient Jailbreak

Updated 12/28/2024

Large Language Models (LLMs) are vulnerable to efficient adversarial attacks using Projected Gradient Descent (PGD) on a continuously relaxed input prompt. This attack bypasses existing alignment methods by crafting adversarial prompts that induce the model to produce undesired or harmful outputs, significantly faster than previous state-of-the-art discrete optimization methods. The effectiveness stems from carefully controlling the error introduced by the continuous relaxation of the discrete…

Attacking large language models with projected gradient descent

Affects: Falcon 7B, Falcon 7B Instruct, Vicuna 7B v1.3

Source: arXiv

LLM Causal Neuron Attack

Updated 12/28/2024

Large Language Models (LLMs) such as Llama 2 and Vicuna exhibit a vulnerability where specific layers (e.g., layer 3 in Llama2-13B, layer 1 in Llama2-7B and Vicuna-13B) overfit to harmful prompts, resulting in a disproportionate influence on the model's output for such prompts. This overfitting creates a narrow "safety" mechanism easily bypassed by adversarial prompts designed to avoid triggering these specific layers. Additionally, a single neuron (e.g., neuron 2100 in Llama2 and Vicuna)…

Causality analysis for evaluating the security of large language models

Affects: GPT-3.5 Turbo, GPT-NeoX, Llama 2-13B-chat-hf +2 more

Source: arXiv

Related research categories