Attack Type

Jailbreak Research

Methods for bypassing model safety measures

704 matching entries

Jailbreak research results

704 entries

SLM Quantization Direct Harms

Published 5/1/2025

Analyzed 12/8/2025

A security vulnerability exists in the quantization process of Small Language Models (SLMs) intended for on-device deployment. When full-precision models are compressed using quantization techniques (reducing weights and activations to 4-bit or 8-bit precision), the safety alignment and refusal mechanisms inherent in the original models are degraded or bypassed. This "Quantization-induced Risk" allows the quantized versions of models to respond to harmful, unethical, or illegal queries…

LiteLMGuard: Seamless and Lightweight On-Device Prompt Filtering for Safeguarding Small Language Models against Quantization-induced Risks and …

Evaluated models: Phi-3

Source: arXiv

Semantic Audio Jailbreak

Published 5/1/2025

Analyzed 12/8/2025

Large Audio-Language Models (LAMs) are vulnerable to adversarial signal-level perturbations that allow for the bypass of safety guardrails (jailbreaking). While these models may possess robust text-based safety alignment, they fail to generalize this robustness to the audio modality. Attackers can utilize the Audio Perturbation Toolkit (APT) to apply transformations in the time domain (Energy Distribution Perturbation, Trimming, Fade In/Out), frequency domain (Pitch Shifting, Temporal…

Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models

Evaluated models: SpeechGPT, SALMONN, DiVA +4 more

Source: arXiv

Single-Query LLM Jailbreak

Published 5/1/2025

Analyzed 5/31/2025

Large Language Models (LLMs) are vulnerable to a novel jailbreak attack, termed ICE (Intent Concealment and Diversion), which leverages hierarchical prompt decomposition and semantic expansion to bypass safety filters. ICE achieves high attack success rates with single queries, exploiting the models' limitations in multi-step reasoning.

Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversion

Evaluated models: Claude 1, Claude 2, ERNIE 3.5 Turbo +6 more

Source: arXiv

Steganographic LLM Jailbreak

Published 5/1/2025

Analyzed 5/31/2025

A steganographic jailbreak attack, termed StegoAttack, allows bypassing safety mechanisms in Large Language Models (LLMs) by embedding malicious queries within benign-appearing text. The attack hides the malicious query in the first word of each sentence of a seemingly innocuous paragraph, leveraging the LLM's autoregressive generation to process and respond to the hidden query, even when employing encryption in the response.

Hiding in Plain Sight: A Steganographic Approach to Stealthy LLM Jailbreaks

Evaluated models: GPT-5, DeepSeek V3.2 Thinking, Qwen 3 Max Thinking

Source: arXiv

Universal Jailbreak Prompt Generator

Published 5/1/2025

Analyzed 5/31/2025

Large Language Models (LLMs) are vulnerable to robust jailbreak prompts generated by the ArrAttack framework. ArrAttack uses a two-stage process: a robustness judgment model trained to identify prompts that bypass existing LLM safety mechanisms, and a robust jailbreak prompt generation model that leverages this information to create highly effective attacks. This allows attackers to bypass multiple defense mechanisms, including perplexity-based detection, input preprocessing, and…

One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs

Evaluated models: GPT-3.5 Turbo, GPT-4, Guanaco 7B +3 more

Source: arXiv

Universal VLLM Visual Bypass

Published 5/1/2025

Analyzed 12/9/2025

A vulnerability exists in Vision-Language Models (VLLMs) that allows for transferable, targeted adversarial attacks. Attackers can generate adversarial image perturbations using an ensemble of open-source surrogate models (primarily CLIP-based visual encoders) which effectively transfer to proprietary, black-box VLLMs. The attack leverages a specific optimization framework that combines a Visual Contrastive Loss with multiple positive/negative visual examples, rather than relying solely on…

Transferable Adversarial Attacks on Black-Box Vision-Language Models

Evaluated models: Qwen 2.5 VL 7B Instruct, Qwen 2.5 VL 72B Instruct, Llama 3.2 11B Vision Instruct +6 more

Source: arXiv

VLM Latent Boundary Crossing

Published 5/1/2025

Analyzed 12/30/2025

Vision-Language Models (VLMs) contain a vulnerability in their multimodal fusion layers where safety-relevant information is linearly separable in the latent space. This allows for a "JailBound" attack, which exploits the implicit internal safety decision boundary. The attack proceeds in two stages: (1) Safety Boundary Probing, where attackers approximate the internal decision hyperplane by training layer-wise logistic regression classifiers on the fusion representations of safe versus unsafe…

JailBound: Jailbreaking Internal Safety Boundaries of Vision-Language Models

Evaluated models: Llama 3.2 11B Vision Instruct, Qwen 2.5 VL 7B Instruct, MiniGPT-4 +3 more

Source: arXiv

Targeted Noise Injection Jailbreak

Published 4/30/2025

Analyzed 5/4/2025

A vulnerability in several Large Language Models (LLMs) allows bypassing safety mechanisms through targeted noise injection. Explainable AI (XAI) techniques reveal specific layers within the LLM architecture most responsible for content filtering. Injecting noise into these layers or preceding layers circumvents safety restrictions, enabling the generation of harmful or previously prohibited outputs.

XBreaking: Understanding how LLMs security alignment can be broken

Evaluated models: Llama 3.2 1B, Llama 3.1 8B, Qwen 2.5 0.5B +4 more

Source: arXiv

Benign-Prompt Jailbreak

Published 4/1/2025

Analyzed 4/12/2025

Large Language Models (LLMs) exhibit Defense Threshold Decay (DTD): generating substantial benign content shifts the model's attention from the input prompt to prior outputs, increasing susceptibility to jailbreak attacks. The "Sugar-Coated Poison" (SCP) attack exploits this by first generating benign content, then transitioning to malicious output.

Sugar-Coated Poison: Benign Generation Unlocks LLM Jailbreaking

Evaluated models: Claude 3.5 Sonnet, DeepSeek R1, GPT-3.5 Turbo +3 more

Source: arXiv

Chained Guardrail Bypass

Published 4/1/2025

Analyzed 12/30/2025

Large Language Model (LLM) agents operating in stateful environments (web browsers, operating systems, and tool-use contexts) are vulnerable to indirect prompt injection and multi-modal adversarial attacks. These vulnerabilities arise when agents process untrusted environmental observations—such as web accessibility trees, screen screenshots, or database query results—that contain concealed malicious instructions. Specifically, attackers can embed prompt injections into HTML accessibility…

DoomArena: A Framework for Testing AI Agents Against Evolving Security Threats

Evaluated models: GPT-4o, GPT-4o Mini, Claude 3.5 Sonnet +2 more

Source: arXiv

Related research categories