Attack Type

Jailbreak Vulnerabilities

Methods for bypassing model safety measures

703 matching entries

Jailbreak research results

703 entries

Diffusion-Driven LLM Jailbreak

Updated 12/28/2024

DiffusionAttacker exploits a vulnerability in Large Language Models (LLMs) allowing manipulation of prompts to elicit harmful responses, even when the model incorporates safety mechanisms. The attack leverages a sequence-to-sequence diffusion model to rewrite harmful prompts, making them appear harmless to the LLM's internal representation while preserving their original semantic meaning. This bypasses safety filters and elicits undesired outputs.

DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak

Affects: Alpaca 7B, Claude 3.5 Sonnet, GPT-3.5 Turbo +4 more

Source: arXiv

Embodied AI Policy Jailbreak

Updated 1/26/2025

LLM-based planning modules in embodied AI systems are vulnerable to Policy Executable (POEX) jailbreak attacks. Attackers can inject carefully crafted adversarial suffixes into user instructions, causing the LLM to generate and execute harmful policies in both simulated and real-world environments. The attacks bypass safety mechanisms by using optimized, human-readable suffixes that evade perplexity-based detection.

POEX: Policy Executable Embodied AI Jailbreak Attacks

Affects: Claude 3.5 Sonnet, GPT-4, GPT-4 Turbo +9 more

Source: arXiv

Index-Gradient LLM Jailbreak

Updated 12/29/2024

Large Language Models (LLMs) are vulnerable to optimization-based jailbreaking attacks that exploit gradients during the iterative process of generating adversarial suffixes. The vulnerability stems from the inefficient exploration of the token space in existing methods like Greedy Coordinate Gradient (GCG), which uniformly samples tokens for replacement regardless of gradient values. This leads to redundant computations and a slow optimization process.

Exploiting the Index Gradients for Optimization-Based Jailbreaking on Large Language Models

Affects: Vicuna 7B, Guanaco 7B, Llama 2 7B Chat +5 more

Source: arXiv

Linked-Task LLM Jailbreak

Updated 12/28/2024

A novel jailbreak paradigm, Simple Assistive Task Linkage (SATA), circumvents LLM safeguards by masking harmful keywords in a malicious query and using a secondary, simple assistive task (e.g., masked language modeling or element lookup by position) to convey the masked keywords' semantics to the LLM. This distracts the LLM and allows it to bypass safety checks, leading to the generation of harmful responses.

SATA: A Paradigm for LLM Jailbreak via Simple Assistive Task Linkage

Affects: Claude-v2, GPT-3.5 Turbo, GPT-4o +3 more

Source: arXiv

Metaphorical LLM Jailbreak

Updated 1/26/2025

Large Language Models (LLMs) are vulnerable to jailbreaking attacks via adversarial metaphors. Attackers can leverage the LLMs' imaginative capabilities to map harmful concepts to innocuous ones, thereby bypassing safety mechanisms and eliciting harmful responses. The attack relies on creating a metaphorical mapping between a harmful target and seemingly benign entities, exploiting the LLM's ability to reason about the analogous relationship without recognizing the underlying malicious intent.

Na'vi or Knave: Jailbreaking Language Models via Metaphorical Avatars

Affects: Claude 3.5 Sonnet, Gemini 1.5 Pro, GLM 3 6B +13 more

Source: arXiv

Multi-Modal VLM Jailbreak

Updated 1/26/2025

A novel jailbreak attack, Multi-Modal Linkage (MML), exploits the vulnerability in Large Vision-Language Models (VLMs) by leveraging an "encryption-decryption" scheme across text and image modalities. MML encrypts malicious queries within images (e.g., using word replacement, image transformations) to bypass initial safety mechanisms. A subsequent text prompt guides the VLM to "decrypt" the content, eliciting harmful outputs. "Evil alignment," framing the attack within a video game scenario…

Jailbreak Large Visual Language Models Through Multi-Modal Linkage

Affects: Claude 3.5 Sonnet, GPT-4o, GPT-4o Mini +1 more

Source: arXiv

Multimodal LLM Jailbreak

Updated 12/29/2024

A hybrid multimodal jailbreaking attack, dubbed JMLLM, exploits vulnerabilities in 13 popular large language models (LLMs) across text, image, and speech modalities. The attack leverages alternating translation, word encryption, feature collapse in images, and harmful text injection to bypass safety mechanisms and elicit harmful responses. Success rates vary across LLMs and modalities, with some models exhibiting significantly higher vulnerability than others.

Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models

Affects: Claude 1, Claude 2, ERNIE 3.5 Turbo +10 more

Source: arXiv

Multimodal Risk Diffusion Jailbreak

Updated 12/29/2024

Multimodal Large Language Models (MLLMs) are vulnerable to a heuristic-induced multimodal risk distribution jailbreak attack. The attack successfully circumvents safety mechanisms by distributing malicious prompts across text and image modalities, preventing detection of harmful intent within either modality alone. An auxiliary LLM generates prompts to guide the target MLLM into reconstructing the malicious prompt and producing the desired harmful output.

Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Models

Affects: Deepseek-vl7B-chat, Gemini 1.5 Pro, Glm-4v-9B +7 more

Source: arXiv

Natural Prompt Jailbreaks

Updated 12/29/2024

Large Language Models (LLMs) trained with safety fine-tuning are vulnerable to a novel attack, Response-Guided Question Augmentation (ReG-QA). This attack leverages the asymmetry in safety alignment between question and answer generation. By providing a safety-aligned LLM with toxic answers generated by an unaligned LLM, ReG-QA generates semantically related, yet naturally phrased questions that bypass safety mechanisms and elicit undesirable responses. The attack does not require adversarial…

Does Safety Training of LLMs Generalize to Semantically Related Natural Prompts?

Affects: Gemma 2 27B IT, Gemma 2 9B IT, GPT-3.5 Turbo +6 more

Source: arXiv

Obfuscated Activations Jailbreak

Updated 12/29/2024

Large Language Models (LLMs) are vulnerable to attacks that generate obfuscated activations, bypassing latent-space defenses such as sparse autoencoders, representation probing, and latent out-of-distribution (OOD) detection. Attackers can manipulate model inputs or training data to produce outputs exhibiting malicious behavior while remaining undetected by these defenses. This occurs because the models can represent harmful behavior through diverse activation patterns, allowing attackers to…

Obfuscated Activations Bypass LLM Latent-Space Defenses

Affects: Gemma 2 2B, Llama 3 8B Instruct

Source: arXiv

Related research categories