Attack Type

Jailbreak Research

Methods for bypassing model safety measures

704 matching entries

Jailbreak research results

704 entries

Dual Jailbreak via TDI/MTO

Published 4/1/2025

Analyzed 5/4/2025

A vulnerability exists in the combination of Large Language Models (LLMs) and their associated safety guardrails, allowing attackers to bypass both defenses and elicit harmful or unintended outputs from LLMs. The vulnerability stems from insufficient detection by guardrails against adversarially crafted prompts, which appear benign but contain hidden malicious intent. The attack, dubbed "DualBreach," leverages a target-driven initialization strategy and multi-target optimization to generate…

DualBreach: Efficient Dual-Jailbreaking via Target-Driven Initialization and Multi-Target Optimization

Evaluated models: GPT-3.5 Turbo, GPT-4, Llama 3 8B Instruct +1 more

Source: arXiv

Generative Reward Hacking

Published 4/1/2025

Analyzed 12/9/2025

State-of-the-art Reward Models (RMs) utilized in Reinforcement Learning from Human Feedback (RLHF) exhibit poor out-of-distribution (OOD) generalization, making them susceptible to adversarial inputs. These models fail to reliably assess prompt-response pairs that diverge from their training distribution, assigning high reward scores to low-quality, nonsensical, or syntactically incorrect responses. This vulnerability allows for "reward hacking," where a policy model optimizes for unintended…

Adversarial training of reward models

Evaluated models: Llama 3.1 8B, Llama 3.3 70B, DeepSeek R1 +1 more

Source: arXiv

Genetic Scenario Shift Jailbreak

Published 4/1/2025

Analyzed 4/21/2025

A vulnerability in Large Language Models (LLMs) allows attackers to bypass safety mechanisms and elicit detailed harmful responses by strategically manipulating input prompts. The vulnerability exploits the LLM's sensitivity to "scenario shifts"—contextual changes in the input that influence the model's output, even when the core malicious request remains the same. A genetic algorithm can optimize these scenario shifts, increasing the likelihood of obtaining detailed harmful responses while…

Geneshift: Impact of different scenario shift on Jailbreaking LLM

Evaluated models: GPT-4o Mini

Source: arXiv

Graph-Based LLM Jailbreak

Published 4/1/2025

Analyzed 5/4/2025

Research ID 866c3b97

Large Language Models (LLMs) employing alignment safeguards and safety mechanisms are vulnerable to graph-based adversarial attacks that bypass these protections. The attack, termed "Graph of Attacks" (GOAT), leverages a graph-based reasoning framework to iteratively refine prompts and exploit vulnerabilities more effectively than previous methods. The attack synthesizes information across multiple reasoning paths to generate human-interpretable prompts that elicit undesired or harmful outputs…

Graph of Attacks: Improved Black-Box and Interpretable Jailbreaks for LLMs

Evaluated models: GPT-4, Llama 2 7B, Vicuna 13B +1 more

Source: arXiv

Graph-Based LLM Jailbreak

Published 4/1/2025

Analyzed 4/21/2025

Research ID c32aa29e

Large Language Models (LLMs) employing safety mechanisms are vulnerable to a graph-based attack that leverages semantic transformations of malicious prompts to bypass safety filters. The attack, termed GraphAttack, uses Abstract Meaning Representation (AMR), Resource Description Framework (RDF), and JSON knowledge graphs to represent malicious intent, systematically applying transformations to evade surface-level pattern recognition used by existing safety mechanisms. A particularly effective…

GraphAttack: Exploiting Representational Blindspots in LLM Safety Mechanisms

Evaluated models: Claude 3.7 Sonnet, GPT-3.5 Turbo, GPT-4 +3 more

Source: arXiv

Humorous LLM Jailbreak

Published 4/1/2025

Analyzed 4/12/2025

Large Language Models (LLMs) are vulnerable to a jailbreaking attack leveraging humorous prompts. Embedding an unsafe request within a humorous context, using a fixed template, bypasses built-in safety mechanisms and elicits unsafe responses. The attack's success relies on a balance; too little or too much humor reduces effectiveness.

Bypassing Safety Guardrails in LLMs Using Humor

Evaluated models: Gemma 3 27B IT, Llama 3.1 8B Instruct, Llama 3.3 70B Instruct +1 more

Source: arXiv

LLM Censorship Vector Control

Published 4/1/2025

Analyzed 5/4/2025

Large Language Models (LLMs) employing safety mechanisms based on supervised fine-tuning and preference alignment exhibit a vulnerability to "steering" attacks. Maliciously crafted prompts or input manipulations can exploit representation vectors within the model to either bypass censorship ("refusal-compliance vector") or suppress the model's reasoning process ("thought suppression vector"), resulting in the generation of unintended or harmful outputs. This vulnerability is demonstrated…

Steering the CensorShip: Uncovering Representation Vectors for LLM" Thought" Control

Evaluated models: DeepSeek R1 Distill Qwen 1.5B, DeepSeek R1 Distill Qwen 32B, DeepSeek R1 Distill Qwen 7B +8 more

Source: arXiv

LLM Guardrail Evasion

Published 4/1/2025

Analyzed 4/21/2025

Large Language Model (LLM) guardrail systems, including those relying on AI-driven text classification models (e.g., fine-tuned BERT models), are vulnerable to evasion via character injection and adversarial machine learning (AML) techniques. Attackers can bypass detection by injecting Unicode characters (e.g., zero-width characters, homoglyphs) or using AML to subtly perturb prompts, maintaining semantic meaning while evading classification. This allows malicious prompts and jailbreaks to…

Bypassing Prompt Injection and Jailbreak Detection in LLM Guardrails

Evaluated models: DeBERTa v3 Base, GPT-4o Mini, mDeBERTa v3 Base

Source: arXiv

MAD Amplified Jailbreaks

Published 4/1/2025

Analyzed 5/4/2025

Multi-Agent Debate (MAD) frameworks leveraging Large Language Models (LLMs) are vulnerable to amplified jailbreak attacks. A novel structured prompt-rewriting technique exploits the iterative dialogue and role-playing dynamics of MAD, circumventing inherent safety mechanisms and significantly increasing the likelihood of generating harmful content. The attack succeeds by using narrative encapsulation, role-driven escalation, iterative refinement, and rhetorical obfuscation to guide agents…

Amplified Vulnerabilities: Structured Jailbreak Attacks on LLM-based Multi-Agent Debate

Evaluated models: GPT-3.5 Turbo, GPT-4, GPT-4o

Source: arXiv

Many-Shot In-Context Override

Published 4/1/2025

Analyzed 12/9/2025

Many-Shot Jailbreaking (MSJ) is an adversarial technique that circumvents the safety alignment of Large Language Models (LLMs) by exploiting their In-Context Learning (ICL) capabilities and extended context windows. By embedding a large number of "shots" (fake dialogue examples) within a single prompt—where a simulated assistant complies with harmful requests—the attacker conditions the model to ignore its safety training. As the number of malicious examples increases (following a power-law…

Mitigating Many-Shot Jailbreaking

Evaluated models: Llama 3.1 8B

Source: arXiv

Related research categories