Attack Type

Jailbreak Research

Methods for bypassing model safety measures

704 matching entries

Jailbreak research results

704 entries

Happy Ending LLM Jailbreak

Published 1/1/2025

Analyzed 3/4/2025

Large language models (LLMs) exhibit increased responsiveness to prompts framed within positive narratives. The Happy Ending Attack (HEA) exploits this by embedding malicious requests within a positive-sentiment scenario culminating in a happy ending. This allows the LLM to generate responses that fulfill the malicious request while perceiving the overall prompt as benign.

Dagger Behind Smile: Fool LLMs with a Happy Ending Story

Evaluated models: Gemini Flash, Gemini Pro, GPT-4o +3 more

Source: arXiv

LALM Audio Jailbreak

Published 1/1/2025

Analyzed 12/30/2025

End-to-end Large Audio Language Models (LALMs) contain an audio-based jailbreak vulnerability allowing attackers to bypass safety alignment guardrails by manipulating audio-specific "hidden semantics." Unlike text-based attacks, this exploitation involves encoding harmful queries into audio and applying signal processing modifications—specifically changes to emphasis, speech speed, intonation, tone, background noise, celebrity accents, or emotional overlays (e.g., laughter, screaming). These…

Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Models

Evaluated models: GPT-4o, Qwen 2 7B

Source: arXiv

LLM Hate Campaign Vulnerability

Published 1/1/2025

Analyzed 2/2/2025

Large Language Models (LLMs) used in hate speech detection systems are vulnerable to adversarial attacks and model stealing, resulting in evasion of hate speech detection. Adversarial attacks modify hate speech text to evade detection, while model stealing creates surrogate models that mimic the target system's behavior.

HateBench: Benchmarking Hate Speech Detectors on LLM-Generated Content and Hate Campaigns

Evaluated models: Baichuan 2, Dolly 2, GPT-3.5 Turbo +2 more

Source: arXiv

LLM Risk Amplification

Published 1/1/2025

Analyzed 12/9/2025

Vision Language Models (VLMs) are vulnerable to visual prompt injection attacks via text-to-image obfuscation. While these models often possess safety guardrails for standard text-based inputs, they fail to apply equivalent safety alignment to textual instructions embedded visually within an image. An attacker can overlay malicious instructions (e.g., requests for illegal acts, hate speech) onto an image file and submit it to the model. The model’s Optical Character Recognition (OCR) or visual…

Lessons from red teaming 100 generative ai products

Evaluated models: GPT-4, Phi-3

Source: arXiv

Multi-Turn LLM Jailbreak

Published 1/1/2025

Analyzed 2/2/2025

Large Language Models (LLMs) are vulnerable to multi-turn adversarial attacks that skillfully decompose malicious requests into seemingly benign interactions, progressively guiding the dialogue towards harmful outputs. This vulnerability allows attackers to bypass LLM safety mechanisms through a series of strategically crafted prompts, exploiting the model's iterative response generation. The attack's success hinges on dynamically adapting each prompt based on the LLM's previous responses…

Siren: A Learning-Based Multi-Turn Attack Framework for Simulating Real-World Human Jailbreak Behaviors

Evaluated models: Claude 3.5 Sonnet, Gemini 1.5 Pro, GPT-4o +3 more

Source: arXiv

Scientific Language Jailbreak

Published 1/1/2025

Analyzed 2/2/2025

Large Language Models (LLMs) are vulnerable to malicious prompts disguised as summaries of scientific papers, even when those papers are fabricated by the attacker. This allows attackers to manipulate LLMs into generating responses exhibiting significantly increased stereotypical bias and toxicity. The vulnerability is exacerbated by multi-turn interactions, where bias scores tend to increase with each subsequent response. The inclusion of author names and publication venues in the fabricated…

LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Language

Evaluated models: Command R+, GPT-4, GPT-4o +3 more

Source: arXiv

Self-Instruct LLM Jailbreak

Published 1/1/2025

Analyzed 1/26/2025

Large Language Models (LLMs) are vulnerable to a self-instruct few-shot jailbreaking attack that leverages pattern and behavior learning to bypass safety mechanisms. The attack efficiently induces harmful outputs by injecting a strategically chosen response prefix into the model's prompt and exploiting the model's tendency to mimic co-occurrence patterns of special tokens preceding the prefix. This allows the attacker to elicit unsafe responses with a small number of carefully crafted…

Self-Instruct Few-Shot Jailbreaking: Decompose the Attack into Pattern and Behavior Learning

Evaluated models: GPT-2, Llama 2 7B Chat, Llama 3 8B Instruct +7 more

Source: arXiv

Shuffle Inconsistency Jailbreak

Published 1/1/2025

Analyzed 1/26/2025

Multimodal Large Language Models (MLLMs) exhibit a vulnerability where shuffling the order of words in text prompts or patches in image prompts can bypass their safety mechanisms, despite the model still understanding the intent of the shuffled input. This "Shuffle Inconsistency" allows attackers to elicit harmful responses by submitting shuffled harmful prompts that would otherwise be blocked.

Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency

Evaluated models: Claude 3.5 Sonnet, Gemini 1.5 Pro, GPT-4o +6 more

Source: arXiv

Sophisticated Reasoning Bypass

Published 1/1/2025

Analyzed 12/30/2025

Large Language Models (LLMs), specifically instruction-following models using standard refusal training and adversarial training (such as Llama-3.1-8B-Instruct and Mistral-7B-V0.2), contain a vulnerability related to safety alignment bypass. The vulnerability arises from the models' inability to generalize safety reasoning to Out-Of-Distribution (OOD) inputs and scenarios involving competing objectives. Attackers can exploit this by employing linguistic manipulation (slang, uncommon dialects…

Enhancing Model Defense Against Jailbreaks with Proactive Safety Reasoning

Evaluated models: Llama 3.1 8B Instruct, Mistral 7B Instruct v0.2

Source: arXiv

Surrogate Classifier Extraction

Published 1/1/2025

Analyzed 3/4/2025

Large Language Models (LLMs) employing alignment techniques for safety embed a "safety classifier" within their architecture. This classifier, responsible for determining whether an input is safe or unsafe, can be approximated by extracting a surrogate classifier from a subset of the LLM's architecture. Attackers can leverage this surrogate classifier to more effectively craft adversarial inputs (jailbreaks) that bypass the LLM's intended safety mechanisms. The attack success rate against the…

Targeting Alignment: Extracting Safety Classifiers of Aligned LLMs

Evaluated models: Gemma 2 9B IT, Gemma 7B IT, Granite 3.1 8B Instruct +5 more

Source: arXiv

Related research categories