Attack Type

Jailbreak Research

Methods for bypassing model safety measures

704 matching entries

Jailbreak research results

704 entries

LLM Self-Introspection Jailbreak

Published 5/1/2025

Analyzed 5/31/2025

A vulnerability exists in Large Language Models (LLMs) that allows attackers to manipulate the model's output by modifying token log probabilities. Attackers can use a lightweight plug-in model (BiasNet) to subtly alter the probabilities, steering the LLM toward generating harmful content even when safety mechanisms are in place. This attack requires only access to the top-k token log probabilities returned by the LLM's API, without needing model weights or internal access.

JULI: Jailbreak Large Language Models by Self-Introspection

Evaluated models: Llama 2 7B Chat, Llama 3 8B, Llama 3 8B Instruct +3 more

Source: arXiv

LLM System Prompt Extraction

Published 5/1/2025

Analyzed 12/30/2025

Large Language Models (LLMs), including Llama-3, Falcon-3, Gemma-2, and GPT-4 variants, are susceptible to system prompt extraction attacks. The vulnerability exists due to the models' instruction-following nature, which allows remote attackers to bypass safety guardrails and retrieve the model's hidden system configuration (system prompt) verbatim. This is successfully exploited using an "Extended Sandwich Attack," where an adversarial extraction command is embedded between benign questions…

System Prompt Extraction Attacks and Defenses in Large Language Models

Evaluated models: GPT-4, GPT-4o, Llama 3 8B +2 more

Source: arXiv

Latent Refusal Suppression

Published 5/1/2025

Analyzed 12/9/2025

Large Language Models (LLMs), specifically Llama-2-7B-Chat and Qwen2.5-Instruct (1.5B and 3B), contain a vulnerability in their post-training safety alignment mechanisms identified as "Refusal Direction Abliteration." The safety alignment in these models creates distinct, isolated neural pathways (a single latent direction in the residual stream) responsible for refusal behavior. An attacker can identify this specific direction by computing the difference in mean activations between harmful…

An Embarrassingly Simple Defense Against LLM Abliteration Attacks

Evaluated models: Llama 2 7B Chat, Qwen 2.5 1.5B Instruct, Qwen 2.5 3B Instruct

Source: arXiv

Latent-Space Jailbreak Optimization

Published 5/1/2025

Analyzed 5/31/2025

The LARGO attack exploits a vulnerability in Large Language Models (LLMs) allowing attackers to bypass safety mechanisms through the generation of "stealthy" adversarial prompts. The attack leverages gradient optimization in the LLM's continuous latent space to craft seemingly innocuous natural language suffixes which, when appended to harmful prompts, elicit unsafe responses. The vulnerability stems from the LLM's inability to reliably distinguish between benign and maliciously crafted latent…

LARGO: Latent Adversarial Reflection through Gradient Optimization for Jailbreaking LLMs

Evaluated models: Llama 2 13B Chat, Llama 2 7B Chat, Phi 3 Mini +1 more

Source: arXiv

Logic-Based LLM Jailbreak

Published 5/1/2025

Analyzed 5/31/2025

Large Language Models (LLMs) employing safety mechanisms based on token-level distribution analysis are vulnerable to a jailbreak attack exploiting distributional discrepancies between alignment data and formally expressed logical statements. The vulnerability allows malicious actors to bypass safety restrictions by translating harmful natural language prompts into equivalent first-order logic expressions. The LLM, trained primarily on natural language, fails to recognize the harmful intent…

Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expression

Evaluated models: DeepSeek R1, DeepSeek V3, GPT-3.5 Turbo +4 more

Source: arXiv

Multilingual LLM Jailbreaks

Published 5/1/2025

Analyzed 6/12/2025

Multilingual prompt injection vulnerability in four closed-source Large Language Models (LLMs): GPT-4o, DeepSeek-R1, Gemini-1.5-Pro, and Qwen-Max. Attackers can bypass safety restrictions and elicit harmful or disallowed content by crafting prompts in English or Chinese, leveraging specific structural techniques (e.g., "Two Sides" prompting) that exploit inconsistencies in the models' safety alignment across languages and prompt formats.

The Tower of Babel Revisited: Multilingual Jailbreak Prompts on Closed-Source Large Language Models

Evaluated models: DeepSeek R1, Gemini 1.5 Pro, GPT-4o +1 more

Source: arXiv

On-Device LLM Hijacking

Published 5/1/2025

Analyzed 12/9/2025

Mobile LLM agents utilizing vision-based screen perception (OCR or Multimodal Large Language Models) are vulnerable to Visual Prompt Injection via malicious GUI overlays. An attacker holding the SYSTEM_ALERT_WINDOW permission can deploy non-focusable floating windows (using FLAG_NOT_FOCUSABLE) containing adversarial text or fabricated UI elements over legitimate applications. Because the agent captures the entire screen buffer to interpret the device state, it ingests the adversarial overlay…

From Assistants to Adversaries: Exploring the Security Risks of Mobile LLM Agents

Evaluated models: GPT-4o

Source: arXiv

Pretraining Modality Gap Jailbreak

Published 5/1/2025

Analyzed 12/30/2025

Large Vision-Language Models (LVLMs) that utilize a projection layer (adapter) to bridge a vision encoder and a Large Language Model (LLM) contain a vulnerability stemming from the "Modality Gap"—a distributional distance between image and text token embeddings. This gap allows the visual modality to bypass the safety alignment (RLHF/instruction tuning) of the backbone LLM. Attackers can trigger harmful, toxic, or illegal responses to queries that would be refused in text-only contexts by…

Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gap

Evaluated models: LLaVA 7B, Vicuna 7B

Source: arXiv

SAE Interpretability Illusion

Published 5/1/2025

Analyzed 1/14/2026

Sparse Autoencoders (SAEs), utilized for interpreting the internal residual stream activations of Large Language Models (LLMs) into human-understandable concepts, are vulnerable to adversarial input perturbations. By employing gradient-based optimization techniques adapted for SAEs (specifically a generalized Greedy Coordinate Gradient), an attacker can craft inputs via suffix appending or token replacement that manipulate the SAE's latent feature activations. This vulnerability allows for the…

Interpretability Illusions with Sparse Autoencoders: Evaluating Robustness of Concept Representations

Evaluated models: Llama 3 8B, Gemma 2 9B

Source: arXiv

SLM Imperceptible Noise Jailbreak

Published 5/1/2025

Analyzed 12/8/2025

An adversarial audio perturbation vulnerability exists in open-source Speech Language Models (SLMs), specifically Qwen2-Audio-7B-Instruct and LLaMa-Omni. The vulnerability allows remote attackers to bypass safety alignment mechanisms and jailbreak the model by injecting imperceptible adversarial noise into audio prompts. By utilizing white-box access and Projected Gradient Descent (PGD) optimization, an attacker can manipulate the continuous speech signal to trigger harmful responses (e.g…

SPIRIT: Patching Speech Language Models against Jailbreak Attacks

Evaluated models: Qwen 2 7B

Source: arXiv

Related research categories