Impact

Safety Research

Security concerns affecting AI safety and alignment

798 matching entries

Safety research results

798 entries

Bee Path Planning Jailbreak

Published Nov 5, 2025Analyzed Dec 8, 2025Source: arXiv

Large Language Models (LLMs) are vulnerable to a multi-turn jailbreak attack orchestrated by an enhanced Artificial Bee Colony (ABC) algorithm. This vulnerability exists because current safety alignment mechanisms (such as RLHF and DPO) can be bypassed by treating the attack process as a path planning problem on a dynamically weighted graph topology. The ABC algorithm automates the search for adversarial dialogue trajectories by maintaining a population of "bees" (candidate attack paths) that…

Let the Bees Find the Weak Spots: A Path Planning Perspective on Multi-Turn Jailbreak Attacks against LLMs

Evaluated models: GPT-3.5, GPT-4, Llama 2 7B+1 more

GPT-3.5
GPT-4
Llama 2 7B
Llama 3.1 8B

Needle-in-Haystack Jailbreak

Published Nov 5, 2025Analyzed Dec 8, 2025Source: arXiv

A safety bypass vulnerability, dubbed "Ninja" (Needle-in-a-haystack jailbreak), exists in long-context Large Language Models (LLMs). The vulnerability exploits a degradation in safety alignment that occurs when a harmful goal is embedded within a massive, benign context window. Unlike traditional adversarial attacks that use unintelligible strings or "many-shot" attacks that use harmful examples, this method utilizes thematically relevant but innocuous text (the "haystack"). The attack…

Jailbreaking in the Haystack

Evaluated models: GPT-4o, Llama 3.1 8B Instruct, Gemini 2.0 Flash+2 more

GPT-4o
Llama 3.1 8B Instruct
Gemini 2.0 Flash
Mistral 7B v0.3
Qwen 2.5 7B Instruct

Pervasive Multi-turn Jailbreaks

Published Nov 5, 2025Analyzed Dec 8, 2025Source: arXiv

Multiple open-weight Large Language Models (LLMs)—specifically those prioritizing capability over safety alignment—exhibit a critical vulnerability to adaptive multi-turn prompt injection and jailbreak attacks. While these models effectively reject isolated, single-turn adversarial inputs (averaging ~13.11% Attack Success Rate), they fail to maintain safety guardrails and policy enforcement across extended conversational contexts. By leveraging iterative strategies such as "Crescendo" (gradual…

Death by a Thousand Prompts: Open Model Vulnerability Analysis

Evaluated models: GPT-oss 20B, Llama 3.3 70B Instruct, Mistral Large 2+5 more

GPT-oss 20B
Llama 3.3 70B Instruct
Mistral Large 2
DeepSeek V3.1
Qwen 3 32B
Gemma 3 1B
Phi-4
GLM 4.5 Air

Autonomous Jailbreak Evolution

Published Nov 4, 2025Analyzed Dec 8, 2025Source: arXiv

Large Language Models (LLMs) are vulnerable to an automated, self-evolving jailbreak attack framework known as ASTRA (Automated Strategy Discovery, Retrieval, and Evolution). This vulnerability exists because current safety alignment mechanisms (such as RLHF) fail to generalize defenses against dynamically distilled and retrieved attack strategies. ASTRA operates on a closed-loop "attack-evaluate-distill-reuse" mechanism. An attacker LLM generates prompts; a separate strategy extractor…

ASTRA: An Automated Framework for Strategy Discovery, Retrieval, and Evolution for Jailbreaking LLMs

Evaluated models: Llama 3 8B Instruct, Llama 3 70B Instruct, DeepSeek R1 0528+5 more

Llama 3 8B Instruct
Llama 3 70B Instruct
DeepSeek R1 0528
GPT-4o 2024-08-06
GPT-4.1 2025-04-14
Gemini 2.0 Flash 001
Gemini 2.5 Flash Preview 04-17
Claude 3.7 Sonnet 20250219

Meta-Optimized LLM Judge Jailbreak

Published Nov 3, 2025Analyzed Nov 20, 2025Source: arXiv

A vulnerability in Large Language Models (LLMs) allows for systematic jailbreaking through a meta-optimization framework called AMIS (Align to MISalign). The attack uses a bi-level optimization process to co-evolve both the jailbreak prompts and the scoring templates used to evaluate them.

Align to Misalign: Automatic LLM Jailbreak with Meta-Optimized LLM Judges

Evaluated models: Claude 3.5 Haiku, Claude 3.5 Sonnet, Claude Sonnet 4+3 more

Claude 3.5 Haiku
Claude 3.5 Sonnet
Claude Sonnet 4
GPT-4o
GPT-4o Mini
Llama 3.1 8B Instruct

Latent Space Discontinuity Exploitation

Published Nov 1, 2025Analyzed Dec 5, 2025Source: arXiv

A vulnerability exists in certain Large Language Models and diffusion models due to discontinuities in their latent space, which arise from data sparsity during training. An attacker can craft inputs containing lexically rare or semantically ambiguous constructs to guide the model's inference process toward these unstable, poorly-conditioned regions. This technique, termed "Alignment Degradation Induction," can degrade or bypass safety alignment mechanisms. Through iterative, multi-turn…

Exploiting Latent Space Discontinuities for Building Universal LLM Jailbreaks and Data Extraction Attacks

Evaluated models: Not reported

Diffusion LLM Direct Jailbreaking

Published Oct 31, 2025Analyzed Nov 20, 2025Source: arXiv

A vulnerability exists where non-autoregressive Diffusion Language Models (DLLMs) can be leveraged to generate highly effective and transferable adversarial prompts against autoregressive LLMs. The technique, named INPAINTING, reframes the resource-intensive search for adversarial prompts into an efficient, amortized inference task. By providing a desired harmful or restricted response to a DLLM, the model can conditionally generate a corresponding low-perplexity prompt that elicits that…

Diffusion LLMs are Natural Adversaries for any LLM

Evaluated models: Gemma 3 1B, GPT-5, LLaDA 8B Base+5 more

Gemma 3 1B
GPT-5
LLaDA 8B Base
Llama 3 8B
Llama 3 8B Instruct
Phi 4 Mini
Qwen 2.5 7B
Vicuna 13B v1.5

LLM Self-Harm Loop

Published Oct 31, 2025Analyzed Dec 8, 2025Source: arXiv

Large Language Models (LLMs), specifically GPT-3.5-turbo, LLaMA3-8B-instruct, and DeepSeek-R1-Distill-Qwen-7B, are vulnerable to a "Self-Harm" jailbreak attack (Self-HarmLLM). This vulnerability exploits the model's ability to understand its own safety boundaries to generate adversarial inputs against itself. An attacker utilizes a two-session approach: in the first session (Mitigation Session), the attacker instructs the model to rewrite a harmful query into a "Mitigated Harmful Query"…

Self-HarmLLM: Can Large Language Model Harm Itself?

Evaluated models: GPT-3.5 Turbo, Llama 3 8B Instruct, DeepSeek R1 Distill Qwen 7B

Mobile Agent Channel Subversion

Published Oct 31, 2025Analyzed Dec 30, 2025Source: arXiv

Mobile LLM-based agents (including Mobile-Agent-E, AppAgent, AutoDroid, and others) are vulnerable to indirect prompt injection attacks delivered via untrusted third-party mobile channels, such as in-app advertisements, system notifications, and embedded webviews. These agents utilize Multimodal Large Language Models (MLLMs) to perceive the device state via screenshots or accessibility trees. The vulnerability exists because the agents concatenate the user's prompt ($p$) with the environmental…

Measuring the Security of Mobile LLM Agents under Adversarial Prompts from Untrusted Third-Party Channels

Evaluated models: GPT-3.5 Turbo, GPT-4 Turbo, GPT-4o+1 more

GPT-3.5 Turbo
GPT-4 Turbo
GPT-4o
GPT-4.1 Mini

Internal Prompt Leak Injection

Published Oct 27, 2025Analyzed Dec 30, 2025Source: arXiv

LLM-based coding agents integrated into IDEs (e.g., VS Code Copilot, Cursor, Windsurf) are vulnerable to a query-agnostic Indirect Prompt Injection (IPI) attack termed "QueryIPI." This vulnerability allows an attacker to achieve Remote Code Execution (RCE) on the developer's machine by injecting a malicious tool definition (e.g., via the Model Context Protocol) into the agent's context.

QueryIPI: Query-agnostic Indirect Prompt Injection on Coding Agents

Evaluated models: Claude Sonnet 4

Related research categories