Impact

Safety Research

Security concerns affecting AI safety and alignment

798 matching entries

Safety research results

798 entries

Tag-Along Agent Jailbreak

Published Feb 2, 2026Analyzed Feb 21, 2026Source: arXiv

A vulnerability exists in tool-augmented Large Language Model (LLM) agents characterized as "Tag-Along Attacks," where an unprivileged external user (or adversarial agent) coerces a safety-aligned Operator agent into executing prohibited tool calls. Unlike Indirect Prompt Injection, this attack targets the direct conversational interface using a technique termed "Imperative Overloading." By mimicking system prompt syntax and utilizing high-priority imperative commands (e.g., "Strict adherence…

David vs. Goliath: Verifiable Agent-to-Agent Jailbreaking via Reinforcement Learning

Evaluated models: Qwen 2.5 32B Instruct AWQ, DeepSeek V3.1, Gemini 2.5 Flash+9 more

Qwen 2.5 32B Instruct AWQ
DeepSeek V3.1
Gemini 2.5 Flash
Qwen 2.5 7B Instruct
Meta-SecAlign 8B
Qwen 2.5 14B Instruct
Qwen 2.5 72B Instruct AWQ
Llama 3.1 8B Instruct
Llama 3.1 70B Instruct AWQ
Llama 4 Maverick
GPT-5 Nano
Claude 3 Haiku

SS-Neuron Cross-Lingual Jailbreak

Published Feb 1, 2026Analyzed Feb 21, 2026Source: arXiv

Large Language Models (LLMs) exhibit a cross-lingual safety vulnerability driven by a dependency on a sparse subset of "Shared Safety Neurons" (SS-Neurons) anchored in high-resource (HR) languages, typically English. Non-high-resource (NHR) languages lack autonomous safety mechanisms and rely on projecting inputs onto this English-aligned safety manifold to trigger refusals. Because this projection is imperfect, safety guardrails can be bypassed by translating malicious prompts into NHR…

Who Transfers Safety? Identifying and Targeting Cross-Lingual Shared Safety Neurons

Evaluated models: Llama 3.1 8B Instruct, Qwen 3 8B, Gemma 2 9B IT

UltraBreak Universal VLM Jailbreak

Published Feb 1, 2026Analyzed Feb 20, 2026Source: arXiv

Vision-Language Models (VLMs) are vulnerable to a universal and transferable adversarial attack dubbed "UltraBreak." This vulnerability allows remote attackers to bypass safety alignment filters and elicit harmful responses (e.g., hate speech, dangerous instructions) by supplying a single, specifically crafted adversarial image alongside a text query. Unlike traditional gradient-based attacks that optimize for specific token sequences (cross-entropy loss) and result in brittle…

Toward Universal and Transferable Jailbreak Attacks on Vision-Language Models

Evaluated models: Qwen VL Chat, Qwen2-VL 7B Instruct, Qwen 2.5 VL 7B Instruct+6 more

Qwen VL Chat
Qwen2-VL 7B Instruct
Qwen 2.5 VL 7B Instruct
LLaVA v1.6 Mistral 7B
Kimi VL A3B Instruct
GLM-4.1V 9B Thinking
GPT-4.1 Nano
Gemini 2.5 Flash-Lite
Claude 3 Haiku

Activation Delta Detector Evasion

Published Jan 31, 2026Analyzed Mar 9, 2026Source: arXiv

Activation-delta-based linear probes used for detecting task drift and prompt injections in Large Language Models (LLMs) can be bypassed using universal adversarial suffixes. By utilizing the Greedy Coordinate Gradient (GCG) algorithm, an attacker can generate a single, optimized suffix that simultaneously fools multiple logistic regression classifiers attached to different hidden layers of the LLM. The attack jointly optimizes the suffix tokens by accumulating gradients from the losses of all…

Bypassing Prompt Injection Detectors through Evasive Injections

Evaluated models: Llama 3 8B, Phi-3 8B

Adversarial Claim Search Deception

Published Jan 31, 2026Analyzed Feb 22, 2026Source: arXiv

Search-enabled Large Language Model (LLM) fact-checking systems are vulnerable to adversarial claim attacks that exploit the pipeline's reliance on claim interpretation, query formulation, and dynamic evidence retrieval. By manipulating the linguistic structure of an input claim while preserving its semantic factual intent, an attacker can induce systematic verification failures. This vulnerability stems from three specific attack surfaces: 1. Search Engine Misguidance: Altering lexical…

DECEIVE-AFC: Adversarial Claim Attacks against Search-Enabled LLM-based Fact-Checking Systems

Evaluated models: GPT-4o

Causal Driver Jailbreak Enhancement

Published Jan 31, 2026Analyzed Feb 20, 2026Source: arXiv

Large Language Models (LLMs), including Qwen2.5, LLaMA-3, and Baichuan2, are vulnerable to causally optimized adversarial attacks where specific interpretable prompt features are manipulated to bypass safety alignment. Research utilizing a "Causal Analyst" framework reveals that specific prompt attributes—specifically "Number of Task Steps" (increasing procedural complexity), "Positive Character" (enforcing specific personas), and "Command Tone"—act as direct causal drivers for "Answer…

A Causal Perspective for Enhancing Jailbreak Attack and Defense

Evaluated models: GPT-4o, Qwen 2.5 7B

Gradient Shift Jailbreak

Published Jan 31, 2026Analyzed Feb 20, 2026Source: arXiv

Large Language Models (LLMs) and Large Reasoning Models (LRMs) are vulnerable to an inference-time jailbreak attack known as "Hybrid Gradient Shift." This vulnerability exploits the statistical discrepancy between an aligned model's output distribution and a pre-alignment reference distribution. By treating safety alignment as a form of miscalibration, an attacker can aggregate the logits of a target model (aligned), a helper model (unaligned/weak), and a predictor model to reconstruct the…

Jailbreaking LLMs via Calibration

Evaluated models: GPT-oss 120B, Llama 3.3 70B Instruct

OCR Image Distraction Jailbreak

Published Jan 31, 2026Analyzed Feb 20, 2026Source: arXiv

Large Vision-Language Models (LVLMs) possessing Optical Character Recognition (OCR) capabilities are vulnerable to a "Text Distraction Jailbreaking" (Text-DJ) attack. The vulnerability exploits a gap between the model's visual text extraction and its safety alignment mechanisms. By converting a decomposed harmful textual query into images and embedding these images within a grid of semantically irrelevant "distraction" text images, an attacker can bypass safety filters. The model's OCR…

Text is All You Need for Vision-Language Model Jailbreaking

Evaluated models: GPT-4o Mini, GPT-4.1 Mini, Gemini 2.5 Flash+3 more

GPT-4o Mini
GPT-4.1 Mini
Gemini 2.5 Flash
Qwen3-VL 4B Instruct
Qwen3-VL 8B Instruct
Qwen3-VL 30B-A3B Instruct

Unified Robustness Gap

Published Jan 31, 2026Analyzed Mar 9, 2026Source: arXiv

Text scoring models, including dense retrievers, rerankers, and reward models, are vulnerable to score manipulation attacks via search-based discrete perturbations and content injection. An attacker can systematically modify candidate texts using rudimentary string manipulations, gradient-guided token swaps (e.g., HotFlip), masked language modeling (MLM) swaps, or query/sentence injections to spuriously increase model scores. This structural failure condition allows an irrelevant passage or a…

Unifying Adversarial Robustness and Training Across Text Scoring Models

Evaluated models: E5 BERT-base, Qwen 3 0.6B, Llama 3.2 3B Instruct+2 more

E5 BERT-base
Qwen 3 0.6B
Llama 3.2 3B Instruct
Llama 3.1 8B Instruct
Skywork Reward V2

Alignment Curse Jailbreak Transfer

Published Jan 30, 2026Analyzed Mar 9, 2026Source: arXiv

End-to-end multimodal large language models (omni-models) that utilize a shared representation space for text and audio are vulnerable to cross-modality jailbreak transfer, a phenomenon termed the "alignment curse." Because these models are trained to strongly align audio and text embeddings in their mid-to-late layers, an attacker can reliably bypass audio-specific safety mechanisms by converting mature, text-based jailbreak prompts into audio using standard Text-to-Speech (TTS) tools. When…

The Alignment Curse: Cross-Modality Jailbreak Transfer in Omni-Models

Evaluated models: GPT-4o, Qwen 2.5 3B

Related research categories