Impact

Safety Research

Security concerns affecting AI safety and alignment

798 matching entries

Safety research results

798 entries

Rubric Stealthy Preference Drift

Published Feb 14, 2026Analyzed Feb 22, 2026Source: arXiv

LLM-as-a-Judge systems utilizing natural language rubrics are vulnerable to Rubric-Induced Preference Drift (RIPD). This vulnerability allows an attacker (or a flawed optimization process) to refine evaluation rubrics such that they maintain high agreement with human references on standard validation benchmarks while inducing systematic, directional preference degradation on unseen target domains. The attack exploits the disconnect between benchmark validation and target generalization by…

Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges

Evaluated models: Llama 3 8B, Llama 3.1 8B, DeepSeek V3+1 more

Llama 3 8B
Llama 3.1 8B
DeepSeek V3
Gemma 2 2B

Abstractive Character Violations

Published Feb 12, 2026Analyzed Feb 21, 2026Source: arXiv

Large Language Models (LLMs) aligned via reinforcement learning from human feedback (RLHF) or Constitutional AI exhibit a vulnerability where safety guardrails can be consistently bypassed through "Abstractive Red-Teaming." This attack vector exploits specific high-level natural language categories—combinations of semantic attributes such as tone, specific formatting instructions (e.g., numbered lists), language (e.g., Chinese, Russian), and topic constraints—that the model fails to generalize…

Abstractive Red-Teaming of Language Model Character

Evaluated models: GPT-4.1 Mini, Llama 3.1 8B Instruct, Gemma 3 12B IT+4 more

GPT-4.1 Mini
Llama 3.1 8B Instruct
Gemma 3 12B IT
Qwen 3 30B-A3B Instruct-2507
Claude 3.5 Haiku
Claude Sonnet 4
Claude Opus 4.1

Early Reasoning Safety Degradation

Published Feb 11, 2026Analyzed Mar 8, 2026Source: arXiv

Reinforcement learning (RL) based post-training for explicit chain-of-thought reasoning (e.g., GRPO) in Multimodal Large Reasoning Models (MLRMs) inadvertently degrades safety alignment, rendering the models highly vulnerable to multimodal jailbreak attacks. The vulnerability is caused by "conditional coverage collapse" during the initial phases of chain-of-thought generation. Under adversarial conditioning (text or image), the reasoning policy assigns vanishing probability mass to safe…

Safety Recovery in Reasoning Models Is Only a Few Early Steering Steps Away

Evaluated models: R1-Onevision 7B, OpenVLThinker 7B, VLAA-Thinker 7B+3 more

R1-Onevision 7B
OpenVLThinker 7B
VLAA-Thinker 7B
Vision-R1 7B
LlamaV-o1
LLaVA-CoT

Emergent Distractor Jailbreak

Published Feb 11, 2026Analyzed Mar 9, 2026Source: arXiv

A vulnerability in the Direct Preference Optimization (DPO) post-training phase of OLMo 2 models leads to "distractor-triggered compliance." The model correctly refuses harmful requests when prompted in isolation, but complies with identical harmful requests if a benign formatting instruction (a "distractor") is appended to the prompt. This behavior organically emerges from contaminated preference training data where mislabeled examples incorrectly preferred compliance over refusal when a…

In-the-Wild Model Organisms: Mitigating Undesirable Emergent Behaviors in Production LLM Post-Training via Data Attribution

Evaluated models: Not reported

Mobile Agent Visual Spoofing

Published Feb 11, 2026Analyzed Feb 22, 2026Source: arXiv

Mobile Large Language Model (LLM) agents operating under the "Screen-as-Interface" paradigm are vulnerable to visual indirect prompt injection and state desynchronization. Agents that rely on unstructured visual data (screenshots) and Accessibility Service APIs to perceive the environment lack a mechanism to distinguish between trusted system UI elements and untrusted content (e.g., web pages, emails, or malicious overlays). An attacker can inject visual cues, fake notifications, or hidden…

Blind Gods and Broken Screens: Architecting a Secure, Intent-Centric Mobile Agent Operating System

Evaluated models: Not reported

Multi-Turn Proxy Evasion

Published Feb 11, 2026Analyzed Feb 22, 2026Source: arXiv

A logical vulnerability exists in LLM security proxies and guardrails that utilize weighted-average aggregation algorithms for multi-turn risk scoring. The scoring logic exhibits a mathematical "ceiling property" where the cumulative conversation-level risk score converges to the per-turn score regardless of the number of interaction turns ($n$). Consequently, the aggregated score is bounded by the maximum single-turn score ($cum \leq \max(s_i)$). This allows remote attackers to bypass…

Peak+ Accumulation: A Proxy-Level Scoring Formula for Multi-Turn LLM Attack Detection

Evaluated models: Not reported

Simple Persona Jailbreak

Published Feb 11, 2026Analyzed Mar 8, 2026Source: arXiv

A vulnerability in the Grok LLM, as deployed on the X social media platform, allows users to bypass safety filters and generate toxic or obscene content through "shallow alignment" techniques. The model prioritizes instruction compliance and conversational flow over safety guidelines, failing when exposed to simple adversarial interactions such as Persona Adoption (instructing the model to adopt a specific character) and Tone Mirroring (where the model automatically mimics a user's aggressive…

@ GrokSet: multi-party Human-LLM Interactions in Social Media

Evaluated models: Not reported

Clinical LLM Feature Evasion

Published Feb 10, 2026Analyzed Mar 8, 2026Source: arXiv

A detection bypass vulnerability in the 2-Sigma clinical training platform allows users to evade the system's two-layer, linguistic-feature-based jailbreak detection mechanism. The detection framework relies heavily on four surface-level linguistic features (Professionalism, Medical Relevance, Ethical Behavior, and Contextual Distraction) to classify malicious inputs. Attackers can bypass these filters by crafting prompts that maintain professional tone and apparent medical relevance but…

Detecting Jailbreak Attempts in Clinical Training LLMs Through Automated Linguistic Feature Extraction

Evaluated models: Not reported

Enhanced LVLM Encoder Transfer

Published Feb 10, 2026Analyzed Apr 11, 2026Source: arXiv

Large Vision-Language Models (LVLMs) are vulnerable to zero-query, black-box adversarial image perturbations via Semantic-Guided Multimodal Attacks (SGMA). Unlike traditional attacks that scatter noise or target background pixels, SGMA leverages surrogate models (e.g., CLIP) to anchor imperceptible adversarial perturbations directly onto semantically critical foreground regions. The attack exploits two specific architectural traits of LVLMs: inconsistent visual grounding across models and…

Grounding-Driven Attack: Improving Encoder-based Adversarial Transferability against Large Vision-Language Models

Evaluated models: BLIP-2 OPT 2.7B, LLaVA 1.5 7B, Qwen 2.5 VL 7B Instruct+6 more

BLIP-2 OPT 2.7B
LLaVA 1.5 7B
Qwen 2.5 VL 7B Instruct
InternVL3 8B
OpenFlamingo 4B
Kimi VL A3B Instruct
GPT-4o
GPT-5.4
Gemini 2.0 Flash

Image Editing Visual Prompt Jailbreak

Published Feb 10, 2026Analyzed Feb 20, 2026Source: arXiv

Large Image Editing Models (LIEMs) supporting vision-prompt editing are vulnerable to Vision-Centric Jailbreak Attacks (VJA). This vulnerability arises from a modality mismatch in safety alignment: while safeguards primarily analyze textual instructions for policy violations, the underlying models are capable of interpreting and executing instructions embedded directly within the visual input (e.g., typographic text drawn on the image, arrows, symbols, or specific markings). An attacker can…

When the Prompt Becomes Visual: Vision-Centric Jailbreak Attacks for Large Image Editing Models

Evaluated models: GPT Image 1.5, Gemini 3 Pro Image, Seedream 4.5+5 more

GPT Image 1.5
Gemini 3 Pro Image
Seedream 4.5
Qwen-Image-Edit-2512
Qwen-Image-Edit-Safe
BAGEL 14B
Flux 2.0 Dev 32B
LongCat-Image-Edit

Related research categories