Impact

Safety Research

Security concerns affecting AI safety and alignment

798 matching entries

Safety research results

798 entries

Embodied Cross-Modal Misalignment

Published Nov 20, 2025Analyzed Jan 14, 2026Source: arXiv

OpenVLA, a Vision-Language-Action (VLA) model, contains a vulnerability regarding multimodal adversarial robustness. The model lacks sufficient cross-modal alignment stability, allowing attackers to disrupt the grounding between visual perception and linguistic instructions. By utilizing the "VLA-Fool" framework, adversaries can inject perturbations via three vectors: (1) Semantically Greedy Coordinate Gradient (SGCG), which alters specific linguistic tokens (referential cues, attributes…

When alignment fails: Multimodal adversarial attacks on vision-language-action models

Evaluated models: Not reported

Game-Theoretic LLM Defection

Published Nov 20, 2025Analyzed Dec 8, 2025Source: arXiv

Large Language Models (LLMs) are vulnerable to a black-box jailbreak technique known as the "Game-Theory Attack" (GTA). This vulnerability exploits the model's instruction-following and objective-shaping capabilities by framing the interaction as a finite-horizon sequential stochastic game. By introducing a game-theoretic scenario template—specifically a "Mechanism-Induced Graded Prisoner’s Dilemma"—the attacker alters the model's effective payoff structure. The vulnerability triggers a…

" To Survive, I Must Defect": Jailbreaking LLMs via the Game-Theory Scenarios

Evaluated models: Not reported

Indirect Environmental Jailbreak

Published Nov 20, 2025Analyzed Dec 30, 2025Source: arXiv

Embodied Artificial Intelligence (AI) agents utilizing Vision-Language Models (VLMs) for perception and planning are vulnerable to Indirect Environmental Jailbreak (IEJ). The vulnerability arises from the system's failure to distinguish between user-issued instructions and text embedded in the physical environment (e.g., writing on walls, sticky notes, or projections). The VLM processes visual text detected in the camera feed as authoritative context or direct commands, allowing a black-box…

The Shawshank Redemption of Embodied AI: Understanding and Benchmarking Indirect Environmental Jailbreaks

Evaluated models: GPT-4o, Qwen3-VL Plus, Gemini 2.0 Flash+3 more

GPT-4o
Qwen3-VL Plus
Gemini 2.0 Flash
GLM 4.5
DeepSeek-VL2
Claude 3.5 Sonnet

LLM Agent Automates Backdoor Injection

Published Nov 20, 2025Analyzed Dec 1, 2025Source: arXiv

A vulnerability in the fine-tuning process of Large Language Models (LLMs) allows for the automated generation of stealthy backdoor attacks using an autonomous LLM agent. This method, termed AutoBackdoor, creates a pipeline to generate semantically coherent trigger phrases and corresponding poisoned instruction-response pairs. Unlike traditional backdoor attacks that rely on fixed, often anomalous triggers, this technique produces natural language triggers that are contextually relevant and…

AutoBackdoor: Automating Backdoor Attacks via LLM Agents

Evaluated models: GPT-4o, GPT-4o Mini, Llama 3.1 8B Instruct+3 more

GPT-4o
GPT-4o Mini
Llama 3.1 8B Instruct
Mistral 7B Instruct v0.3
Qwen 2.5 14B Instruct
Qwen 2.5 7B Instruct

Adversarial Poetry Jailbreak

Published Nov 19, 2025Analyzed Dec 9, 2025Source: arXiv

Large Language Models (LLMs) from multiple vendors are vulnerable to a "poetic jailbreak" attack, a form of stylistic obfuscation where safety guardrails are bypassed by formatting harmful requests as poetry. By encoding prohibited instructions (e.g., malware creation, CBRN protocols) into verse—utilizing metaphors, rhyme schemes, and rhythmic structure—an attacker can evade intent recognition heuristics. The model perceives the input primarily as a creative writing constraint rather than a…

Adversarial Poetry as a Universal Single-Turn Jailbreak Mechanism in Large Language Models

Evaluated models: DeepSeek Chat V3.1, DeepSeek V3.2 Exp, Qwen 3 32B+22 more

DeepSeek Chat V3.1
DeepSeek V3.2 Exp
Qwen 3 32B
Gemini 2.5 Flash
Kimi K2
Gemini 2.5 Pro
Gemini 2.5 Flash-Lite
DeepSeek R1
Magistral Medium 2506
Qwen 3 Max
Mistral Large 2411
Mistral Small 3.2 24B Instruct
Llama 4 Maverick
Llama 4 Scout
Kimi K2 Thinking
Grok 4 Fast
GPT-oss 20B
Grok 4
GPT-oss 120B
Claude Sonnet 4.5
GPT-5
Claude Opus 4.1
GPT-5 Mini
GPT-5 Nano
Claude Haiku 4.5

Conceptual Triggers Bypass Safety

Published Nov 19, 2025Analyzed Dec 5, 2025Source: arXiv

Large Language Models are vulnerable to a conceptual manipulation attack, termed Morphology Inspired Conceptual Manipulation (MICM), that bypasses standard safety filters to generate content aligned with harmful extremist ideologies. The attack does not use explicit keywords or standard jailbreak syntax. Instead, it embeds a curated set of seemingly innocuous phrases, called Concept-embedded Triggers (CETs), into a prompt template. These CETs represent an abstract "conceptual configuration" of…

When Harmless Words Harm: A New Threat to LLM Safety via Conceptual Triggers

Evaluated models: DeepSeek R1, DeepSeek V3, GPT-4o+3 more

DeepSeek R1
DeepSeek V3
GPT-4o
GPT-4o Mini
Mistral 7B v0.3
Qwen 3 8B

Embedded Templates Bypass Moderation

Published Nov 18, 2025Analyzed Dec 1, 2025Source: arXiv

A jailbreak vulnerability, termed Embedded Jailbreak Template (EJT), allows for the generation of harmful content by bypassing the safety mechanisms of Large Language Models (LLMs). The attack uses a generator LLM to contextually integrate a harmful query into a pre-existing jailbreak template. Unlike fixed templates which insert a query into a static placeholder, EJT rewrites multiple parts of the template to embed the harmful intent naturally. This process preserves the original template's…

Beyond Fixed and Dynamic Prompts: Embedded Jailbreak Templates for Advancing LLM Security

Evaluated models: BERT, DeBERTa v3 Base, GPT-4o

Evolutionary Language Model Jailbreak

Published Nov 17, 2025Analyzed Dec 1, 2025Source: arXiv

A vulnerability exists in aligned Large Language Models (LLMs) that can be exploited by the FORGEDAN evolutionary framework to bypass safety and alignment mechanisms. The attack, which operates in a black-box setting, uses a genetic algorithm to automatically evolve effective jailbreak prompts. The framework combines multi-strategy textual perturbations (at the character, word, and sentence levels) with a semantic fitness function based on RoBERTa embeddings. This allows it to iteratively…

FORGEDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models

Evaluated models: DeepSeek V3, Gemma 2 9B, Qwen 2.5 7B+2 more

DeepSeek V3
Gemma 2 9B
Qwen 2.5 7B
RoBERTa
Transpec 13B

ITS Typography Jailbreak

Published Nov 17, 2025Analyzed Dec 8, 2025Source: arXiv

Large Vision Language Models (LVLMs) are vulnerable to a jailbreaking attack that combines image typography manipulation with multi-turn prompting. The vulnerability exploits the model's visual encoder and instruction-following capabilities by embedding a harmful textual query directly into a benign image as a visible caption (using specific fonts and blending techniques). An attacker then engages the model in a three-turn conversation: first asking a benign question about the visual object…

Jailbreaking Large Vision Language Models in Intelligent Transportation Systems

Evaluated models: GPT-4o, Qwen 2 7B, LLaVA 7B

EvoSynth: Evolutionary Attack Synthesis

Published Nov 16, 2025Analyzed Dec 1, 2025Source: arXiv

Large Language Models (LLMs) are vulnerable to a novel class of jailbreak attacks generated through the evolutionary synthesis of executable, code-based attack algorithms. Unlike traditional methods that refine or combine static prompts, this technique uses an automated multi-agent system (EvoSynth) to autonomously engineer and evolve the underlying code that generates the attack. These generated algorithms exhibit high structural and dynamic complexity, using features like control flow, state…

Evolve the Method, Not the Prompts: Evolutionary Synthesis of Jailbreak Attacks on LLMs

Evaluated models: Claude Sonnet 4.5, DeepSeek V3.2 Exp, GPT-4o+7 more

Claude Sonnet 4.5
DeepSeek V3.2 Exp
GPT-4o
GPT-5 Chat
Llama 3.1 70B Instruct
Llama 3.1 8B Instruct
Llama Guard 2 8B
Llama Guard 3 8B
Llama Guard 4 12B
Qwen Max

Related research categories