Impact

Safety Research

Security concerns affecting AI safety and alignment

798 matching entries

Safety research results

798 entries

Maladaptive Therapeutic Reinforcement

Published Apr 6, 2026Analyzed Apr 10, 2026Source: arXiv

Large Language Models (LLMs) aligned for helpfulness and empathy are vulnerable to a Persona-based Client Simulation Attack (PCSA) that exploits the model's inability to distinguish therapeutic empathy from maladaptive validation. By embedding harmful intents within coherent, multi-turn psychological counseling narratives and employing clinical resistance strategies (such as intellectualization or metaphorical expression), attackers can compel the model to prioritize rapport-building over…

Do No Harm: Exposing Hidden Vulnerabilities of LLMs via Persona-based Client Simulation Attack in Psychological Counseling

Evaluated models: GPT-3.5 Turbo, GPT-5.1, Llama 3.1 8B+5 more

GPT-3.5 Turbo
GPT-5.1
Llama 3.1 8B
Llama 3.1 70B
Crispers 7B
Psycho 8B
Qwen 3 14B
Qwen 2.5 72B

LLM Robot JSON Backdoor

Published Apr 4, 2026Analyzed Apr 10, 2026Source: arXiv

A supply-chain vulnerability in LLM-mediated robotic control systems allows attackers to execute unauthorized physical actions via structured backdoor attacks embedded in LoRA adapters. By poisoning the fine-tuning dataset to map specific natural-language trigger phrases directly to malicious, syntactically valid JSON control commands (structured-output poisoning), the backdoor bypasses natural-language reasoning layers and propagates deterministically to downstream robotic middleware (e.g…

From Prompt to Physical Action: Structured Backdoor Attacks on LLM-Mediated Robotic Control Systems

Evaluated models: Llama 3.1 8B Instruct, Gemma 2 9B IT, DeepSeek R1 Distill Llama 8B+2 more

Llama 3.1 8B Instruct
Gemma 2 9B IT
DeepSeek R1 Distill Llama 8B
Mistral 7B Instruct v0.3
Llama 3 8B

Multi-Strategy Prompt Evasion

Published Apr 4, 2026Analyzed Apr 10, 2026Source: arXiv

A vulnerability in LLM input filtering mechanisms allows attackers to bypass keyword, semantic, and state-of-the-art intent-aware defenses using composite prompt injections. By combining Obfuscation (OBF) techniques with Semantic/Social manipulation—specifically Emotional Manipulation (EM) or Reward Framing (RF)—attackers exploit a "representation gap" between the model and the defense. The underlying LLM decodes the obfuscated payload, while the defense mechanisms fail to parse the raw…

AttackEval: A Systematic Empirical Study of Prompt Injection Attack Effectiveness Against Large Language Models

Evaluated models: Not reported

Agent Implicit Doc Execution

Published Apr 3, 2026Analyzed Apr 10, 2026Source: arXiv

LLM-based coding agents are vulnerable to Document-Driven Implicit Payload Execution (DDIPE) via supply-chain poisoning of third-party agent skills. Attackers can embed malicious logic directly into legitimate-looking code examples and configuration templates within skill documentation files (e.g., SKILL.md). Because coding agents ingest this metadata into their context windows and treat the documentation as an authoritative reference, the underlying LLM silently reproduces and executes the…

Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems

Evaluated models: Claude Sonnet 4.6, GLM-4.7, MiniMax M2.5+2 more

Claude Sonnet 4.6
GLM-4.7
MiniMax M2.5
GPT-5.4
Gemini 2.5 Pro

Jailbreak Saturates Alignment Defenses

Published Apr 3, 2026Analyzed Apr 10, 2026Source: arXiv

A cognitive overload vulnerability in OpenAI gpt-oss-20b allows attackers to bypass instruction hierarchy and deliberative alignment safety mechanisms using "Compound Jailbreaks." By combining multiple non-contradictory but cognitively demanding tasks within a single prompt, the attack saturates the finite reasoning resources allocated for safety judgments. Because the model's safety training relies on probabilistic redistribution rather than capability elimination, this cognitive exhaustion…

Generalization Limits of Reinforcement Learning Alignment

Evaluated models: GPT-oss 20B

Observation Poisons Agent Memory

Published Apr 3, 2026Analyzed Apr 10, 2026Source: arXiv

Memory-augmented LLM web agents utilizing raw trajectory memory are vulnerable to Environment-injected Trajectory-based Agent Memory Poisoning (eTAMP). Attackers can embed malicious instructions within user-generated web content (e.g., product pages, forum posts). When the agent processes this content during a routine task, the instructions are passively ingested into its raw trajectory memory. During subsequent, entirely separate tasks on different websites, semantic retrieval mechanisms pull…

Poison Once, Exploit Forever: Environment-Injected Memory Poisoning Attacks on Web Agents

Evaluated models: GPT-5 Mini, GPT-5.2, GPT-oss 120B+3 more

GPT-5 Mini
GPT-5.2
GPT-oss 120B
Qwen 2.5 VL 72B
Qwen3-VL-32B
Qwen3.5-122B-A10B

Semantic Masking Image Jailbreak

Published Apr 2, 2026Analyzed Apr 10, 2026Source: arXiv

Multiple Text-to-Image (T2I) generation systems and their associated multi-stage moderation pipelines are vulnerable to low-effort semantic obfuscation attacks. Attackers can systematically bypass Input Compliance Checks (ICC), Semantic Safety Checks (SSC), and Post-Generation Moderation (PGM) by embedding restricted concepts into benign natural language contexts. By utilizing techniques such as Material Substitution, Artistic Reframing, Pseudo-Educational Framing, and Ambiguous Action…

Low-Effort Jailbreak Attacks Against Text-to-Image Safety Filters

Evaluated models: Sora, Stable Diffusion v1.4

Weight Orthogonalization Attack Empowerment

Published Apr 2, 2026Analyzed Apr 10, 2026Source: arXiv

An issue in large language models (LLMs) with white-box weight access allows attackers to permanently bypass safety guardrails via Weight Orthogonalization (WO). By calculating a model's "refusal vector"—the mean-difference vector between harmful and harmless instruction activations in the residual stream—an attacker can orthogonalize the model's weights to prevent it from writing to this refusal direction ($W^{\prime}\leftarrow W-rr^{\intercal}W$). Unlike jailbreak-tuning or data poisoning…

Understanding the Effects of Safety Unalignment on Large Language Models

Evaluated models: Qwen 3 4B Instruct 2507, Llama 3.1 8B Instruct, Qwen 2.5 14B+3 more

Qwen 3 4B Instruct 2507
Llama 3.1 8B Instruct
Qwen 2.5 14B
Qwen 3 4B Thinking 2507
DeepSeek R1 Distill Llama 8B
DeepSeek R1 Distill Qwen 14B

Agent Skill Injection

Published Apr 1, 2026Analyzed Apr 10, 2026Source: arXiv

LLM-based personal agents are vulnerable to Indirect Prompt Injection (IPI) defense bypasses via declarative context reframing and implicit file provenance trust. Attackers can bypass agent safety filters by phrasing malicious instructions as declarative compliance alerts rather than imperative commands. Because agents are designed to report discrepancies as expected behavior, declarative framing bypasses intent-sensitive safety mechanisms. Additionally, attackers can exploit the agent's…

ClawSafety: Safe LLMs, Unsafe Agents

Evaluated models: Claude Sonnet 4.6, Gemini 2.5 Pro, DeepSeek V3+2 more

Claude Sonnet 4.6
Gemini 2.5 Pro
DeepSeek V3
Kimi K2.5
GPT-5.1

Merging Latent Malice

Published Apr 1, 2026Analyzed Apr 10, 2026Source: arXiv

A vulnerability in LLM parameter-space merging algorithms enables a latent supply-chain attack where adversaries embed pre-computed malicious weight perturbations into open-source models. Using a constrained optimization framework, attackers inject latent components into the Multilayer Perceptron (MLP) up-projection matrices of source models. These perturbations are mathematically constrained to preserve the source model's individual safety alignment (via directional consistency) and benign…

When Safe Models Merge into Danger: Exploiting Latent Vulnerabilities in LLM Fusion

Evaluated models: Tulu-2-7B, Llama 3.1 Tulu 3 8B DPO, OpenChat 3.5 0106

Related research categories