Impact

Safety Research

Security concerns affecting AI safety and alignment

798 matching entries

Safety research results

798 entries

LLM Virtual Criminal Agents

Published Jan 20, 2026Analyzed Mar 8, 2026Source: arXiv

A vulnerability exists in the safety alignment of state-of-the-art Large Language Models (LLMs) when deployed as autonomous agents in dynamic, interactive environments. While current safety guardrails effectively block static, single-turn harmful queries, they fail to prevent multi-step emergent criminal behavior in agentic loops. When situated in an open-ended sandbox simulation (such as the VirtualCrime framework), these LLMs successfully bypass alignment to proactively plan, coordinate, and…

VirtualCrime: Evaluating Criminal Potential of Large Language Models via Sandbox Simulation

Evaluated models: GPT-4.1 2025-04-14, GPT-5 Chat 2025-10-03, Claude Haiku 4.5 20251001+5 more

GPT-4.1 2025-04-14
GPT-5 Chat 2025-10-03
Claude Haiku 4.5 20251001
Claude Sonnet 4.5 20250929
Gemini 2.5 Pro
DeepSeek R1 0528
Doubao 1.6 Thinking 250715
Qwen 3 Max

MLLM Over-Reasoning Safety Risk

Published Jan 20, 2026Analyzed Feb 21, 2026Source: arXiv

Multimodal Large Language Models (MLLMs) exhibit a vulnerability to "Reasoning-based Multi-Image Attacks," where safety guardrails are bypassed by distributing harmful intent across multiple images (2–4 inputs). Unlike single-image jailbreaks that rely on visual obfuscation, this vulnerability exploits the model's reasoning capabilities. By presenting images that share a specific relationship (e.g., Temporal Jump, Spatial Juxtaposition, or Causality), an attacker can compel the model to infer…

The Side Effects of Being Smart: Safety Risks in MLLMs' Multi-Image Reasoning

Evaluated models: GPT-4o, GPT-4o Mini, Gemini 1.5 Pro+11 more

GPT-4o
GPT-4o Mini
Gemini 1.5 Pro
Gemini 1.5 Flash
Qwen 2.5 VL 3B Instruct
Qwen 2.5 VL 32B Instruct
LLaVA 1.5 7B
Llama 3 LLaVA-NeXT 8B
InternVL3 8B
InternVL3 38B
InternVL3 78B
MiniCPM-o 2.6
Skywork-R1V3 38B
GLM-4.1V 9B Thinking

Metacognitive Prompting Lowers Resistance

Published Jan 20, 2026Analyzed Feb 22, 2026Source: arXiv

Large Language Models (LLMs) are vulnerable to multi-turn persuasive conversational attacks that induce the adoption of counterfactual beliefs. By leveraging the Source–Message–Channel–Receiver (SMCR) communication framework, attackers can systematically erode a model's confidence in established facts and compel the model to output misinformation. Specific attack vectors include manipulating source attribution (authority framing), message content (logical, credibility, or emotional appeals)…

Vulnerability of LLMs' Belief Systems? LLMs Belief Resistance Check Through Strategic Persuasive Conversation Interventions

Evaluated models: GPT-4o, Llama 3.2 3B, Llama 3.3 70B+2 more

GPT-4o
Llama 3.2 3B
Llama 3.3 70B
Mistral 7B
Qwen 2.5 7B

Physical Navigation Prompt Injection

Published Jan 20, 2026Analyzed Feb 21, 2026Source: arXiv

LLM-based navigation agents, including NavGPT and prompt-tuned outdoor agents, are vulnerable to adaptive prompt injection attacks. This vulnerability allows remote attackers to hijack the physical movement of the agent by embedding optimized malicious instructions into benign natural language inputs. The issue arises because the agents parse user instructions to generate executable plans without sufficient separation between control logic and untrusted input. The PINA (Prompt Injection Attack…

PINA: Prompt Injection Attack against Navigation Agents

Evaluated models: GPT-3.5, GPT-4, Llama 2 7B

Self-Evolving Red-Team Agents

Published Jan 20, 2026Analyzed Mar 8, 2026Source: arXiv

A vulnerability in the safety alignment of several major Large Language Models (LLMs) allows attackers to bypass content filters using complex, automatically generated adversarial prompts. Discovered via the AgenticRed evolutionary framework, the flaw is exploited by wrapping malicious intents in structured formats (such as strict JSON output contracts), combined with prefix injection and refusal suppression. By explicitly commanding the model to begin its response with a compliant prefix and…

AgenticRed: Evolving Agentic Systems for Red-Teaming

Evaluated models: GPT-3.5 Turbo, GPT-4o, GPT-5.1+9 more

GPT-3.5 Turbo
GPT-4o
GPT-5.1
GPT-5.2
Claude 3.5 Sonnet
Claude Haiku 4.5
DeepSeek R1
DeepSeek V3.2
Qwen 3 Max
Qwen 3 8B
Llama 2 7B
Llama 3 8B

Detector Telemetry Camouflage

Published Jan 19, 2026Analyzed Feb 21, 2026Source: arXiv

Single-pass hallucination detectors relying on internal telemetry (uncertainty, hidden-state geometry, and attention patterns) are vulnerable to white-box, model-side adversarial attacks. An attacker can employ the CORVUS (Camouflaging Open-weight Representations, Volumes, Uncertainty, and Structure) technique to fine-tune lightweight Low-Rank Adapters (LoRA) on the target LLM. This method optimizes a specific loss objective that camouflages detector-visible telemetry signals—specifically…

CORVUS: Red-Teaming Hallucination Detectors via Internal Signal Camouflage in Large Language Models

Evaluated models: Llama 2 7B, Llama 3 8B, Qwen 2.5 14B+1 more

Llama 2 7B
Llama 3 8B
Qwen 2.5 14B
Vicuna 7B

Drunk Language Jailbreak

Published Jan 19, 2026Analyzed Mar 8, 2026Source: arXiv

A vulnerability exists in aligned Large Language Models (LLMs) where inducing "drunk language" behavior—simulating the text of an intoxicated human—bypasses safety guardrails and contextual privacy protections. Attackers can exploit this anthropomorphic flaw through inference-time persona prompting or lightweight post-training (causal fine-tuning or reinforcement learning on drunk text corpora). By forcing the model to adopt a stylistic and semantic framework associated with impaired human…

In Vino Veritas and Vulnerabilities: Examining LLM Safety via Drunk Language Inducement

Evaluated models: GPT-3.5, GPT-4, GPT-4o+3 more

GPT-3.5
GPT-4
GPT-4o
Llama 2 7B
Llama 3 8B
Mistral 7B

MLLM Chart Deception

Published Jan 19, 2026Analyzed Feb 21, 2026Source: arXiv

Code-generation Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) are vulnerable to directed misuse for the generation of misleading data visualizations. This vulnerability, described as the "ChartAttack" framework, allows an attacker to prompt the model to manipulate chart annotation code (e.g., JSON specifications for Matplotlib or Vega-Lite) to apply specific "misleaders"—design choices that distort data interpretation without altering the underlying data values. By…

ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation

Evaluated models: Qwen 2.5 14B, LLaVA 7B, Phi-3

Sockpuppet Prefix Jailbreak

Published Jan 19, 2026Analyzed Feb 21, 2026Source: arXiv

Large Language Models (LLMs), specifically open-weight instruction-tuned models (including Llama-3.1-8B, Qwen3-8B, and Gemma-7B) and certain closed-weight APIs allowing partial response pre-filling, are vulnerable to "Sockpuppetting" or Output Prefix Injection. This vulnerability exploits the model's autoregressive nature and self-consistency bias. By injecting a target acceptance sequence (e.g., "Sure, here is...") directly into the start of the assistant message block within the chat…

Sockpuppetting: Jailbreaking LLMs by Combining Prefilling with Optimization

Evaluated models: Llama 3.1 8B, Qwen 3 8B, Gemma 7B

VLM In-the-Loop Adversary

Published Jan 19, 2026Analyzed Mar 9, 2026Source: arXiv

The VILTA (VLM-in-the-Loop Trajectory Adversary) framework is vulnerable to Prompt Injection and Data Poisoning via un-sanitized scene representation inputs. The system integrates a Vision-Language Model (Gemini-2.5-Flash) into a closed-loop reinforcement learning environment, feeding it Bird’s-Eye-View (BEV) imagery alongside text-based vehicle dynamics data (e.g., position, speed, and risk_category) to generate challenging driving trajectories. An attacker who can manipulate the input…

VILTA: A VLM-in-the-Loop Adversary for Enhancing Driving Policy Robustness

Evaluated models: Gemini 2.5 Flash

Related research categories