Skip to main content
LLM Security Database
Skip to research search
Last analyzed Sep 9, 2026

Language Model Security Database

985 research findings · 1119 evaluated models

Filtered research findings

610 entries

Matches every word across titles, descriptions, sources, affected systems, and models.

LLM Guardrail Bypass

Published Analyzed Source: arXiv

Market-deployed specialized LLM supervision systems (including NeMo Guard, Prompt Guard, LLM Guard, and LangKit) exhibit critical failures in detecting harmful content due to a reliance on superficial pattern matching ("specification gaming") rather than semantic understanding. These systems fail to generalize to inputs that do not match specific training patterns, resulting in near-zero detection rates for straightforward harmful prompts in categories such as CBRN (Chemical, Biological…

The bitter lesson of misuse detection
Evaluated models: GPT-3.5, GPT-4, Claude 3.5 Sonnet+4 more
  • GPT-3.5
  • GPT-4
  • Claude 3.5 Sonnet
  • Grok 2
  • Gemini 1.5 Pro
  • Mistral Large
  • DeepSeek V3

Forged Assistant Message Jailbreak

Published Analyzed Source: arXiv

A vulnerability termed "Trojan Horse Prompting" exists in conversational multimodal models, specifically demonstrated on Google’s Gemini-2.0-flash-preview-image-generation. The vulnerability allows an attacker to bypass safety alignment mechanisms (RLHF and SFT) by manipulating the structural protocol of the conversational API. Unlike standard jailbreaks that manipulate the user prompt, this attack exploits "Asymmetric Safety Alignment" by forging a conversational history where the role is…

Trojan Horse Prompting: Jailbreaking Conversational Multimodal Models by Forging Assistant Message
Evaluated models: Gemini 2.0 Flash Preview Image Generation

Multi-Agent Mole Attack

Published Analyzed Source: arXiv

A vulnerability exists in Large Language Model (LLM)-based Multi-Agent Systems (MAS) that allows a malicious agent to covertly disrupt collaborative decision-making processes without triggering standard safety filters or anomaly detection. This "intention-hiding" attack occurs when an agent adopts a persona that appears linguistically fluent and role-consistent but strategically steers the group toward incorrect outcomes or resource exhaustion. The attacker leverages specific semantic…

Who's the Mole? Modeling and Detecting Intention-Hiding Malicious Agents in LLM-Based Multi-Agent Systems
Evaluated models: GPT-4o

Response-Primed LLM Jailbreak

Published Analyzed Source: arXiv

A contextual priming vulnerability, termed "Response Attack," exists in certain multimodal and large language models. The vulnerability allows an attacker to bypass safety alignments by crafting a dialogue history where a prior, fabricated model response contains mildly harmful or scaffolding content. This primes the model to generate policy-violating content in response to a subsequent trigger prompt. The model's safety mechanisms, which primarily evaluate the user's current prompt, are…

Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Models
Evaluated models: DeepSeek R1 Distill Llama 70B, Gemini 2.0 Flash, Gemini 2.5 Flash+5 more
  • DeepSeek R1 Distill Llama 70B
  • Gemini 2.0 Flash
  • Gemini 2.5 Flash
  • GPT-4.1
  • GPT-4o
  • Llama 3 70B Instruct
  • Llama 3 8B Instruct
  • QwQ 32B

Compositional Malware Generation

Published Analyzed Source: arXiv

Aligned Large Language Models (LLMs) exhibit a "compositional blindness" vulnerability wherein safety alignment mechanisms evaluate user prompts in isolation, failing to detect malicious intent when it is systematically decomposed into multiple benign-appearing sub-tasks. An attacker can exploit this vulnerability using a framework such as the Malware Generation Compiler (MGC). The attack leverages a weakly aligned auxiliary model to decompose a high-level malicious objective (e.g…

MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generation
Evaluated models: Mistral 7B Instruct v0.3, GPT-4o Mini, Claude 3.5 Sonnet+1 more
  • Mistral 7B Instruct v0.3
  • GPT-4o Mini
  • Claude 3.5 Sonnet
  • Hermes 3 Llama 3.1 405B

LLM Suicide Prompt Jailbreak

Published Analyzed Source: arXiv

Large Language Models (LLMs) employing safety filters designed to prevent generation of content related to self-harm and suicide can be bypassed through multi-step adversarial prompting. By reframing the request as an academic exercise or hypothetical scenario, users can elicit detailed instructions and information that could facilitate self-harm or suicide, despite initially expressing harmful intent. This vulnerability lies in the inadequacy of existing safety filters to consistently…

For Argument's Sake, Show Me How to Harm Myself!': Jailbreaking LLMs in Suicide and Self-Harm Contexts
Evaluated models: Claude 3.7 Sonnet, Gemini 2.0 Flash, GPT-4o

Adaptive Cipher Jailbreak

Published Analyzed Source: arXiv

Large Language Models (LLMs) are vulnerable to obfuscation-based jailbreak attacks using the MetaCipher framework. MetaCipher employs a reinforcement learning algorithm to iteratively select from a pool of 21 ciphers to encrypt malicious keywords within prompts, evading standard safety mechanisms that rely on keyword detection. The framework adaptively learns optimal cipher choices to maximize the success rate of the jailbreak, even against LLMs with reasoning capabilities. Successful attacks…

MetaCipher: A General and Extensible Reinforcement Learning Framework for Obfuscation-Based Jailbreak Attacks on Black-Box LLMs
Evaluated models: Claude 3.7 Sonnet, DeepSeek Chat, DeepSeek R1+9 more
  • Claude 3.7 Sonnet
  • DeepSeek Chat
  • DeepSeek R1
  • Falcon 3 10B Instruct
  • Gemini 2.0 Flash
  • Gemini 2.5 Pro
  • GPT-4o
  • InternLM 2.5 20B
  • Llama 3.3 70B Instruct
  • o1-mini
  • Qwen 2.5 72B Instruct
  • QwQ 32B

Variational Jailbreak Inference

Published Analyzed Source: arXiv

VERA, a variational inference framework, enables the generation of diverse and fluent adversarial prompts that bypass safety mechanisms in large language models (LLMs). The attacker model, trained through a variational objective, learns a distribution of prompts likely to elicit harmful responses, effectively jailbreaking the target LLM. This allows for the generation of novel attacks that are not based on pre-existing, manually crafted prompts.

VERA: Variational Inference Framework for Jailbreaking Large Language Models
Evaluated models: Baichuan 2 7B, Gemini Pro, GPT-3.5 Turbo+8 more
  • Baichuan 2 7B
  • Gemini Pro
  • GPT-3.5 Turbo
  • Llama 2 13B
  • Llama 2 13B Chat
  • Llama 2 7B Chat
  • Llama 3 8B
  • Mistral 7B
  • Orca 2 7B
  • Vicuna 7B
  • Zephyr 7B

Iterative Semantic Jailbreak

Published Analyzed Source: arXiv

The MIST attack exploits a vulnerability in black-box large language models (LLMs) allowing iterative semantic tuning of prompts to elicit harmful responses. The attack leverages synonym substitution and optimization strategies to bypass safety mechanisms without requiring access to the model's internal parameters or weights. The vulnerability lies in the susceptibility of the LLM to semantically similar prompts that trigger unsafe outputs.

MIST: Jailbreaking Black-box Large Language Models via Iterative Semantic Tuning
Evaluated models: Claude 3.5 Sonnet, GPT-4 Turbo, GPT-4o+3 more
  • Claude 3.5 Sonnet
  • GPT-4 Turbo
  • GPT-4o
  • GPT-4o Mini
  • Llama 2 7B Chat
  • Vicuna 7B v1.5

Doppelgänger Agent Hijack

Published Analyzed Source: arXiv

Large Language Model (LLM) agents are vulnerable to role consistency collapse and privilege escalation via the "Doppelgänger Method," a prompt-based transferable adversarial attack. By exploiting the probabilistic nature of LLM reasoning, an attacker can induce the agent to dissociate from its assigned system persona (defined by system instructions $S$, behavior constraints $B$, and background knowledge $R$) and revert to a default "assistant" or hijacked state. This vulnerability allows…

Doppelgänger Method: Breaking Role Consistency in LLM Agent via Prompt-based Transferable Adversarial Attack
Evaluated models: GPT-4, GPT-4.1, GPT-4.5 Preview+6 more
  • GPT-4
  • GPT-4.1
  • GPT-4.5 Preview
  • GPT-4o
  • o3-mini
  • Gemini 2.5 Flash
  • HCX-002
  • HCX-003
  • HCX-DASH-002

Research methodology

Entries summarize publicly available primary-source security research. Model names reflect only systems explicitly evaluated by the cited paper, and measurements are research-reported unless independent verification is stated.

Discovery and publication are separate: new papers enter a candidate queue, then require human review and explicit approval before publication. Updates follow that review process, not a guaranteed daily publishing schedule. “Last analyzed” is the newest analysis date among published entries; it does not mean that all recent research has been covered.