Skip to main content
LLM Security Database
Skip to research search
Last analyzed Sep 9, 2026

Language Model Security Database

985 research findings · 1119 evaluated models

Filtered research findings

704 entries

Matches every word across titles, descriptions, sources, affected systems, and models.

Agent Action Hijacking

Published Analyzed Source: arXiv

A vulnerability in LLM-based agents, dubbed AI Agent Injection (AI²), allows attackers to hijack the agent's actions by manipulating the agent's memory retrieval mechanism. The attack involves two main steps: (1) Stealing action-aware knowledge from the agent's memory using crafted adversarial queries targeting the retriever module and (2) Generating Trojan prompts consisting of a Trojan string and hijacking instructions. The Trojan string is designed to manipulate the retriever into…

Towards Action Hijacking of Large Language Model-based Agent
Evaluated models: BERT, GPT-3, GPT-4+2 more
  • BERT
  • GPT-3
  • GPT-4
  • M3E
  • MiniLM

AdvPrefix Jailbreak Enhancement

Published Analyzed Source: arXiv

Large Language Models (LLMs) employing prefix-forcing safety measures are vulnerable to jailbreak attacks if the set of "safe" prefixes is insufficiently diverse or does not account for model-specific response styles. Attackers can leverage this by crafting prompts that elicit alternative prefixes, effectively bypassing the intended safety mechanisms. The vulnerability stems from over-reliance on a limited set of prefixes (e.g., "Sure, here is...") and a failure to generalize safety mechanisms…

AdvPrefix: An Objective for Nuanced LLM Jailbreaks
Evaluated models: Not reported

Obfuscated Activations Jailbreak

Published Analyzed Source: arXiv

Large Language Models (LLMs) are vulnerable to attacks that generate obfuscated activations, bypassing latent-space defenses such as sparse autoencoders, representation probing, and latent out-of-distribution (OOD) detection. Attackers can manipulate model inputs or training data to produce outputs exhibiting malicious behavior while remaining undetected by these defenses. This occurs because the models can represent harmful behavior through diverse activation patterns, allowing attackers to…

Obfuscated Activations Bypass LLM Latent-Space Defenses
Evaluated models: Gemma 2 2B, Llama 3 8B Instruct

Audio Adversarial Jailbreak

Published Analyzed Source: arXiv
Research ID e50b7819

Large Audio-Language Models (LALMs) are vulnerable to a stealthy adversarial jailbreak attack, AdvWave, which leverages a dual-phase optimization to overcome gradient shattering caused by audio discretization. The attack crafts adversarial audio by adding perceptually realistic environmental noise, making it difficult to detect. The attack also dynamically adapts the adversarial target based on the LALM's response patterns.

AdvWave: Stealthy Adversarial Jailbreak Attack against Large Audio-Language Models
Evaluated models: GPT-4o, Llama Omni, Qwen 2 Audio+1 more
  • GPT-4o
  • Llama Omni
  • Qwen 2 Audio
  • SpeechGPT

Index-Gradient LLM Jailbreak

Published Analyzed Source: arXiv

Large Language Models (LLMs) are vulnerable to optimization-based jailbreaking attacks that exploit gradients during the iterative process of generating adversarial suffixes. The vulnerability stems from the inefficient exploration of the token space in existing methods like Greedy Coordinate Gradient (GCG), which uniformly samples tokens for replacement regardless of gradient values. This leads to redundant computations and a slow optimization process.

Exploiting the Index Gradients for Optimization-Based Jailbreaking on Large Language Models
Evaluated models: Vicuna 7B, Guanaco 7B, Llama 2 7B Chat+5 more
  • Vicuna 7B
  • Guanaco 7B
  • Llama 2 7B Chat
  • Mistral 7B Instruct v0.2
  • GPT-3.5 Turbo
  • GPT-4
  • GPT-4o
  • Claude 3 Haiku

Semantic Confusion Jailbreak

Published Analyzed Source: arXiv

The Antelope attack exploits vulnerabilities in Text-to-Image (T2I) models' safety filters by crafting adversarial prompts. These prompts, while appearing benign, induce the generation of NSFW images by leveraging semantic similarity between harmless and harmful concepts. The attack involves replacing explicit terms in an original prompt with seemingly innocuous alternatives and appending carefully selected suffix tokens. This manipulation bypasses both text-based and image-based filters…

Antelope: Potent and Concealed Jailbreak Attack Strategy
Evaluated models: GPT-4o, Midjourney, Stable Diffusion+2 more
  • GPT-4o
  • Midjourney
  • Stable Diffusion
  • Stable Diffusion v1.4
  • Stable Diffusion v2.1

Targeted Model Editing Jailbreak

Published Analyzed Source: arXiv

A white-box attack, Targeted Model Editing (TME), allows bypassing safety filters in large language models (LLMs) by minimally altering internal model structures, specifically the MLP layers, without modifying inputs. The attack identifies and removes safety-critical transformations (SCTs) in model matrices, enabling the LLM to respond to malicious queries with harmful outputs.

Model-Editing-Based Jailbreak against Safety-aligned Large Language Models
Evaluated models: Gemma 2 9B IT, Llama 2 7B Chat, Llama 3 8B Instruct+1 more
  • Gemma 2 9B IT
  • Llama 2 7B Chat
  • Llama 3 8B Instruct
  • Mistral 7B Instruct

Metaphorical LLM Jailbreak

Published Analyzed Source: arXiv

Large Language Models (LLMs) are vulnerable to jailbreaking attacks via adversarial metaphors. Attackers can leverage the LLMs' imaginative capabilities to map harmful concepts to innocuous ones, thereby bypassing safety mechanisms and eliciting harmful responses. The attack relies on creating a metaphorical mapping between a harmful target and seemingly benign entities, exploiting the LLM's ability to reason about the analogous relationship without recognizing the underlying malicious intent.

Na'vi or Knave: Jailbreaking Language Models via Metaphorical Avatars
Evaluated models: Claude 3.5 Sonnet, Gemini 1.5 Pro, GLM 3 6B+13 more
  • Claude 3.5 Sonnet
  • Gemini 1.5 Pro
  • GLM 3 6B
  • GLM 4 9B
  • GPT-3.5 Turbo
  • GPT-4
  • InternLM 2.5 7B
  • Llama 3.1 70B
  • Llama 3.1 8B
  • Mistral 7B
  • Mixtral 8x7B
  • o1
  • Qwen 1.5 110B
  • Qwen 2 72B
  • Qwen 2 7B
  • Yi 1.5 34B

Targeted Bit-Flip Jailbreak

Published Analyzed Source: arXiv

A vulnerability exists in large language models (LLMs) where targeted bitwise corruptions in model parameters can induce a "jailbroken" state, causing the model to generate harmful responses without input modification. Fewer than 25 bit-flips are sufficient to achieve this in many cases. The vulnerability stems from the susceptibility of the model's memory representation to fault injection attacks.

PrisonBreak: Jailbreaking Large Language Models with Fewer Than Twenty-Five Targeted Bit-flips
Evaluated models: Llama 2 13B, Llama 2 7B, Llama 3 8B+4 more
  • Llama 2 13B
  • Llama 2 7B
  • Llama 3 8B
  • Qwen 2 1.5B
  • Qwen 2 7B
  • Vicuna 13B
  • Vicuna 7B

Bimodal Black-Box Jailbreak

Published Analyzed Source: arXiv

A bimodal adversarial attack, PBI-Attack, can manipulate Large Vision-Language Models (LVLMs) into generating toxic or harmful content by iteratively optimizing both textual and visual inputs in a black-box setting. The attack leverages a surrogate LVLM to inject malicious features from a harmful corpus into a benign image, then iteratively refines both image and text perturbations to maximize the toxicity of the model’s output as measured by a toxicity detection model (Perspective API or…

BAMBA: A Bimodal Adversarial Multi-Round Black-Box Jailbreak Attacker for LVLMs
Evaluated models: GPT-4, InstructBLIP, MiniGPT-4+1 more
  • GPT-4
  • InstructBLIP
  • MiniGPT-4
  • Qwen VL

Research methodology

Entries summarize publicly available primary-source security research. Model names reflect only systems explicitly evaluated by the cited paper, and measurements are research-reported unless independent verification is stated.

Discovery and publication are separate: new papers enter a candidate queue, then require human review and explicit approval before publication. Updates follow that review process, not a guaranteed daily publishing schedule. “Last analyzed” is the newest analysis date among published entries; it does not mean that all recent research has been covered.