Skip to main content
LLM Security Database
Skip to research search
Last analyzed Sep 9, 2026

Language Model Security Database

985 research findings · 1119 evaluated models

Filtered research findings

337 entries

Matches every word across titles, descriptions, sources, affected systems, and models.

LangChain Poisoning Jailbreak

Published Analyzed Source: arXiv

A vulnerability in Retrieval-Augmented Generation (RAG) systems utilizing LangChain allows for indirect jailbreaks of Large Language Models (LLMs). By poisoning the external knowledge base accessed by the LLM through LangChain, attackers can manipulate the LLM's responses, causing it to generate malicious or inappropriate content. The attack exploits the LLM's reliance on the external knowledge base and bypasses direct prompt-based jailbreak defenses.

Poisoned langchain: Jailbreak llms by langchain
Evaluated models: ChatGLM2 6B, ChatGLM3 6B, ERNIE 3.5+3 more
  • ChatGLM2 6B
  • ChatGLM3 6B
  • ERNIE 3.5
  • Llama 2 7B
  • Qwen 14B Chat
  • Xinghuo-3.5

Chat Template Jailbreak

Published Analyzed Source: arXiv

Large Language Models (LLMs) fine-tuned using chat templates are vulnerable to ChatBug, allowing malicious actors to bypass safety mechanisms by crafting prompts that intentionally deviate from the expected template format or overflow message fields. This exploits the LLM’s reliance on the template structure without enforcing similar constraints on user input.

ChatBug: A Common Vulnerability of Aligned LLMs Induced by Chat Templates
Evaluated models: Claude 2.1, GPT-3.5 Turbo

Code-Switching LLM Jailbreak

Published Analyzed Source: arXiv

Large Language Models (LLMs) exhibit increased vulnerability to adversarial prompts employing code-switching techniques, where multiple languages are interwoven within a single query. This vulnerability stems from an unintended correlation between the resource availability of the languages used in the prompt and the LLM's safety alignment. LLMs trained on imbalanced multilingual data are more susceptible to attacks leveraging low-resource languages, resulting in a higher rate of unsafe or…

Code-Switching Red-Teaming: LLM Evaluation for Safety and Multilingual Understanding
Evaluated models: Not reported

LLM Robot Bias & Violence

Published Analyzed Source: arXiv

Large Language Models (LLMs) used to control robots exhibit biases leading to discriminatory and unsafe behaviors. When provided with personal characteristics (e.g., race, gender, disability), LLMs generate biased outputs resulting in discriminatory actions (e.g., assigning lower rescue priority to certain groups) and accept or deem feasible dangerous or unlawful instructions (e.g., removing a person's mobility aid).

Llm-driven robots risk enacting discrimination, violence, and unlawful actions
Evaluated models: GPT-3.5, GPT-3.5 Turbo, GPT-4+1 more
  • GPT-3.5
  • GPT-3.5 Turbo
  • GPT-4
  • Mistral 7B

Transfer-Based LLM Jailbreak

Published Analyzed Source: arXiv

This vulnerability allows attackers to bypass safety mechanisms in Llama-2-7B-Chat and other safety-aligned LLMs using crafted adversarial prompts. The vulnerability stems from a gap between the gradient of the adversarial loss with respect to the one-hot representation of tokens and the actual effect of token replacements on the model's output. This gap allows for the generation of adversarial prompts that elicit harmful responses despite safety training. The paper demonstrates that…

Improved Generation of Adversarial Examples Against Safety-aligned LLMs
Evaluated models: GPT-3.5 Turbo, Llama 2 13B Chat, Llama 2 7B Chat+2 more
  • GPT-3.5 Turbo
  • Llama 2 13B Chat
  • Llama 2 7B Chat
  • Mistral 7B Instruct v0.2
  • Phi 3 Mini 4k Instruct

Automated T2I Jailbreak

Published Analyzed Source: arXiv

Commercial text-to-image (T2I) generative AI systems are vulnerable to automated prompt injection attacks that bypass their safety mechanisms and induce the generation of copyrighted material. An attacker can use an automated pipeline to generate prompts that precisely describe copyrighted images, effectively circumventing word-based detection filters.

Automatic Jailbreaking of the Text-to-Image Generative AI Systems
Evaluated models: DALL-E 3

Adversarial Speech Jailbreak

Published Analyzed Source: arXiv

Multimodal Large Language Models (LLMs) processing speech input are vulnerable to adversarial attacks. Imperceptible perturbations added to audio input can cause the model to generate unsafe or harmful text responses, overriding built-in safety mechanisms. The attacks are effective even with limited knowledge of the model's internal workings, demonstrating transferability across different models.

SpeechGuard: Exploring the adversarial robustness of multimodal large language models
Evaluated models: Flan-T5 XL, Llama 7B, Llama 2 13B Chat+2 more
  • Flan-T5 XL
  • Llama 7B
  • Llama 2 13B Chat
  • Mistral 7B Instruct
  • SpeechGPT

Multi-turn Semantic Jailbreak

Published Analyzed Source: arXiv

A vulnerability in large language models (LLMs) allows attackers to elicit unsafe or unethical responses through a chain of semantically relevant multi-turn prompts. The attack, termed "Chain of Attack" (CoA), exploits the model's contextual understanding and adaptive response capabilities to gradually steer the conversation towards the desired harmful output, even if single-turn prompts are rejected due to safety mechanisms. The attack leverages semantic similarity scoring (e.g., using…

Chain of attack: a semantic-driven contextual multi-turn attacker for llm
Evaluated models: Baichuan 2 7B Chat, ChatGLM2 6B, GPT-3.5 Turbo+2 more
  • Baichuan 2 7B Chat
  • ChatGLM2 6B
  • GPT-3.5 Turbo
  • Llama 2 7B Chat
  • Vicuna 13B v1.5

Momentum-Boosted LLM Jailbreak

Published Analyzed Source: arXiv

A momentum-accelerated gradient-based attack (MAC) against Large Language Models (LLMs) significantly improves the efficiency and success rate of jailbreak attacks. MAC leverages a momentum term within the gradient descent optimization process to enhance the stability and speed of generating adversarial prompts that bypass LLM safety measures. This allows adversaries to elicit harmful or undesirable outputs from the model more quickly than previous methods.

Boosting jailbreak attack with momentum
Evaluated models: Vicuna 7B

LLM Fake News Generation

Published Analyzed Source: arXiv

Large Language Models (LLMs) are vulnerable to a novel prompting technique, "conditional Variational-autoencoder-Like Prompt" (VLPrompt), which enables the generation of highly convincing fake news articles. VLPrompt overcomes limitations of previous methods by eliminating the need for additional human-collected data while maintaining contextual coherence and detail. This allows for the automated mass-production of realistic-sounding fake news.

Exploring the deceptive power of llm-generated fake news: A study of real-world detection challenges
Evaluated models: Not reported

Research methodology

Entries summarize publicly available primary-source security research. Model names reflect only systems explicitly evaluated by the cited paper, and measurements are research-reported unless independent verification is stated.

Discovery and publication are separate: new papers enter a candidate queue, then require human review and explicit approval before publication. Updates follow that review process, not a guaranteed daily publishing schedule. “Last analyzed” is the newest analysis date among published entries; it does not mean that all recent research has been covered.