Language Model Security Database

Large language models that support a developer role in their API are vulnerable to a jailbreaking attack that leverages malicious developer messages. An attacker can craft a developer message that overrides the model's safety alignment by setting a permissive persona, providing explicit instructions to bypass refusals, and using few-shot examples of harmful query-response pairs. This technique, named D-Attack, is effective on its own. A more advanced variant, DH-CoT, enhances the attack by…

IntentionReasoner, specifically the 1.5B and 3B parameter versions optimized via Reinforcement Learning (RL), contains a safety regression vulnerability where the RL alignment process degrades the model's resistance to jailbreak attacks compared to the Supervised Fine-Tuning (SFT) baseline. While RL improves general utility and rewriting quality, it inadvertently increases the Attack Success Rate (ASR) for adversarial inputs in smaller architectures. This allows sophisticated jailbreak prompts…

Large Language Models (LLMs) are vulnerable to an adaptive black-box jailbreaking framework named MAJIC (Markovian Adaptive Jailbreaking via Iterative Composition). This vulnerability allows an attacker to bypass safety alignment mechanisms by modeling the selection and composition of prompt disguise strategies as a Markov chain. Unlike static attacks, MAJIC initializes a transition matrix using a proxy model to determine the probability of a specific strategy succeeding after a prior strategy…

Multimodal Large Language Models (MLLMs) employed in autonomous driving (AD) systems are vulnerable to a physically realizable adversarial patch attack dubbed "PhysPatch." This vulnerability exists because MLLMs inherit susceptibility to visual adversarial perturbations from their vision backbones. The attack utilizes a semantic-aware mask initialization strategy combined with a potential field algorithm to identify physically plausible regions for patch placement within a driving scene (e.g…

Large Language Models (LLMs) implementing "Thinking Mode" (also known as Reasoning Mode or Chain-of-Thought) exhibit a heightened susceptibility to jailbreak attacks compared to their non-reasoning counterparts. When a model is prompted to reason step-by-step (often delimited by specific tokens like <think> and </think>), the internal reasoning process frequently overrides safety alignment training. Research indicates that during the generation of the thinking chain, the model often…

A universal prompt injection vulnerability, termed "Involuntary Jailbreak," affects multiple large language models. The attack uses a single prompt that instructs the model to learn a pattern from abstract string operators (X and Y). The model is then asked to generate its own examples of questions that should be refused (harmful questions) and provide detailed, non-refusal answers to them, in order to satisfy the learned operator logic. This reframes the generation of harmful content as a…

A logic-based jailbreak vulnerability exists in Large Language Models (LLMs) known as "PUZZLED," where safety alignment mechanisms are bypassed by embedding harmful instructions within word-based puzzles. The attacker identifies sensitive keywords in a malicious prompt, masks them (e.g., replacing "bomb" with "[WORD1]"), and presents the masked terms as a cognitive task—specifically Word Searches, Anagrams, or Crosswords—accompanied by linguistic clues (word length, part-of-speech, and…

Large Language Models (LLMs) are vulnerable to a jailbreak attack termed Paper Summary Attack (PSA). An attacker can bypass safety alignment mechanisms by framing a harmful query within the context of a summarized academic paper. The model's propensity to trust the authoritative structure and tone of a research paper summary overrides its safety filters, leading it to process and respond to the embedded malicious instruction. The vulnerability is particularly potent when using summaries of…

LLM agents integrating with external environments (e.g., via tool use, web retrieval, or RAG) are vulnerable to indirect prompt injection attacks. Malicious instructions embedded in untrusted data sources—such as emails, webpages, or tool outputs—are ingested by the agent and treated as valid context. Because the backend Large Language Model (LLM) struggles to distinguish between system instructions, user instructions, and third-party data, these embedded prompts can hijack the execution flow…

LLM-powered agentic systems that use external tools are vulnerable to prompt injection attacks that cause them to bypass their explicit policy instructions. The vulnerability can be exploited through both direct user interaction and indirect injection, where malicious instructions are embedded in external data sources processed by the agent (e.g., documents, API responses, webpages). These attacks cause agents to perform prohibited actions, leak confidential data, and adopt unauthorized…