Language Model Security Database

A universal prompt injection vulnerability, termed "Involuntary Jailbreak," affects multiple large language models. The attack uses a single prompt that instructs the model to learn a pattern from abstract string operators (X and Y). The model is then asked to generate its own examples of questions that should be refused (harmful questions) and provide detailed, non-refusal answers to them, in order to satisfy the learned operator logic. This reframes the generation of harmful content as a…

Large Language Models (LLMs), including Llama 2, Mistral, and Vicuna, are susceptible to a white-box adversarial attack that circumvents safety alignment mechanisms (such as RLHF). The vulnerability exists due to the models' susceptibility to intrinsic optimization of adversarial suffixes using Exponentiated Gradient Descent (EGD). Unlike previous methods that rely on inefficient discrete token searches (e.g., Greedy Coordinate Gradient) or standard projected gradient descent, this attack…

A logic-based jailbreak vulnerability exists in Large Language Models (LLMs) known as "PUZZLED," where safety alignment mechanisms are bypassed by embedding harmful instructions within word-based puzzles. The attacker identifies sensitive keywords in a malicious prompt, masks them (e.g., replacing "bomb" with "[WORD1]"), and presents the masked terms as a cognitive task—specifically Word Searches, Anagrams, or Crosswords—accompanied by linguistic clues (word length, part-of-speech, and…

Large Language Models (LLMs) are vulnerable to a jailbreak attack termed Paper Summary Attack (PSA). An attacker can bypass safety alignment mechanisms by framing a harmful query within the context of a summarized academic paper. The model's propensity to trust the authoritative structure and tone of a research paper summary overrides its safety filters, leading it to process and respond to the embedded malicious instruction. The vulnerability is particularly potent when using summaries of…

LLM agents integrating with external environments (e.g., via tool use, web retrieval, or RAG) are vulnerable to indirect prompt injection attacks. Malicious instructions embedded in untrusted data sources—such as emails, webpages, or tool outputs—are ingested by the agent and treated as valid context. Because the backend Large Language Model (LLM) struggles to distinguish between system instructions, user instructions, and third-party data, these embedded prompts can hijack the execution flow…

LLM-powered agentic systems that use external tools are vulnerable to prompt injection attacks that cause them to bypass their explicit policy instructions. The vulnerability can be exploited through both direct user interaction and indirect injection, where malicious instructions are embedded in external data sources processed by the agent (e.g., documents, API responses, webpages). These attacks cause agents to perform prohibited actions, leak confidential data, and adopt unauthorized…

Retrieval-Augmented Generation (RAG) systems utilizing dense (e.g., BERT-based) or sparse (e.g., BM25) retrievers are vulnerable to black-box adversarial prompt injection attacks. By employing a gradient-free Differential Evolution (DE) optimization algorithm (referred to as DeRAG), an attacker can generate short adversarial suffixes (typically ≤ 5 tokens). When these suffixes are appended to a user query, they manipulate the retriever's ranking mechanism to promote a specific, malicious, or…

Aligned Large Language Models (LLMs) exhibit a "compositional blindness" vulnerability wherein safety alignment mechanisms evaluate user prompts in isolation, failing to detect malicious intent when it is systematically decomposed into multiple benign-appearing sub-tasks. An attacker can exploit this vulnerability using a framework such as the Malware Generation Compiler (MGC). The attack leverages a weakly aligned auxiliary model to decompose a high-level malicious objective (e.g…

Reasoning-capable Large Language Models (LLMs) are vulnerable to a class of indirect prompt injection known as Copy-Guided Attacks (CGA). This vulnerability exploits the intrinsic behavior of reasoning models to copy tokens from the input prompt (such as variable names, function identifiers, or code snippets) into their intermediate reasoning traces (Chain-of-Thought). By embedding adversarial trigger sequences into external payloads—specifically within data the model is expected to analyze—an…

A vulnerability exists in Diffusion-based Large Language Models (dLLMs) that allows for bypassing safety alignment mechanisms through interleaved mask-text prompts. The vulnerability stems from two core architectural features of dLLMs: bidirectional context modeling and parallel decoding. The model's drive to maintain contextual consistency forces it to fill masked tokens with content that aligns with the surrounding, potentially malicious, text. The parallel decoding process prevents dynamic…