Language Model Security Database

Large Language Models (LLMs) with user-controlled response prefilling features are vulnerable to a novel jailbreak attack. By manipulating the prefilled text, attackers can influence the model's subsequent token generation, bypassing safety mechanisms and eliciting harmful or unintended outputs. Two attack vectors are demonstrated: Static Prefilling (SP), using a fixed prefill string, and Optimized Prefilling (OP), iteratively optimizing the prefill string for maximum impact. The vulnerability…

Large Language Models (LLMs) employing alignment safeguards and safety mechanisms are vulnerable to graph-based adversarial attacks that bypass these protections. The attack, termed "Graph of Attacks" (GOAT), leverages a graph-based reasoning framework to iteratively refine prompts and exploit vulnerabilities more effectively than previous methods. The attack synthesizes information across multiple reasoning paths to generate human-interpretable prompts that elicit undesired or harmful outputs…

Improper Input Validation in Large Language Model (LLM) systems configured as automated evaluators ("LLM-as-a-judge") allows remote attackers to manipulate evaluation scores and comparative verdicts via adversarial prompt injection. The vulnerability arises when the model processes untrusted input containing linguistic masquerading, context separators, and disruptor commands (e.g., "Basic Injection", "Contextual Misdirection", and "Adaptive Search-Based Attack"). Successful exploitation…

Large Language Models (LLMs) employing safety mechanisms based on supervised fine-tuning and preference alignment exhibit a vulnerability to "steering" attacks. Maliciously crafted prompts or input manipulations can exploit representation vectors within the model to either bypass censorship ("refusal-compliance vector") or suppress the model's reasoning process ("thought suppression vector"), resulting in the generation of unintended or harmful outputs. This vulnerability is demonstrated…

Multi-Agent Debate (MAD) frameworks leveraging Large Language Models (LLMs) are vulnerable to amplified jailbreak attacks. A novel structured prompt-rewriting technique exploits the iterative dialogue and role-playing dynamics of MAD, circumventing inherent safety mechanisms and significantly increasing the likelihood of generating harmful content. The attack succeeds by using narrative encapsulation, role-driven escalation, iterative refinement, and rhetorical obfuscation to guide agents…

A vulnerability exists in the combination of Large Language Models (LLMs) and their associated safety guardrails, allowing attackers to bypass both defenses and elicit harmful or unintended outputs from LLMs. The vulnerability stems from insufficient detection by guardrails against adversarially crafted prompts, which appear benign but contain hidden malicious intent. The attack, dubbed "DualBreach," leverages a target-driven initialization strategy and multi-target optimization to generate…

Large Language Model (LLM) agents operating in stateful environments (web browsers, operating systems, and tool-use contexts) are vulnerable to indirect prompt injection and multi-modal adversarial attacks. These vulnerabilities arise when agents process untrusted environmental observations—such as web accessibility trees, screen screenshots, or database query results—that contain concealed malicious instructions. Specifically, attackers can embed prompt injections into HTML accessibility…

Large Language Models (LLMs) employing safety mechanisms are vulnerable to a graph-based attack that leverages semantic transformations of malicious prompts to bypass safety filters. The attack, termed GraphAttack, uses Abstract Meaning Representation (AMR), Resource Description Framework (RDF), and JSON knowledge graphs to represent malicious intent, systematically applying transformations to evade surface-level pattern recognition used by existing safety mechanisms. A particularly effective…

LLM-powered GUI agents utilizing screenshot-based interpretation (such as those powered by GPT-4o, Claude 3.7 Sonnet, Gemini 2.0 Flash, and DeepSeek V3 0324) are vulnerable to Fine-Print Injection (FPI) and Deceptive Default (DD) attacks due to a lack of visual saliency filtering. Unlike human users who prioritize prominent UI elements, these agents perform "indiscriminate parsing," processing low-salience text (e.g., privacy policies, terms of service, footer disclaimers) with the same…

Large Language Model (LLM) guardrail systems, including those relying on AI-driven text classification models (e.g., fine-tuned BERT models), are vulnerable to evasion via character injection and adversarial machine learning (AML) techniques. Attackers can bypass detection by injecting Unicode characters (e.g., zero-width characters, homoglyphs) or using AML to subtly perturb prompts, maintaining semantic meaning while evading classification. This allows malicious prompts and jailbreaks to…