Language Model Security Database

Large Language Models (LLMs) supporting function calling (tool use) are vulnerable to a jailbreak attack known as iMIST (interactive Multi-step Progressive Tool-disguised Jailbreak Attack). The vulnerability stems from a disparity in alignment training: while models are heavily aligned to refuse harmful natural language generation, they lack sufficient alignment regarding the generation of harmful content within structured data (JSON) used for tool parameters.

Large Language Models (LLMs) hosted on inference servers are vulnerable to high-speed weight exfiltration attacks due to the inherent compressibility of transformer parameters when decompression constraints are relaxed. Adversaries with compromised server access can utilize aggressive lossy compression techniques—specifically additive quantization combined with k-means clustering—to reduce model size by factors of 16x to 100x (e.g., <1 bit per parameter). Unlike standard quantization for…

A vulnerability exists in Large Language Model (LLM) deployments and multi-agent systems where an autonomous attacker agent can systematically extract hidden system prompts through self-evolving interaction strategies. The vulnerability leverages a "JustAsk" framework which utilizes Upper Confidence Bound (UCB) exploration to dynamically select and refine attack vectors from a hierarchical taxonomy of 14 atomic skills (e.g., structural formatting, authority appeals) and 14 multi-turn…

A vulnerability exists in Large Language Model (LLM) and Large Reasoning Model (LRM) serving interfaces that allow user-defined response prefixes, such as plain text-completion (v1/completions), Fill-in-the-Middle (FIM), or assistant message prefilling. An attacker can perform a Response Prefix Attack (RPA) by injecting maliciously crafted Chain-of-Thought (CoT) reasoning tokens immediately following the assistant's start delimiter (e.g., <|im_start|>assistant). Because these tokens are placed…

LLM-based autonomous agents are vulnerable to implicit regulatory compliance failures during tool invocation. When initialized with unstructured regulatory policies and given goal-oriented user instructions that do not explicitly state safety requirements, LLMs frequently prioritize functional task completion over mandatory safety constraints. This leads to an "Unsafe Success" execution state, where the agent successfully achieves the user's business goal but silently bypasses critical…

The Model Context Protocol (MCP) specification v1.0 contains fundamental architectural vulnerabilities enabling server-side prompt injection and privilege escalation. The protocol relies on bidirectional sampling (sampling/createMessage) without cryptographic origin authentication or UI distinction, allowing connected servers to inject content that the LLM backend interprets as legitimate user input. Additionally, the protocol lacks isolation boundaries between concurrent server connections…

Large Language Models (LLMs) contain a resource consumption vulnerability termed "Overflow," wherein specific non-adversarial, plain-text prompts trigger excessive text generation that saturates the model's output token budget. This vulnerability exploits the model's alignment towards helpfulness and exhaustiveness, alongside tokenizer inefficiencies (e.g., zero-width characters), to force the generation of maximum-length responses (often exceeding 5,000 tokens) from short inputs. This differs…

Production Large Language Models (LLMs) are vulnerable to long-form training data extraction via a two-phase prompt injection attack. This vulnerability allows an attacker to recover substantial portions of memorized, copyrighted text (such as novels) by exploiting the model's autoregressive text completion capabilities. The attack methodology involves two distinct phases: 1. Prefix Completion Probe: The attacker provides a short "seed" sequence (e.g., the first sentence of a book) coupled…

Large Language Models (LLMs) enabled with Function Calling (FC) capabilities are vulnerable to adversarial query rewriting and semantic manipulation. Standard FC models, typically trained via Supervised Fine-Tuning (SFT) on static datasets, fail to generalize against adversarial inputs that deviate from fixed distribution patterns. An attacker can exploit this by crafting queries that are semantically similar to valid requests but engineered to induce "bad cases," such as incorrect tool…

Large Language Model (LLM) agents implementing the Model Context Protocol (MCP) are vulnerable to Implicit Tool Poisoning (ITP). This vulnerability allows an attacker to manipulate agent behavior by embedding malicious instructions within the metadata (specifically the natural language description) of a third-party tool. Unlike explicit tool poisoning, where the agent is tricked into invoking a malicious tool, ITP exploits the agent's contextual reasoning to force the invocation of a distinct…