Language Model Security Database

Large Language Models (LLMs) integrated into applications reveal unique behavioral fingerprints through responses to crafted queries. LLMmap exploits this by sending carefully constructed prompts and analyzing the responses to identify the specific LLM version with high accuracy (over 95% in testing against 42 LLMs). This allows attackers to tailor attacks exploiting known vulnerabilities specific to the identified LLM version.

Large Language Models (LLMs) used for code generation are vulnerable to Malicious Programming Prompts (MaPP), where an attacker injects a short string (under 500 bytes) into the prompt, causing the LLM to generate code containing vulnerabilities while maintaining functional correctness. The attack exploits the LLM's ability to follow instructions, even those inserted maliciously, to embed unintended behaviors. The injected code can range from general vulnerabilities (e.g., setting a…

A training-time attack against open-source LLMs that injects adversarial embeddings into the model's token embeddings without modifying model weights. This allows an attacker to introduce backdoors, jailbreaks, or prompt stealing capabilities by simply modifying specific token embeddings within the model file, maintaining model utility for non-triggered inputs. The attack leverages soft prompt tuning to optimize adversarial embeddings, which are then assigned to chosen trigger tokens.

Large language models (LLMs) employing safety measures like filters and alignment training remain vulnerable to information leakage via "Decomposition Attacks". These attacks decompose a malicious query into multiple benign sub-queries, eliciting responses from the LLM that, when aggregated, reveal sensitive information without triggering safety filters or producing directly harmful outputs.

A vulnerability in LLM finetuning APIs allows covert malicious finetuning. Attackers can create a dataset where individual data points appear innocuous but, when used for finetuning, teach the LLM to respond to encoded harmful requests with encoded harmful responses. This bypasses existing safety checks and evaluations because the training data appears benign.

Large Language Models (LLMs) are vulnerable to prompt injection attacks that can bypass their internal copyright compliance mechanisms, causing them to generate verbatim copyrighted text. The vulnerability stems from insufficient robustness against prompt engineering techniques that manipulate the model into ignoring or circumventing its safety filters designed for copyright protection.

Large Language Models (LLMs) used to control robots exhibit biases leading to discriminatory and unsafe behaviors. When provided with personal characteristics (e.g., race, gender, disability), LLMs generate biased outputs resulting in discriminatory actions (e.g., assigning lower rescue priority to certain groups) and accept or deem feasible dangerous or unlawful instructions (e.g., removing a person's mobility aid).

LLMs, even when individually assessed as "safe," can be combined by an adversary to achieve malicious outcomes. This vulnerability exploits the complementary strengths of multiple models—a high-capability model that refuses malicious requests and a low-capability model that does not—through task decomposition. Adversaries can either manually decompose tasks into benign (solved by the high-capability model) and easily-malicious subtasks (solved by the low-capability model) or automate the…

Multimodal Large Language Models (LLMs) processing speech input are vulnerable to adversarial attacks. Imperceptible perturbations added to audio input can cause the model to generate unsafe or harmful text responses, overriding built-in safety mechanisms. The attacks are effective even with limited knowledge of the model's internal workings, demonstrating transferability across different models.

Commercial text-to-image (T2I) generative AI systems are vulnerable to automated prompt injection attacks that bypass their safety mechanisms and induce the generation of copyrighted material. An attacker can use an automated pipeline to generate prompts that precisely describe copyrighted images, effectively circumventing word-based detection filters.