Language Model Security Database

Large Language Models (LLMs) are vulnerable to a targeted jailbreak attack, termed Atoxia, which can force the generation of specific harmful content. The attack operates by providing a target toxic answer to an attacker model, which then generates a corresponding adversarial query and a misleading "answer opening" (prefix). When the query and the answer prefix are presented to a vulnerable LLM, the model is induced to continue the generation, bypassing its safety alignment and completing the…

A Cross-Prompt Injection Attack (XPIA) can be amplified by appending a Greedy Coordinate Gradient (GCG) suffix to the malicious injection. This increases the likelihood that a Large Language Model (LLM) will execute the injected instruction, even in the presence of a user's primary instruction, leading to data exfiltration. The success rate of the attack depends on the LLM's complexity; medium-complexity models show increased vulnerability.

The ALERT-Motion framework demonstrates a vulnerability in text-to-motion (T2M) models where an attacker can craft subtly modified text prompts (adversarial prompts) that cause the model to generate motions significantly different from those intended by the benign prompt, yet semantically similar to a target motion specified by the attacker. The attack leverages a large language model (LLM) to autonomously generate these adversarial prompts, bypassing simple keyword-based detection mechanisms…

A vulnerability in large language models (LLMs) allows attackers to generate harmful content by manipulating the continuous input embeddings without appending suffixes or using specific questions. The attack leverages gradient descent to optimize the input vector, causing the model to produce a predefined malicious output. Mitigation strategies, such as input clipping, help reduce the effectiveness but do not fully eliminate the threat.

Embodied Large Language Models (LLMs) are vulnerable to manipulation via voice-based interactions, leading to the execution of harmful physical actions. Attacks exploit three vulnerabilities: (1) cascading LLM jailbreaks resulting in malicious robotic commands; (2) misalignment between linguistic outputs (verbal refusal) and physical actions (command execution); and (3) conceptual deception, where seemingly benign instructions lead to harmful outcomes due to incomplete world knowledge within…

Large language models (LLMs) are vulnerable to "editing attacks," where malicious actors manipulate the model's knowledge base to inject misinformation or bias. This is achieved by using existing knowledge editing techniques to subtly alter the model's internal representations, causing it to generate outputs reflecting the injected content, even on seemingly unrelated prompts. The attack can be remarkably stealthy, with minimal impact on the model's overall performance in other areas.

Large Language Models (LLMs) are vulnerable to adversarial attacks that utilize low-perplexity prompts to elicit unsafe content. These prompts, while statistically likely to occur in normal conversation, can trigger the generation of harmful or toxic outputs that evade standard safety filters. The vulnerability stems from the model's inability to reliably distinguish between benign and malicious intents within the statistical distribution of natural language.

Large Language Models (LLMs) used for code generation are vulnerable to Malicious Programming Prompts (MaPP), where an attacker injects a short string (under 500 bytes) into the prompt, causing the LLM to generate code containing vulnerabilities while maintaining functional correctness. The attack exploits the LLM's ability to follow instructions, even those inserted maliciously, to embed unintended behaviors. The injected code can range from general vulnerabilities (e.g., setting a…

A training-time attack against open-source LLMs that injects adversarial embeddings into the model's token embeddings without modifying model weights. This allows an attacker to introduce backdoors, jailbreaks, or prompt stealing capabilities by simply modifying specific token embeddings within the model file, maintaining model utility for non-triggered inputs. The attack leverages soft prompt tuning to optimize adversarial embeddings, which are then assigned to chosen trigger tokens.

Appending a single whitespace character (space) or certain punctuation marks to the end of an LLM's input template can bypass safety mechanisms and cause the model to generate unsafe, biased, or factually incorrect outputs, even if the original prompt was benign. This vulnerability is due to the statistical properties of single-character tokens in the model's training data, causing unintended behavior in the model's token prediction.