Language Model Security Database

Large Language Models (LLMs) are vulnerable to optimization-based jailbreaking attacks that exploit gradients during the iterative process of generating adversarial suffixes. The vulnerability stems from the inefficient exploration of the token space in existing methods like Greedy Coordinate Gradient (GCG), which uniformly samples tokens for replacement regardless of gradient values. This leads to redundant computations and a slow optimization process.

Large Language Models (LLMs) are vulnerable to attacks that generate obfuscated activations, bypassing latent-space defenses such as sparse autoencoders, representation probing, and latent out-of-distribution (OOD) detection. Attackers can manipulate model inputs or training data to produce outputs exhibiting malicious behavior while remaining undetected by these defenses. This occurs because the models can represent harmful behavior through diverse activation patterns, allowing attackers to…

A vulnerability exists in large language models (LLMs) where targeted bitwise corruptions in model parameters can induce a "jailbroken" state, causing the model to generate harmful responses without input modification. Fewer than 25 bit-flips are sufficient to achieve this in many cases. The vulnerability stems from the susceptibility of the model's memory representation to fault injection attacks.

A white-box attack, Targeted Model Editing (TME), allows bypassing safety filters in large language models (LLMs) by minimally altering internal model structures, specifically the MLP layers, without modifying inputs. The attack identifies and removes safety-critical transformations (SCTs) in model matrices, enabling the LLM to respond to malicious queries with harmful outputs.

A vulnerability exists in Federated Parameter-Efficient Fine-Tuning (FedPEFT) systems for large language models (LLMs). Malicious clients can exploit the PEFT mechanism (e.g., LoRA, (IA)³, LayerNorm) to inject adversarial training data, compromising the model's safety alignment even with a small percentage of trainable parameters and a minority of malicious participants. The attack, termed "PEFT-as-an-Attack" (PaaA), circumvents the LLM's safety guardrails, causing it to generate harmful…

Large Language Models (LLMs) are vulnerable to a novel prompt injection attack using universal and context-independent triggers. These triggers, once discovered for a specific LLM, allow precise control over the model's output regardless of the prompt context or desired output content, enabling adversaries to force the generation of arbitrary text. The attack utilizes a gradient-based optimization technique to discover these triggers.

Large Language Models (LLMs) are vulnerable to jailbreaking attacks that manipulate attention scores to redirect the model's focus away from safety protocols. The AttnGCG attack method increases the attention score on adversarial suffixes within the input prompt, causing the model to prioritize the malicious content over safety guidelines, leading to the generation of harmful outputs.

A novel jailbreak attack, dubbed SI-GCG, against Large Language Models (LLMs) leverages a fixed harmful template and optimized suffix selection to bypass safety mechanisms and elicit harmful responses with high transferability. The attack utilizes a scenario induction template and a refined optimization process to improve the consistency and effectiveness of the jailbreak across different LLMs. The vulnerability stems from the inability of current safety measures to adequately defend against…

Faster-GCG is an optimized jailbreak attack that exploits vulnerabilities in aligned Large Language Models (LLMs) by efficiently finding adversarial prompt suffixes. The attack leverages gradient information to iteratively refine a harmful prompt, overcoming limitations of prior methods like GCG by incorporating a regularization term to improve gradient approximation, using deterministic greedy sampling, and preventing self-looping during optimization. This allows for significantly higher…

Multimodal fusion models, such as Chameleon models, utilize non-differentiable tokenization functions for image inputs, hindering direct gradient-based attacks. This vulnerability allows attackers with white-box access to bypass safety mechanisms by using a "tokenizer shortcut," a differentiable approximation of the tokenization process, to perform continuous optimization of image inputs. This enables the generation of adversarial images that elicit harmful responses from the model, even for…