Language Model Security Database

A vulnerability exists in several large language models (LLMs) allowing attackers to manipulate the models' output logits, biasing the probability distribution toward the generation of harmful content. The attack does not involve modifying the input prompt, but rather directly manipulates the internal probability scores assigned to output tokens during the generation process. By strategically increasing the logits of tokens forming a harmful response while decreasing those belonging to safety…

Medical Multimodal Large Language Models (MedMLLMs) are vulnerable to cross-modality attacks. Attackers can craft "mismatched malicious attacks" (2M-attacks) by providing MedMLLMs with image-text pairs where the image modality and/or anatomical region do not match the textual query, causing the model to generate incorrect or harmful responses. These attacks can be further optimized ("optimized mismatched malicious attacks"—O2M-attacks) using multimodal cross-optimization (MCM) techniques to…

A momentum-accelerated gradient-based attack (MAC) against Large Language Models (LLMs) significantly improves the efficiency and success rate of jailbreak attacks. MAC leverages a momentum term within the gradient descent optimization process to enhance the stability and speed of generating adversarial prompts that bypass LLM safety measures. This allows adversaries to elicit harmful or undesirable outputs from the model more quickly than previous methods.

A vulnerability in large language models (LLMs) allows attackers to elicit unsafe or unethical responses through a chain of semantically relevant multi-turn prompts. The attack, termed "Chain of Attack" (CoA), exploits the model's contextual understanding and adaptive response capabilities to gradually steer the conversation towards the desired harmful output, even if single-turn prompts are rejected due to safety mechanisms. The attack leverages semantic similarity scoring (e.g., using…

A vulnerability in large language models (LLMs) allows for near-perfect jailbreaking via iterative prompt refinement and self-explanation. The attacker uses the LLM itself to iteratively refine adversarial prompts by requesting self-explanations of failed attempts, ultimately generating prompts that bypass safety mechanisms and elicit harmful content. A subsequent "Rate+Enhance" step further maximizes the harmfulness of the generated output.

Large language models (LLMs) are vulnerable to enhanced jailbreak attacks by appending multiple end-of-sentence (EOS) tokens to malicious prompts. This bypasses internal safety mechanisms, causing the LLM to respond to harmful queries that it would otherwise reject. The EOS tokens subtly shift the LLM’s internal representation of the prompt, making it appear less harmful without significantly altering the semantic meaning of the malicious content.

This vulnerability allows attackers to bypass safety mechanisms in Llama-2-7B-Chat and other safety-aligned LLMs using crafted adversarial prompts. The vulnerability stems from a gap between the gradient of the adversarial loss with respect to the one-hot representation of tokens and the actual effect of token replacements on the model's output. This gap allows for the generation of adversarial prompts that elicit harmful responses despite safety training. The paper demonstrates that…

A vulnerability in multimodal large language models (MLLMs) allows for efficient jailbreaking attacks by leveraging visual input to bypass safety mechanisms. The attack constructs a multimodal model by adding a visual module to the target LLM, then uses a modified PGD algorithm to optimize visual input to generate jailbreaking embeddings. These embeddings are then converted back into text and appended to harmful queries, successfully eliciting objectionable content from the target LLM.

Multimodal Large Language Models (MLLMs) are vulnerable to a universal jailbreak attack, termed Visual Role-Play (VRP), which leverages role-playing image characters to elicit harmful responses. VRP generates images depicting high-risk characters (e.g., cybercriminals) described by an LLM, paired with a benign role-play instruction and a malicious query. This combined input tricks the MLLM into generating malicious content by enacting the character's persona.

A vulnerability in the voice mode of GPT-4o allows bypassing safety restrictions through a novel "Voice Jailbreak" attack. This attack leverages principles of fictional storytelling (setting, character, plot) to craft audio prompts that persuade the LLM to generate responses violating OpenAI's usage policies, including generating content related to illegal activities, hate speech, physical harm, fraud, pornography, and privacy violations. The attack's success rate is significantly higher than…