Language Model Security Database

Large Language Models (LLMs) are susceptible to automated jailbreak attacks using a fuzzing framework that generates variations of existing jailbreak prompts. This vulnerability allows bypassing built-in safety mechanisms, leading to the generation of harmful or unintended outputs. The vulnerability stems from the LLMs' inability to consistently recognize and reject semantically similar, but subtly different prompt variations generated through automated mutation techniques.

A universal black-box jailbreaking vulnerability exists in Large Language Models (LLMs) due to their susceptibility to adversarial prompts crafted using a genetic algorithm (GA). The GA optimizes a universal adversarial prompt suffix that, when appended to various user inputs, causes the LLM to generate unintended and potentially harmful outputs, bypassing safety mechanisms. This attack requires no knowledge of the LLM's internal architecture or parameters.

Large Language Models (LLMs) are vulnerable to a "Chain of Utterances" (CoU) based prompt injection attack. This attack exploits the LLM's ability to engage in multi-turn conversations and role-playing, tricking it into providing harmful or unsafe responses even when presented with safety guidelines. The attack leverages a crafted conversation between two agents ("Red-LM," a malicious agent, and "Base-LM," a seemingly helpful agent) to elicit unethical responses from the Base-LM by subtly…

Large Language Models (LLMs) such as GPT-4, while employing safety alignment techniques, exhibit vulnerability to "CipherChat" attacks. CipherChat leverages cipher prompts (e.g., ASCII, Unicode, Caesar cipher, Morse code) combined with system role descriptions and few-shot enciphered demonstrations to bypass safety mechanisms trained on natural language. This allows an attacker to elicit unsafe responses from the LLM, effectively evading safety filters. The vulnerability is amplified by the…

Large language models (LLMs) are vulnerable to a "self-deception" attack, where carefully crafted prompts induce the model to bypass its internal safety mechanisms and generate outputs that would normally be blocked (e.g., harmful, biased, or illegal content). This occurs by exploiting inconsistencies in the model's internal reasoning processes, making it generate outputs that contradict its own safety policies. The attack does not involve direct code injection or data poisoning but rather…

The MASTER KEY framework exploits timing-based characteristics of Large Language Model (LLM) chatbot responses to infer internal defense mechanisms and automatically generate jailbreak prompts. This allows bypassing safety restrictions and eliciting responses violating usage policies, including generation of illegal, harmful, privacy-violating, and adult content. The framework utilizes a three-step process: reverse-engineering defenses via time-based analysis, creating proof-of-concept…

A vulnerability in multi-modal large language models (LLMs) allows adversaries to bypass safety mechanisms through compositional adversarial attacks. The attack leverages the alignment between vision and language encoders, injecting malicious triggers into benign-looking images. These images, when paired with innocuous prompts, cause the LLM to generate harmful content. The attack requires access only to the vision encoder (e.g., CLIP), not the LLM itself, lowering the barrier to attack.

Aligned large language models (LLMs) are vulnerable to a universal and transferable adversarial suffix attack. Appending a specific, automatically generated suffix to a wide range of prompts, even those requesting objectionable content, causes the models to generate harmful or objectionable responses instead of refusing the request. The attack's success rate is significantly higher on GPT-based models.

A vulnerability in vision-integrated Large Language Models (VLMs) allows an attacker to circumvent safety mechanisms through the use of adversarially crafted visual examples. A single, carefully constructed image can universally "jailbreak" the model, causing it to generate harmful content in response to a wide range of subsequent prompts, even those not included in the adversarial example's training data. This vulnerability extends beyond simple misclassification to encompass the execution of…

Large Language Models (LLMs), specifically ChatGPT versions 3.5 and 4.0, are vulnerable to prompt engineering attacks that circumvent built-in content restrictions. Attackers can craft malicious prompts, categorized into "pretending," "attention shifting," and "privilege escalation" techniques, to elicit responses containing prohibited content (e.g., instructions for illegal activities, generation of harmful content). The vulnerability stems from the LLM's inability to reliably distinguish…