Language Model Security Database

A vulnerability exists in multimodal Large Language Models (LLMs) integrated with external tools. Adversarial images, visually indistinguishable from benign images, can manipulate the LLM to execute unintended tool commands, compromising the confidentiality and integrity of user resources. The attack is effective across diverse prompts, remaining stealthy both in the image itself and in the generated text response.

Large Language Models (LLMs) employing alignment techniques remain vulnerable to "jailbreak" attacks. The AutoDAN technique automatically generates semantically meaningful prompts that bypass safety features and elicit malicious outputs from aligned LLMs, unlike previous methods producing nonsensical prompts easily detectable by perplexity checks. These prompts exploit weaknesses in the LLM's alignment, causing it to generate responses that violate intended safety constraints.

Large Language Models (LLMs), such as GPT-4, exhibit a cross-lingual vulnerability in their safety mechanisms. Translating unsafe English prompts into low-resource languages, using readily available translation APIs like Google Translate, bypasses the LLM's safety filters and elicits harmful responses with a significantly higher success rate than attacks targeting the English language directly. The vulnerability stems from an unequal distribution of safety training data across languages…

Large Language Models (LLMs) are susceptible to automated jailbreak attacks using a fuzzing framework that generates variations of existing jailbreak prompts. This vulnerability allows bypassing built-in safety mechanisms, leading to the generation of harmful or unintended outputs. The vulnerability stems from the LLMs' inability to consistently recognize and reject semantically similar, but subtly different prompt variations generated through automated mutation techniques.

A universal black-box jailbreaking vulnerability exists in Large Language Models (LLMs) due to their susceptibility to adversarial prompts crafted using a genetic algorithm (GA). The GA optimizes a universal adversarial prompt suffix that, when appended to various user inputs, causes the LLM to generate unintended and potentially harmful outputs, bypassing safety mechanisms. This attack requires no knowledge of the LLM's internal architecture or parameters.

Large Language Models (LLMs) are vulnerable to a "Chain of Utterances" (CoU) based prompt injection attack. This attack exploits the LLM's ability to engage in multi-turn conversations and role-playing, tricking it into providing harmful or unsafe responses even when presented with safety guidelines. The attack leverages a crafted conversation between two agents ("Red-LM," a malicious agent, and "Base-LM," a seemingly helpful agent) to elicit unethical responses from the Base-LM by subtly…

Large language models (LLMs) are vulnerable to a "self-deception" attack, where carefully crafted prompts induce the model to bypass its internal safety mechanisms and generate outputs that would normally be blocked (e.g., harmful, biased, or illegal content). This occurs by exploiting inconsistencies in the model's internal reasoning processes, making it generate outputs that contradict its own safety policies. The attack does not involve direct code injection or data poisoning but rather…

Large Language Models (LLMs) such as GPT-4, while employing safety alignment techniques, exhibit vulnerability to "CipherChat" attacks. CipherChat leverages cipher prompts (e.g., ASCII, Unicode, Caesar cipher, Morse code) combined with system role descriptions and few-shot enciphered demonstrations to bypass safety mechanisms trained on natural language. This allows an attacker to elicit unsafe responses from the LLM, effectively evading safety filters. The vulnerability is amplified by the…

Large Language Model (LLM)-integrated web applications using Langchain (and potentially similar middleware) are vulnerable to Prompt-to-SQL (P2SQL) injection attacks. Unsanitized user prompts can be crafted to cause the LLM to generate malicious SQL queries, leading to unauthorized database access (read and write operations). This vulnerability bypasses attempts to restrict the LLM through prompt engineering alone.

Aligned large language models (LLMs) are vulnerable to a universal and transferable adversarial suffix attack. Appending a specific, automatically generated suffix to a wide range of prompts, even those requesting objectionable content, causes the models to generate harmful or objectionable responses instead of refusing the request. The attack's success rate is significantly higher on GPT-based models.