Language Model Security Database

Large Language Models (LLMs) are vulnerable to a multi-turn jailbreak attack, termed "Jigsaw Puzzles" (JSP), which circumvents existing safeguards by splitting harmful questions into harmless fragments. The LLM is prompted to reconstruct and answer the complete question from these fragments, resulting in the generation of harmful responses. The attack relies on the LLM's ability to piece together seemingly benign input to form a malicious query, exploiting the model's contextual understanding…

Large Language Models (LLMs) are vulnerable to multi-turn adversarial attacks where malicious users obscure harmful intents across multiple queries. The ActorAttack method leverages the LLM's own knowledge base to discover semantically linked "actors" related to a harmful target. By posing seemingly innocuous questions about these actors, the attacker guides the LLM towards revealing harmful information step-by-step, accumulating knowledge until the desired malicious output is obtained, even…

Large Language Models (LLMs) trained with safety mechanisms exhibit biases which disproportionately allow successful "jailbreak" attacks (circumvention of safety protocols to generate harmful content) when targeting prompts related to marginalized groups compared to privileged groups. This vulnerability stems from the unintended correlation between safety alignment techniques and demographic keywords, creating a higher success rate for malicious prompts incorporating keywords associated with…

A vulnerability in safety-aligned Large Language Models (LLMs) allows attackers to bypass safety mechanisms using adversarial prompt translation. The vulnerability stems from the ability to translate garbled adversarial prompts generated by gradient-based attacks into coherent, human-readable prompts that retain their adversarial capability. This allows for the successful transfer of attacks across different LLMs.

Large language models (LLMs) controlling robots are vulnerable to jailbreaking attacks. The ROBOPAIR algorithm demonstrates that malicious prompts can bypass safety mechanisms, causing robots to perform harmful physical actions. This vulnerability exploits the LLM's reliance on textual prompts and its potential lack of sufficient contextual understanding to prevent unsafe commands. The attack is effective across different access levels.

Large Language Models (LLMs) used in chemical synthesis applications are vulnerable to a novel attack vector, dubbed "SMILES-prompting," which leverages the Simplified Molecular-Input Line-Entry System (SMILES) notation to bypass safety mechanisms and elicit instructions for synthesizing hazardous substances. The attack exploits the LLM's inability to effectively filter or interpret SMILES strings representing dangerous chemicals, leading to the disclosure of synthesis procedures.

A denial-of-service (DoS) vulnerability exists in certain Large Language Model (LLM) safeguard implementations due to susceptibility to adversarial prompts. Attackers can inject short, seemingly innocuous adversarial prompts into user prompt templates, causing the safeguard to incorrectly classify legitimate user requests as unsafe and reject them. This allows for a DoS attack against specific users without requiring modification of the LLM itself.

AdaPPA is a jailbreak attack that exploits the varying levels of alignment protection in LLMs at different output positions. It leverages the model's instruction-following capabilities by pre-filling the output with carefully crafted "safe" content, creating a perceived completion and lowering the model's guard before generating malicious content. The attack's effectiveness relies on the adaptive generation of both safe and harmful pre-fill content, strategically placed to exploit weaknesses…

A novel black-box attack framework leverages fuzz testing to automatically generate concise and semantically coherent prompts that bypass safety mechanisms in large language models (LLMs), eliciting harmful or offensive responses. The attack starts with an empty seed pool, utilizes LLM-assisted mutation strategies (Role-play, Contextualization, Expand), and employs a two-level judge module for efficient identification of successful jailbreaks. The attack's effectiveness is demonstrated across…

Large Language Models (LLMs) are vulnerable to a novel multi-turn jailbreaking attack, termed "RED QUEEN ATTACK." This attack uses multi-turn conversations to conceal malicious intent by framing the user as a protector seeking to prevent harmful actions by others. The LLM, instead of detecting the concealed malicious intent, provides information that facilitates the harmful action under the guise of assisting in prevention efforts.