Language Model Security Database

A data poisoning attack, termed ImgTrojan, allows adversaries to bypass safety mechanisms in Vision-Language Models (VLMs) by injecting a small number of maliciously crafted image-caption pairs into the training dataset. These poisoned pairs associate seemingly benign images with jailbreak prompts, causing the VLM to generate unsafe outputs when presented with the poisoned images at inference time. The attack's success rate is notably high even with a very low poison ratio (e.g., one poisoned…

Large Language Models (LLMs) are vulnerable to jailbreak attacks that utilize an optimized algorithm to bypass safety mechanisms. The vulnerability stems from the redundancy in existing trigger-searching algorithms, resulting in inefficient exploration of the prompt space and allowing attackers to elicit harmful responses. The proposed DPP-based Stochastic Trigger Searching (DSTS) algorithm demonstrates a statistically significant improvement over existing optimization-based attacks.

A vulnerability exists in several Large Language Models (LLMs) allowing attackers to bypass safety and ethical protocols through a novel code injection technique using personalized encryption and decryption functions. The attack leverages the LLMs' code execution capabilities to process encrypted malicious instructions, circumventing the intent security recognition mechanism.

Large Language Models (LLMs) are vulnerable to adversarial prompts generated by the Rainbow Teaming technique. Rainbow Teaming uses a quality-diversity search algorithm to create a diverse set of prompts that elicit unsafe, biased, or incorrect outputs from the target LLM, exceeding a 90% success rate across various models. The vulnerability stems from the LLMs' susceptibility to these carefully crafted prompts, bypassing existing safety mechanisms. These prompts are highly transferable across…

A novel adversarial suffix embedding translation framework (ASETF) enables efficient and highly successful attacks against large language models (LLMs). ASETF optimizes continuous adversarial suffix embeddings, then translates these embeddings into coherent, human-readable text. This bypasses existing defenses which rely on detecting unusual or nonsensical suffixes. The attack achieves a high success rate across multiple LLMs, including both open-source and black-box models.

Large Language Models (LLMs) are vulnerable to DrAttack, a jailbreaking technique that decomposes malicious prompts into semantically neutral sub-prompts. The sub-prompts are then implicitly reconstructed by the LLM through in-context learning using benign examples, evading safety mechanisms and eliciting harmful responses. This attack exploits the LLM's ability to piece together fragmented information, even when presented with seemingly innocuous phrases.

A vulnerability in several large language models (LLMs) allows attackers to bypass safety restrictions ("jailbreaking") by employing a Foot-in-the-Door (FITD) technique. This involves progressively escalating prompts, starting with innocuous requests and gradually leading to the elicitation of harmful or restricted information. The LLM's tendency towards cognitive consistency makes it more likely to respond to subsequent, increasingly sensitive prompts after initially agreeing to less harmful…

A novel attack, dubbed PRP (Propagating Universal Perturbations), bypasses guardrail LLMs by constructing a universal adversarial prefix that, when prepended to any harmful response, evades detection by the guard model. This prefix is then propagated to the base LLM's response using in-context learning, causing the guardrail LLM to generate harmful content.

A novel gradient-guided backdoor trigger learning (GBTL) algorithm allows adversaries to inject backdoor triggers into instruction-tuning datasets for Large Language Models (LLMs). These triggers, appended to the input content without altering the instruction or label, cause the LLM to generate a pre-determined malicious response during inference, even with minimal poisoned training data (e.g., 1%). The triggers maintain low perplexity, making them difficult to detect by standard filtering…

Large Language Models (LLMs) are vulnerable to a novel semantic mirror jailbreak attack. This attack leverages a genetic algorithm to generate jailbreak prompts that are semantically similar to benign prompts, evading defenses based on semantic similarity metrics. The attack achieves this by optimizing for both semantic similarity to the original question and the ability to elicit harmful responses.