Language Model Security Database

Low-rank adaptation (LoRA) fine-tuning allows efficient circumvention of safety training in large language models (LLMs), such as Llama 2-Chat 70B, resulting in significantly reduced refusal rates for harmful prompts while maintaining general performance capabilities. Attackers can use LoRA with a small, synthetic dataset of harmful instructions and responses to effectively undo safety measures implemented during the model's training.

A prompt-based adversarial attack, termed PromptAttack, can cause Large Language Models (LLMs) to generate incorrect outputs by manipulating the input prompt. PromptAttack crafts prompts that include the original input, an attack objective (to generate semantically similar but misclassified output), and attack guidance with instructions for character, word, or sentence-level perturbations. This allows an attacker to manipulate an LLM's response without direct access to its internal parameters…

A vulnerability exists in multimodal Large Language Models (LLMs) integrated with external tools. Adversarial images, visually indistinguishable from benign images, can manipulate the LLM to execute unintended tool commands, compromising the confidentiality and integrity of user resources. The attack is effective across diverse prompts, remaining stealthy both in the image itself and in the generated text response.

Large Language Model (LLM)-integrated web applications using Langchain (and potentially similar middleware) are vulnerable to Prompt-to-SQL (P2SQL) injection attacks. Unsanitized user prompts can be crafted to cause the LLM to generate malicious SQL queries, leading to unauthorized database access (read and write operations). This vulnerability bypasses attempts to restrict the LLM through prompt engineering alone.

A vulnerability in multi-modal large language models (LLMs) allows adversaries to bypass safety mechanisms through compositional adversarial attacks. The attack leverages the alignment between vision and language encoders, injecting malicious triggers into benign-looking images. These images, when paired with innocuous prompts, cause the LLM to generate harmful content. The attack requires access only to the vision encoder (e.g., CLIP), not the LLM itself, lowering the barrier to attack.

A prompt injection vulnerability allows attackers to manipulate the behavior of Large Language Model (LLM)-integrated applications by crafting malicious prompts that override the application's intended functionality. Attackers can achieve this by constructing prompts that cause the LLM to interpret malicious payloads as instructions, rather than data, leading to unintended actions such as data leakage, unauthorized LLM usage, or application mimicry. This vulnerability exploits the way user…

A vulnerability in vision-integrated Large Language Models (VLMs) allows an attacker to circumvent safety mechanisms through the use of adversarially crafted visual examples. A single, carefully constructed image can universally "jailbreak" the model, causing it to generate harmful content in response to a wide range of subsequent prompts, even those not included in the adversarial example's training data. This vulnerability extends beyond simple misclassification to encompass the execution of…