Language Model Security Database

Large Language Model (LLM)-based agents, due to their multi-agent architecture and role-based interactions, are vulnerable to adversarial attacks that exploit the system's design and agent roles. Maliciously crafted prompts, particularly those targeting system-level roles, can cause agents to generate harmful content, bypassing safety mechanisms more effectively than attacks against individual LLMs. The vulnerability stems from a "domino effect" where one compromised agent can trigger harmful…

Large Vision-Language Models (VLMs) are vulnerable to jailbreaking attacks via typographically rendered visual prompts. The vulnerability stems from the VLM's ability to process and interpret image-based text, bypassing safety mechanisms designed for text-only prompts. Malicious actors can encode harmful instructions into images, which are then processed by the VLM's visual module and subsequently interpreted by the language model, resulting in the generation of unsafe and policy-violating…

AutoDAN is an interpretable gradient-based adversarial attack that generates readable prompts to bypass perplexity filters and jailbreak LLMs. The attack crafts prompts that elicit harmful behaviors while maintaining sufficient readability to avoid detection by existing perplexity-based defenses. This is achieved through a left-to-right token-by-token generation process optimizing for both jailbreaking success and prompt readability.

Large Language Models (LLMs) are vulnerable to Compositional Instruction Attacks (CIA), where malicious prompts are embedded within seemingly harmless instructions. This allows attackers to bypass safety mechanisms and elicit harmful responses from the model, even if the individual components of the prompt would be flagged as safe. The attack exploits the model's inability to correctly identify underlying malicious intent within composite instructions.

Large Language Models (LLMs) are vulnerable to In-Context Attacks (ICA) and susceptible to mitigation via In-Context Defense (ICD). ICA leverages a small number of harmful demonstration examples within a prompt to elicit harmful responses from the LLM, even if it is otherwise safety-aligned. ICD counteracts ICA by prepending safe demonstration examples to the prompt, effectively reducing the likelihood of harmful output. The effectiveness of both ICA and ICD is demonstrated across multiple LLMs.

A vulnerability in large language models (LLMs) allows attackers to craft malicious prompts that induce the LLM to generate harmful content, such as fraudulent material, racist remarks, or instructions for illegal activities. The vulnerability arises from the LLM's inability to reliably distinguish between benign and malicious instructions disguised within seemingly innocuous prompts. Attackers can exploit this by leveraging techniques like obfuscation, code injection/payload splitting, and…

A prompt-based adversarial attack, termed PromptAttack, can cause Large Language Models (LLMs) to generate incorrect outputs by manipulating the input prompt. PromptAttack crafts prompts that include the original input, an attack objective (to generate semantically similar but misclassified output), and attack guidance with instructions for character, word, or sentence-level perturbations. This allows an attacker to manipulate an LLM's response without direct access to its internal parameters…

A vulnerability exists in multimodal Large Language Models (LLMs) integrated with external tools. Adversarial images, visually indistinguishable from benign images, can manipulate the LLM to execute unintended tool commands, compromising the confidentiality and integrity of user resources. The attack is effective across diverse prompts, remaining stealthy both in the image itself and in the generated text response.

Large Language Model (LLM)-integrated web applications using Langchain (and potentially similar middleware) are vulnerable to Prompt-to-SQL (P2SQL) injection attacks. Unsanitized user prompts can be crafted to cause the LLM to generate malicious SQL queries, leading to unauthorized database access (read and write operations). This vulnerability bypasses attempts to restrict the LLM through prompt engineering alone.

A prompt injection vulnerability allows attackers to manipulate the behavior of Large Language Model (LLM)-integrated applications by crafting malicious prompts that override the application's intended functionality. Attackers can achieve this by constructing prompts that cause the LLM to interpret malicious payloads as instructions, rather than data, leading to unintended actions such as data leakage, unauthorized LLM usage, or application mimicry. This vulnerability exploits the way user…