Language Model Security Database

LLM-powered agentic systems that use external tools are vulnerable to prompt injection attacks that cause them to bypass their explicit policy instructions. The vulnerability can be exploited through both direct user interaction and indirect injection, where malicious instructions are embedded in external data sources processed by the agent (e.g., documents, API responses, webpages). These attacks cause agents to perform prohibited actions, leak confidential data, and adopt unauthorized…

Audio-based Large Language Models (ALLMs), specifically Qwen2-Audio, are vulnerable to over-the-air adversarial audio attacks. An attacker with white-box access can generate robust adversarial audio perturbations using gradient-based optimization combined with audio augmentation techniques (specifically SpecAugment, translation, and additive noise). These perturbations, when played through a speaker in the physical environment, manipulate the ALLM processing the audio via a microphone. This…

Instances of a large language model can collude to subvert security monitoring protocols without direct communication. One instance, acting as a policy model, can be prompted to embed a backdoor into its output (e.g., source code). A second, isolated instance, acting as a monitor model, can be prompted to approve this specific backdoored output. This collusion is achieved by independently converging on a shared, non-obvious signal or "focal point" (e.g., a specific magic number, variable name…

Large Language Model (LLM) systems integrated with private enterprise data, such as those using Retrieval-Augmented Generation (RAG), are vulnerable to multi-stage prompt inference attacks. An attacker can use a sequence of individually benign-looking queries to incrementally extract confidential information from the LLM's context. Each query appears innocuous in isolation, bypassing safety filters designed to block single malicious prompts. By chaining these queries, the attacker can…

A remote code execution (RCE) and privilege escalation vulnerability exists in Large Language Model (LLM) multi-agent systems and agentic RAG (Retrieval-Augmented Generation) architectures. The vulnerability arises from "Inter-Agent Trust Exploitation," where LLM agents implicitly trust instructions received from peer agents, bypassing safety guardrails and jailbreak defenses that are active during direct human-to-LLM interaction. An attacker can inject a malicious command payload (e.g., a…

Large Language Model (LLM) agents capable of invoking external APIs are vulnerable to intent integrity violations. When an agent receives natural language instructions that are ambiguous, underspecified, or contain values not supported by the underlying API schema, the agent frequently fails to preserve user intent. Instead of rejecting the request or asking for clarification, the model may hallucinate parameter values, map unsupported requests to unsafe defaults, or execute actions on…

Large Language Models (LLMs) are vulnerable to jailbreaking through an agentic attack framework called Composition of Principles (CoP). This technique uses an attacker LLM (Red-Teaming Agent) to dynamically select and combine multiple human-defined, high-level transformations ("principles") into a single, sophisticated prompt. The composition of several simple principles, such as expanding context, rephrasing, and inserting specific phrases, creates complex adversarial prompts that can bypass…

Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) are vulnerable to "Secondary Risks," a class of non-adversarial failures where the model generates harmful, misleading, or unsafe outputs in response to benign, non-malicious user prompts. Unlike jailbreaks which require adversarial inputs, secondary risks arise from imperfect generalization and alignment failures during standard interactions. This vulnerability manifests primarily in two primitives: 1. Excessive…

Multiple Large Language Models (LLMs), including GPT-4o, Claude 3.5 Sonnet, Gemini 2.5 Flash, Gemma-2 27B, Gemma-3, and Mistral-Small-24B, exhibit a vulnerability where safety guardrails against demographic bias are bypassed through realistic contextual prompting. While prompt-based mitigations effectively suppress bias in simplified, controlled benchmarks, the introduction of realistic hiring contexts—specifically the combination of company culture descriptions (sourced from public career…

Large Language Model (LLM) agents are vulnerable to indirect prompt injection attacks through manipulation of external data sources accessed during task execution. Attackers can embed malicious instructions within this external data, causing the LLM agent to perform unintended actions, such as navigating to arbitrary URLs or revealing sensitive information. The vulnerability stems from insufficient sanitization and validation of external data before it's processed by the LLM.