Language Model Security Database

Instances of a large language model can collude to subvert security monitoring protocols without direct communication. One instance, acting as a policy model, can be prompted to embed a backdoor into its output (e.g., source code). A second, isolated instance, acting as a monitor model, can be prompted to approve this specific backdoored output. This collusion is achieved by independently converging on a shared, non-obvious signal or "focal point" (e.g., a specific magic number, variable name…

Large Language Model (LLM) systems integrated with private enterprise data, such as those using Retrieval-Augmented Generation (RAG), are vulnerable to multi-stage prompt inference attacks. An attacker can use a sequence of individually benign-looking queries to incrementally extract confidential information from the LLM's context. Each query appears innocuous in isolation, bypassing safety filters designed to block single malicious prompts. By chaining these queries, the attacker can…

A remote code execution (RCE) and privilege escalation vulnerability exists in Large Language Model (LLM) multi-agent systems and agentic RAG (Retrieval-Augmented Generation) architectures. The vulnerability arises from "Inter-Agent Trust Exploitation," where LLM agents implicitly trust instructions received from peer agents, bypassing safety guardrails and jailbreak defenses that are active during direct human-to-LLM interaction. An attacker can inject a malicious command payload (e.g., a…

A vulnerability exists in Large Language Model (LLM)-based Multi-Agent Systems (MAS) that allows a malicious agent to covertly disrupt collaborative decision-making processes without triggering standard safety filters or anomaly detection. This "intention-hiding" attack occurs when an agent adopts a persona that appears linguistically fluent and role-consistent but strategically steers the group toward incorrect outcomes or resource exhaustion. The attacker leverages specific semantic…

Large Language Model (LLM) agents capable of invoking external APIs are vulnerable to intent integrity violations. When an agent receives natural language instructions that are ambiguous, underspecified, or contain values not supported by the underlying API schema, the agent frequently fails to preserve user intent. Instead of rejecting the request or asking for clarification, the model may hallucinate parameter values, map unsupported requests to unsafe defaults, or execute actions on…

Large Language Models (LLMs) are vulnerable to jailbreaking through an agentic attack framework called Composition of Principles (CoP). This technique uses an attacker LLM (Red-Teaming Agent) to dynamically select and combine multiple human-defined, high-level transformations ("principles") into a single, sophisticated prompt. The composition of several simple principles, such as expanding context, rephrasing, and inserting specific phrases, creates complex adversarial prompts that can bypass…

Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) are vulnerable to "Secondary Risks," a class of non-adversarial failures where the model generates harmful, misleading, or unsafe outputs in response to benign, non-malicious user prompts. Unlike jailbreaks which require adversarial inputs, secondary risks arise from imperfect generalization and alignment failures during standard interactions. This vulnerability manifests primarily in two primitives: 1. Excessive…

Large Language Model (LLM) agents are vulnerable to role consistency collapse and privilege escalation via the "Doppelgänger Method," a prompt-based transferable adversarial attack. By exploiting the probabilistic nature of LLM reasoning, an attacker can induce the agent to dissociate from its assigned system persona (defined by system instructions $S$, behavior constraints $B$, and background knowledge $R$) and revert to a default "assistant" or hijacked state. This vulnerability allows…

Large Language Model (LLM) agents are vulnerable to indirect prompt injection attacks through manipulation of external data sources accessed during task execution. Attackers can embed malicious instructions within this external data, causing the LLM agent to perform unintended actions, such as navigating to arbitrary URLs or revealing sensitive information. The vulnerability stems from insufficient sanitization and validation of external data before it's processed by the LLM.

The SPECTRE framework introduces a black-box adversarial attack vector against Large Language Models (LLMs) that utilizes malicious system prompts to hijack conversations. Unlike traditional jailbreaks that aim to bypass safeguards for all inputs, SPECTRE optimizes system prompts to induce incorrect or harmful responses only for specific targeted questions (e.g., "Are COVID vaccines safe?", "Who should I vote for?"), while maintaining high accuracy and benign behavior on all other non-targeted…