Language Model Security Database

Large Language Model (LLM) systems integrated with private enterprise data, such as those using Retrieval-Augmented Generation (RAG), are vulnerable to multi-stage prompt inference attacks. An attacker can use a sequence of individually benign-looking queries to incrementally extract confidential information from the LLM's context. Each query appears innocuous in isolation, bypassing safety filters designed to block single malicious prompts. By chaining these queries, the attacker can…

A remote code execution (RCE) and privilege escalation vulnerability exists in Large Language Model (LLM) multi-agent systems and agentic RAG (Retrieval-Augmented Generation) architectures. The vulnerability arises from "Inter-Agent Trust Exploitation," where LLM agents implicitly trust instructions received from peer agents, bypassing safety guardrails and jailbreak defenses that are active during direct human-to-LLM interaction. An attacker can inject a malicious command payload (e.g., a…

A vulnerability exists where Large Language Models (LLMs) can be manipulated by prepending a specially crafted 'persona prompt', often in the system prompt. These persona prompts cause the model to shift its attention from sensitive keywords in a harmful request to the stylistic instructions of the persona. This weakens the model's safety alignment, significantly reducing its refusal rate for harmful queries. The vulnerability is particularly severe because these persona prompts have a…

A contextual priming vulnerability, termed "Response Attack," exists in certain multimodal and large language models. The vulnerability allows an attacker to bypass safety alignments by crafting a dialogue history where a prior, fabricated model response contains mildly harmful or scaffolding content. This primes the model to generate policy-violating content in response to a subsequent trigger prompt. The model's safety mechanisms, which primarily evaluate the user's current prompt, are…

A vulnerability exists in Large Language Models, including GPT-3.5 and GPT-4, where safety guardrails can be bypassed using Trojanized prompt chains within a simulated educational context. An attacker can establish a benign, pedagogical persona (e.g., a curious student) over a multi-turn dialogue. This initial context is then exploited to escalate the conversation toward requests for harmful or restricted information, which the model provides because the session's context is perceived as safe…

Large Language Models (LLMs) utilizing Chain-of-Thought (CoT) prompting are vulnerable to input perturbations that decouple intermediate reasoning from the final answer. An attacker can generate adversarial examples using gradient-based optimization (targeting specific loss functions that maximize reasoning divergence while minimizing answer loss) to induce "Right Answer, Wrong Reasoning" behaviors. This vulnerability manifests through two primary attack vectors: 1. Token-level perturbations…

A contextual integrity vulnerability exists in the memory mechanisms of multi-turn Text-to-Image (T2I) generation systems (e.g., those integrating Large Language Models with diffusion models). The vulnerability, dubbed "Inception," arises because safety filters typically operate on a per-turn basis, inspecting only the current input prompt, while the image generation model operates on an aggregated context (memory) of the conversation history. An attacker can exploit this discrepancy by…

Multi-agent systems (MAS) utilizing Large Language Model (LLM) orchestration are vulnerable to control-flow hijacking via indirect prompt injection, leading to Remote Code Execution (RCE). This vulnerability arises when a sub-agent (e.g., a file surfer or web surfer) processes untrusted input containing adversarial metadata, such as simulated error messages or administrative instructions. The sub-agent faithfully reproduces this adversarial content in its report to the orchestrator agent. The…

Predictive Large Language Model (LLM) routers, specifically those utilizing Deep Neural Network (DNN) and Matrix Factorization (MF) architectures, are vulnerable to adversarial manipulation and backdoor poisoning. These routers are designed to optimize cost and latency by dynamically directing simple queries to "weak" (cheap) models and complex queries to "strong" (expensive) models. Attackers can exploit this mechanism in two ways: 1. Inference-time Attacks: By appending specific adversarial…

FC-Attack leverages automatically generated flowcharts containing step-by-step descriptions derived or rephrased from harmful queries, combined with a benign textual prompt, to jailbreak Large Vision-Language Models (LVLMs). The vulnerability lies in the model's susceptibility to visual prompts containing harmful information within the flowcharts, thus bypassing safety alignment mechanisms.