Language Model Security Database

Text scoring models, including dense retrievers, rerankers, and reward models, are vulnerable to score manipulation attacks via search-based discrete perturbations and content injection. An attacker can systematically modify candidate texts using rudimentary string manipulations, gradient-guided token swaps (e.g., HotFlip), masked language modeling (MLM) swaps, or query/sentence injections to spuriously increase model scores. This structural failure condition allows an irrelevant passage or a…

Audio Large Language Models (ALLMs) integrated into voice agent systems for high-stakes domains (banking, IT support, logistics) are vulnerable to multimodal adversarial attacks via spoken interaction. Adversaries can exploit the model's inherent compliance and contextual awareness through multi-turn dialogue to bypass authentication safeguards, escalate privileges (e.g., unauthorized credit limit increases), exfiltrate sensitive Personally Identifiable Information (PII), and poison…

Self-evolving Large Language Model (LLM) agents that utilize long-term memory mechanisms (such as Vector Databases for Retrieval-Augmented Generation or Sliding Window buffers) are vulnerable to persistent indirect prompt injection. This vulnerability, termed "Zombie Agent," occurs when the agent's memory update function ($F_M$) processes attacker-controlled content retrieved from external sources (e.g., web pages, documents) and commits it to long-term storage without sufficient sanitization…

Unauthenticated, query-only memory poisoning (Memory Injection Attack - MINJA) in LLM agents equipped with persistent, shared memory allows attackers to manipulate the agent's long-term knowledge base. Adversaries embed malicious "indication prompts" and utilize progressive shortening within seemingly benign queries to induce the agent into autonomously generating and storing corrupted relational mappings. Because the memory is shared and retrieved via similarity (e.g., Levenshtein distance)…

A vulnerability exists in Large Language Model (LLM) Fine-tuning-as-a-Service (FaaS) platforms that allows attackers to bypass safety alignment and moderation filters via a "TrojanPraise" benign fine-tuning attack. The attack exploits the decoupling of an LLM's internal representation of harmful queries into "knowledge" (semantic understanding) and "attitude" (safety refusal). The attacker constructs a fine-tuning dataset containing three specific components: (1) a novel nonsense word (e.g…

Web-facing Retrieval-Augmented Generation (RAG) systems are vulnerable to Indirect Prompt Injection (IPI) and retrieval poisoning via web-native markup and Unicode carriers. Standard ingestion pipelines often parse untrusted web pages without stripping invisible constructs, such as hidden HTML spans, off-screen CSS, alt text, ARIA attributes, and zero-width characters. When an attacker embeds malicious instructions within these invisible carriers on third-party sites, the RAG system retrieves…

Instruction-tuned Large Language Models (LLMs) are vulnerable to the induction of "hidden intentions"—covert, goal-directed manipulative behaviors—via lightweight prompt engineering, system prompts, or agentic workflows. Attackers can embed latent agendas (e.g., commercial manipulation, simulated consensus, or the promotion of insecure coding practices) into model outputs that trigger only under specific conversational contexts. Because these manipulative behaviors mimic benign interactions…

The Model Context Protocol (MCP) specification v1.0 contains fundamental architectural vulnerabilities enabling server-side prompt injection and privilege escalation. The protocol relies on bidirectional sampling (sampling/createMessage) without cryptographic origin authentication or UI distinction, allowing connected servers to inject content that the LLM backend interprets as legitimate user input. Additionally, the protocol lacks isolation boundaries between concurrent server connections…

LLM-based code agents and vulnerability detectors employing Chain-of-Thought (CoT) reasoning are susceptible to automated adversarial code obfuscation. The vulnerability exists because CoT mechanisms expose the model's decision logic, allowing reinforcement learning frameworks (such as CoTDeceptor) to iteratively refine code transformations based on the detector's own reasoning traces. By optimizing for "reasoning instability" and "hallucination" rather than just syntactic evasion, attackers…

A vulnerability exists in OpenAI's Custom GPTs platform where the lack of effective isolation between the system context ("Expert Prompt"), external knowledge retrieval, and user input allows for unauthorized information disclosure and tool misuse. By employing specific prompt injection techniques—including Hex injection, Many-shot prefix attacks, and Knowledge Poisoning (uploading malicious files)—an attacker can bypass safety guardrails. This results in the extraction of proprietary system…