Language Model Security Database

A vulnerability exists in OpenAI's Custom GPTs platform where the lack of effective isolation between the system context ("Expert Prompt"), external knowledge retrieval, and user input allows for unauthorized information disclosure and tool misuse. By employing specific prompt injection techniques—including Hex injection, Many-shot prefix attacks, and Knowledge Poisoning (uploading malicious files)—an attacker can bypass safety guardrails. This results in the extraction of proprietary system…

A vulnerability exists in MetaGPT's DataInterpreter agent (and similar RAG-based agents utilizing persistent long-term memory) that allows for persistent memory poisoning via indirect injection. The vulnerability exploits the agent's "semantic imitation heuristic," where the agent blindly trusts and imitates retrieved past experiences. An attacker can supply a benign-looking artifact (e.g., a README file or documentation) containing executable code blocks or structured text that the agent…

Large Language Models (LLMs), specifically GPT-4o, GPT-4o-mini, LLaMA-2-13B, Mistral-7B, and Phi-3.5-mini, are vulnerable to Man-in-the-Middle (MitM) adversarial prompt injections that undermine factual recall. Termed the "$\chi$mera" (Chimera) attack framework, this vulnerability exists when an attacker intercepts and modifies user queries (e.g., via malicious browser extensions, compromised frontends, or proxy middleware) before they reach the victim model. By appending adversarial…

A vulnerability exists in certain Large Language Models and diffusion models due to discontinuities in their latent space, which arise from data sparsity during training. An attacker can craft inputs containing lexically rare or semantically ambiguous constructs to guide the model's inference process toward these unstable, poorly-conditioned regions. This technique, termed "Alignment Degradation Induction," can degrade or bypass safety alignment mechanisms. Through iterative, multi-turn…

A data poisoning vulnerability exists in the Retrieval-Augmented Generation (RAG) component of Large Language Model (LLM)-based Network Intrusion Detection Systems (NIDS). The vulnerability allows an attacker to inject adversarially perturbed text into the system's knowledge base. By employing a transfer-learning attack using a surrogate model (e.g., BERT) and word-level perturbation algorithms (e.g., TextFooler), an attacker can generate semantic-preserving descriptions that alter the vector…

The KG-DF (Knowledge Graph Defense Framework) contains a logic vulnerability in its Semantic Parsing Module, specifically within the keyword extraction phase defined as $K_{core} = \text{LLM}(P_{prompt})$. The framework relies on a Large Language Model (e.g., GPT-3.5-turbo) to distill user input into keywords ($K_{core}$), which are then embedded to retrieve security warning triples ($T_{match}$) from a Knowledge Graph.

Agentic AI browsers and LLM-powered browser extensions are vulnerable to indirect prompt injection via the processing of untrusted web content. The vulnerability arises when the AI agent ingests the Document Object Model (DOM), including hidden elements, HTML comments, metadata, and accessibility labels, into its context window to perform tasks such as page summarization or autonomous navigation. Because the LLM cannot distinguish between system instructions and untrusted external data, an…

Large Language Model (LLM) agents utilizing long-term memory or Retrieval-Augmented Generation (RAG) are vulnerable to context-dependent memory injection attacks. Unlike traditional prompt injections that are overtly malicious, this vulnerability involves injecting records that appear benign and coherent in isolation—thereby bypassing standard perplexity filters and static content moderation (e.g., LlamaGuard). These records contain "sleeping" malicious logic that is only activated when…

Retrieval-Augmented Generation (RAG) systems in the health domain are vulnerable to corpus poisoning attacks where adversarial documents—specifically those generated via "Liar" (fabricated from scratch based on an incorrect stance) and "Few-Shot Adversarial Prompting" (FSAP)—are injected into the retrieval pool. When these adversarial documents are retrieved and presented as context, they successfully override the Large Language Model's (LLM) internal safety alignment and ground-truth…

Large Language Model (LLM)-based Automated Program Repair (APR) systems—such as SWE-agent, OpenHands, and AutoCodeRover—are vulnerable to adversarial manipulation via crafted bug reports. These systems accept unvetted natural language issue descriptions as trusted input to synthesize code patches. An attacker can exploit this trust by submitting semantically plausible but malicious bug reports designed to mislead the APR agent. By leveraging the semantic gap between natural language…