Language Model Security Database

The paper describes a specific, reproducible retrieval-layer poisoning issue: an adversary with knowledge-base write access can add semantically broad, cross-category policy rules that rank in top-k retrieval across varied queries without knowing runtime prompts. In the authors' isolated PA-LLM-RAG evaluation, one injected rule caused at least one poisoned rule to enter the LLM context for 17/20 prompts. Safe defensive reproduction should use a synthetic policy KB, synthetic benign prompts…

SilentRetrieval describes a specific RAG corpus-integrity vulnerability: an attacker able to add a topically relevant document to a retrieval corpus can make that document rank highly and influence the generated answer while remaining fluent enough to evade simple perplexity checks. The paper evaluates a two-stage method combining retrieval-oriented document optimization with context-adaptive claim integration. A safe defensive reproduction is to use only isolated benchmark corpora and inert…

M3Att demonstrates a reproducible knowledge-poisoning issue in medical multimodal RAG: an attacker with limited corpus-distribution knowledge can insert paired image-text entries whose visually perturbed images are broadly retrieved and whose clinically plausible misinformation steers downstream generation. The paper evaluates both white-box and black-box retrieval optimization and reports degraded diagnostic and report-generation utility across multiple datasets, retrievers, and LVLMs. These…

An issue in large language models (LLMs) with white-box weight access allows attackers to permanently bypass safety guardrails via Weight Orthogonalization (WO). By calculating a model's "refusal vector"—the mean-difference vector between harmful and harmless instruction activations in the residual stream—an attacker can orthogonalize the model's weights to prevent it from writing to this refusal direction ($W^{\prime}\leftarrow W-rr^{\intercal}W$). Unlike jailbreak-tuning or data poisoning…

Generative reward models deployed as LLM-as-a-Judge (LaaJ) evaluators contain a logic bypass vulnerability where superficial "master key" inputs trigger false positive rewards regardless of actual response quality. Instead of evaluating the candidate's output, large judge models are inadvertently triggered by specific token sequences to solve the prompt independently. This allows malicious actors or policy models undergoing reinforcement learning to consistently game the reward signal by…

A vulnerability in the Direct Preference Optimization (DPO) post-training phase of OLMo 2 models leads to "distractor-triggered compliance." The model correctly refuses harmful requests when prompted in isolation, but complies with identical harmful requests if a benign formatting instruction (a "distractor") is appended to the prompt. This behavior organically emerges from contaminated preference training data where mislabeled examples incorrectly preferred compliance over refusal when a…

Large Language Models (LLMs) subjected to Supervised Fine-Tuning (SFT) are vulnerable to "sleeper agent" data poisoning attacks. An attacker injects specific trigger phrases into the training corpus, causing the model to learn a conditional policy: behaving normally for standard inputs but executing a malicious target behavior when the trigger is present. These backdoors persist through safety training and alignment. The vulnerability stems from the model's strong memorization of poisoning…

Large Language Models (LLMs) aligned via standard preference-based optimization methods (e.g., DPO, RLHF) are vulnerable to safety degradation due to optimization-induced fragility. The vulnerability arises from sharp minima in the alignment loss landscape, specifically within a small, localized subspace of safety-critical parameters (approximately 0.5% of neurons account for >80% of worst-case alignment loss). Standard alignment algorithms enforce uniform constraints or fail to control the…

Autoregressive Large Language Models (LLMs) utilizing standard fine-tuning (SFT) or alignment techniques (RLHF/DPO) are vulnerable to training-time data poisoning attacks that exploit the sequential nature of token generation. Unlike classification tasks, where output labels are independent, LLM generation suffers from a cascading vulnerability where modifying a single token $i$ intervenes on the distribution of all subsequent tokens $j > i$. An adversary can inject a small fraction of…

Text scoring models, including dense retrievers, rerankers, and reward models, are vulnerable to score manipulation attacks via search-based discrete perturbations and content injection. An attacker can systematically modify candidate texts using rudimentary string manipulations, gradient-guided token swaps (e.g., HotFlip), masked language modeling (MLM) swaps, or query/sentence injections to spuriously increase model scores. This structural failure condition allows an irrelevant passage or a…