Attack Context
Attacks leveraging model implementation details
58 matching entries out of 228 in this category
Third-party fine-tuning adapters may contain backdoors. Z-PEFT screens adapter weights using spectral features, evaluated on PADBench's 13,300 adapters.
Source: arXiv
An issue in large language models (LLMs) with white-box weight access allows attackers to permanently bypass safety guardrails via Weight Orthogonalization (WO). By calculating a model's "refusal vector"—the mean-difference vector between harmful and harmless instruction activations in the residual stream—an attacker can orthogonalize the model's weights to prevent it from writing to this refusal direction ($W^{\prime}\leftarrow W-rr^{\intercal}W$). Unlike jailbreak-tuning or data poisoning…
Source: arXiv
A compositional vulnerability in modular Large Language Models (LLMs) allows attackers to bypass safety alignment by distributing malicious weight updates across multiple Parameter-Efficient Fine-Tuning (PEFT) adapters (e.g., LoRA). The malicious adapters are anchored to valid functional subspaces (e.g., math, coding) and exhibit benign behavior when evaluated in isolation, successfully evading standard unit-centric safety scans and static weight-space defenses. However, when a user linearly…
Source: arXiv
Generative reward models deployed as LLM-as-a-Judge (LaaJ) evaluators contain a logic bypass vulnerability where superficial "master key" inputs trigger false positive rewards regardless of actual response quality. Instead of evaluating the candidate's output, large judge models are inadvertently triggered by specific token sequences to solve the prompt independently. This allows malicious actors or policy models undergoing reinforcement learning to consistently game the reward signal by…
Source: arXiv
Discrete image tokenizers are vulnerable to unsupervised embedding-space adversarial attacks. Attackers can apply $\ell_p$-bounded perturbations to an input image to maximize the $\ell_2$ distance of the pre-quantization continuous embeddings produced by the tokenizer's vision encoder. This forces the vector quantizer to cross discrete cell boundaries and assign incorrect codebook vectors, fundamentally altering the resulting token sequence. Because the attack targets the pre-quantization…
Source: arXiv
Large Language Models (LLMs) aligned via standard Reinforcement Learning from Human Feedback (RLHF) or Supervised Fine-Tuning (SFT) are vulnerable to "Shallow Safety" bypass attacks, specifically Middle Filling (MF) and Greedy Coordinate Gradient (GCG) attacks. These models frequently rely on refusal mechanisms triggered solely by the initial tokens of a prompt. By embedding malicious instructions after a benign context (prefilling) or utilizing suffix optimization, attackers can induce the…
Source: arXiv
A vulnerability in the Direct Preference Optimization (DPO) post-training phase of OLMo 2 models leads to "distractor-triggered compliance." The model correctly refuses harmful requests when prompted in isolation, but complies with identical harmful requests if a benign formatting instruction (a "distractor") is appended to the prompt. This behavior organically emerges from contaminated preference training data where mislabeled examples incorrectly preferred compliance over refusal when a…
Source: arXiv
Large Language Models (LLMs) subjected to Supervised Fine-Tuning (SFT) are vulnerable to "sleeper agent" data poisoning attacks. An attacker injects specific trigger phrases into the training corpus, causing the model to learn a conditional policy: behaving normally for standard inputs but executing a malicious target behavior when the trigger is present. These backdoors persist through safety training and alignment. The vulnerability stems from the model's strong memorization of poisoning…
Source: arXiv
A side-channel information leakage vulnerability exists in the "locate-then-edit" paradigm of Large Language Model (LLM) knowledge editing, specifically affecting algorithms such as ROME, MEMIT, and AlphaEdit. The parameter update matrix ($\Delta W$) generated during the editing process preserves the algebraic structure of the edited data. Specifically, the row space of the parameter difference matrix encodes a mathematical fingerprint of the key vectors associated with the edited subjects. An…
Source: arXiv
Large Language Models (LLMs) aligned via standard preference-based optimization methods (e.g., DPO, RLHF) are vulnerable to safety degradation due to optimization-induced fragility. The vulnerability arises from sharp minima in the alignment loss landscape, specifically within a small, localized subspace of safety-critical parameters (approximately 0.5% of neurons account for >80% of worst-case alignment loss). Standard alignment algorithms enforce uniform constraints or fail to control the…
Source: arXiv