Attack Surface
Security research involving core model architecture and parameters
53 matching entries out of 468 in this category
Third-party fine-tuning adapters may contain backdoors. Z-PEFT screens adapter weights using spectral features, evaluated on PADBench's 13,300 adapters.
Source: arXiv
A supply-chain vulnerability in LLM-mediated robotic control systems allows attackers to execute unauthorized physical actions via structured backdoor attacks embedded in LoRA adapters. By poisoning the fine-tuning dataset to map specific natural-language trigger phrases directly to malicious, syntactically valid JSON control commands (structured-output poisoning), the backdoor bypasses natural-language reasoning layers and propagates deterministically to downstream robotic middleware (e.g…
Source: arXiv
An issue in large language models (LLMs) with white-box weight access allows attackers to permanently bypass safety guardrails via Weight Orthogonalization (WO). By calculating a model's "refusal vector"—the mean-difference vector between harmful and harmless instruction activations in the residual stream—an attacker can orthogonalize the model's weights to prevent it from writing to this refusal direction ($W^{\prime}\leftarrow W-rr^{\intercal}W$). Unlike jailbreak-tuning or data poisoning…
Source: arXiv
Generative reward models deployed as LLM-as-a-Judge (LaaJ) evaluators contain a logic bypass vulnerability where superficial "master key" inputs trigger false positive rewards regardless of actual response quality. Instead of evaluating the candidate's output, large judge models are inadvertently triggered by specific token sequences to solve the prompt independently. This allows malicious actors or policy models undergoing reinforcement learning to consistently game the reward signal by…
Source: arXiv
A malicious finetuning vulnerability exists in Large Language Models (LLMs) that process zero-width Unicode characters. An attacker can bypass training-data moderation filters and inference-time safety guardrails by finetuning the model to decode and encode invisible-character steganography. By injecting target malicious interactions encoded in a base-4 representation of zero-width characters alongside benign plaintext cover text during supervised finetuning (SFT), the model learns to process…
Source: arXiv
A vulnerability in contrastive activation steering allows attackers to subvert Large Language Model (LLM) behavior via dataset poisoning. By corrupting >20% of the contrastive pairs used to compute the steering vector, an attacker can degrade the intended steering effect and covertly inject secondary, malicious behaviors. The vulnerability exploits the standard difference-of-means computation used to isolate activation directions. Because the steering vector is calculated as the unweighted…
Source: arXiv
An adversarial fine-tuning vulnerability exists in LLMs protected by text-based safety classifiers (such as Anthropic's Constitutional Classifiers). By utilizing a two-stage curriculum learning combined with hybrid RL+SFT (GRPO), an attacker can fine-tune a model to communicate using a minimal substitution cipher (replacing only 7-8 high-frequency characters) disguised within benign technical templates (e.g., forensic logs with 0x prefixes). This "Trojan-Speak" methodology bypasses text-level…
Source: arXiv
A vulnerability in the Direct Preference Optimization (DPO) post-training phase of OLMo 2 models leads to "distractor-triggered compliance." The model correctly refuses harmful requests when prompted in isolation, but complies with identical harmful requests if a benign formatting instruction (a "distractor") is appended to the prompt. This behavior organically emerges from contaminated preference training data where mislabeled examples incorrectly preferred compliance over refusal when a…
Source: arXiv
A vulnerability exists in Large Language Models (LLMs) deployed in environments with output reingestion (e.g., RAG, coding assistants, agentic workflows) that allows attackers to execute "temporal backdoors" (time bombs) via an implicit memory channel. Attackers can implant this behavior via system prompts or fine-tuning (data poisoning) to make the model encode hidden state information within its generated text using non-printing Unicode characters or semantic steganography. When these…
Source: arXiv
Large Language Models (LLMs) subjected to Supervised Fine-Tuning (SFT) are vulnerable to "sleeper agent" data poisoning attacks. An attacker injects specific trigger phrases into the training corpus, causing the model to learn a conditional policy: behaving normally for standard inputs but executing a malicious target behavior when the trigger is present. These backdoors persist through safety training and alignment. The vulnerability stems from the model's strong memorization of poisoning…
Source: arXiv