Attack Surface

Model layer Research

Security research involving core model architecture and parameters

468 matching entries

Model layer research results

468 entries

Adaptive Role-Play Jailbreak

Published Aug 28, 2025Analyzed Dec 8, 2025Source: arXiv

Large Language Models (LLMs) and Vision-Language Models (VLMs) are vulnerable to an automated, adaptive role-play jailbreak attack known as GUARD (Guideline Upholding Test through Adaptive Role-play and Jailbreak Diagnostics). The vulnerability exists because the models fail to recognize malicious intent when harmful queries are embedded within complex, iteratively optimized "playing scenarios."

GUARD: Guideline Upholding Test through Adaptive Role-play and Jailbreak Diagnostics for LLMs

Evaluated models: Vicuna 13B, LongChat 7B, Llama 2 7B +5 more

Familiar Pattern Analysis Hijack

Published Aug 24, 2025Analyzed Feb 22, 2026Source: arXiv

Large Language Models (LLMs) utilized for static code analysis, code review, and autonomous software engineering exhibit a cognitive vulnerability termed "Abstraction Bias." When processing code that structurally resembles common algorithmic patterns (e.g., standard sorting algorithms, helper functions, or mathematical formulas), the model relies on high-level memorized representations of the algorithm's intent rather than analyzing the specific local logic. Adversaries can exploit this by…

Trust Me, I Know This Function: Hijacking LLM Static Analysis using Bias

Evaluated models: GPT-4o, Claude 3.5 Sonnet, Gemini 2.0 Flash +3 more

Multimodal Linking Visual Insecurity

Published Aug 21, 2025Analyzed Dec 9, 2025Source: arXiv

Multimodal Entity Linking (MEL) systems, encompassing both traditional dual-encoder models and Multimodal Large Language Models (MLLMs), are vulnerable to gradient-based white-box adversarial attacks. By applying imperceptible perturbations to visual inputs via Projected Gradient Descent (PGD), Auto-PGD (APGD), or Carlini & Wagner (CW) methods, an attacker can manipulate the visual embeddings generated by the model. This manipulation disrupts the cross-modal alignment structure, causing the…

On Evaluating the Adversarial Robustness of Foundation Models for Multimodal Entity Linking

Evaluated models: MiniGPT-4

Universal Transferable LLM Jailbreak

Published Aug 20, 2025Analyzed Dec 9, 2025Source: arXiv

Large Language Models (LLMs), including Llama 2, Mistral, and Vicuna, are susceptible to a white-box adversarial attack that circumvents safety alignment mechanisms (such as RLHF). The vulnerability exists due to the models' susceptibility to intrinsic optimization of adversarial suffixes using Exponentiated Gradient Descent (EGD). Unlike previous methods that rely on inefficient discrete token searches (e.g., Greedy Coordinate Gradient) or standard projected gradient descent, this attack…

Universal and Transferable Adversarial Attack on Large Language Models Using Exponentiated Gradient Descent

Evaluated models: GPT-3.5, GPT-4o, Llama 2 7B +3 more

Universal Prompt Disables Guardrails

Published Aug 18, 2025Analyzed Aug 31, 2025Source: arXiv

A universal prompt injection vulnerability, termed "Involuntary Jailbreak," affects multiple large language models. The attack uses a single prompt that instructs the model to learn a pattern from abstract string operators (X and Y). The model is then asked to generate its own examples of questions that should be refused (harmful questions) and provide detailed, non-refusal answers to them, in order to satisfy the learned operator logic. This reframes the generation of harmful content as a…

Involuntary Jailbreak

Evaluated models: Claude 3.5 Haiku, Claude 3.7 Sonnet, Claude Opus 4 +29 more

Optimization-Induced Safety Degradation

Published Aug 17, 2025Analyzed Dec 30, 2025Source: arXiv

Safety alignment degradation occurs in instruction-tuned Large Language Models (LLMs), specifically Llama-2-7B, Llama-3.2-1B, Qwen2.5, and Phi-3, during the fine-tuning process on benign downstream datasets (e.g., Dolly, Alpaca). This vulnerability results from suboptimal optimization configurations—specifically aggressive learning rates, small batch sizes, and insufficient gradient accumulation—which cause the model parameters to diverge from the pre-trained safety optimization landscape (the…

Rethinking safety in llm fine-tuning: An optimization perspective

Evaluated models: GPT-4, GPT-4o, Llama 2 7B +3 more

Automated LLM Fingerprinting

Published Aug 12, 2025Analyzed Dec 9, 2025Source: arXiv

Large Language Models (LLMs) exposed via public APIs are vulnerable to model fingerprinting attacks where an attacker can identify the exact backend model family and version (e.g., distinguishing Mistral-7B-v0.1 from v0.3) by analyzing response patterns. While traditional fingerprinting relies on manual query curation, this vulnerability is exacerbated by Reinforcement Learning (RL) based query optimization. An attacker can train an RL agent (specifically using Proximal Policy Optimization) to…

Attacks and defenses against llm fingerprinting

Evaluated models: Mistral 7B, Qwen 2 5B, Gemma 2 2B +1 more

Magic Token Safety Bypass

Published Aug 12, 2025Analyzed Feb 21, 2026Source: arXiv

The Magic-Token-Guided Co-Training (MTC) framework for Large Language Models (LLMs) introduces a mechanism where distinct behavioral modes are activated via hardcoded system-level strings known as "magic tokens." A specific vulnerability exists in the implementation of the "negative" (neg) behavior mode, which is explicitly trained to generate unfiltered, risk-prone, and harmful content for internal red-teaming. The framework relies on the secrecy of the magic token (e.g., a random string like…

Efficient Switchable Safety Control in LLMs via Magic-Token-Guided Co-Training

Evaluated models: Qwen 3 8B

Back-Translation Watermark Stripping

Published Aug 11, 2025Analyzed Dec 30, 2025Source: arXiv

Implementations of Large Language Model (LLM) watermarking algorithms—specifically KGW (Kirchenbauer et al.), Semantic Invariant Robust (SIR) Watermark, Entropy-based Text Watermarking (EWD), and Unbiased Watermarking—are vulnerable to watermark stripping via adversarial text perturbation. When watermarked text generated by models such as OPT-1.3B is subjected to automated paraphrasing or back-translation (e.g., English $\to$ French $\to$ English), the embedded statistical signals are…

Signature vs. Substance: Evaluating the Balance of Adversarial Resistance and Linguistic Quality in Watermarking Large Language Models

Evaluated models: Llama 3 8B

Thinking Mode Jailbreak Amplification

Published Aug 9, 2025Analyzed Dec 8, 2025Source: arXiv

Large Language Models (LLMs) implementing "Thinking Mode" (also known as Reasoning Mode or Chain-of-Thought) exhibit a heightened susceptibility to jailbreak attacks compared to their non-reasoning counterparts. When a model is prompted to reason step-by-step (often delimited by specific tokens like <think> and </think>), the internal reasoning process frequently overrides safety alignment training. Research indicates that during the generation of the thinking chain, the model often…

The Cost of Thinking: Increased Jailbreak Risk in Large Language Models

Evaluated models: Qwen 3 0.6B, Qwen 3 1.7B, Qwen 3 4B +8 more

Related research categories