Language Model Security Database

Large Language Models (LLMs) utilized for static code analysis, code review, and autonomous software engineering exhibit a cognitive vulnerability termed "Abstraction Bias." When processing code that structurally resembles common algorithmic patterns (e.g., standard sorting algorithms, helper functions, or mathematical formulas), the model relies on high-level memorized representations of the algorithm's intent rather than analyzing the specific local logic. Adversaries can exploit this by…

A Time-of-Check to Time-of-Use (TOCTOU) vulnerability exists in LLM-enabled agentic systems that execute multi-step plans involving sequential tool calls. The vulnerability arises because plans are not executed atomically. An agent may perform a "check" operation (e.g., reading a file, checking a permission) in one tool call, and a subsequent "use" operation (e.g., writing to the file, performing a privileged action) in another tool call. A temporal gap between these calls, often used for LLM…

Multimodal Entity Linking (MEL) systems, encompassing both traditional dual-encoder models and Multimodal Large Language Models (MLLMs), are vulnerable to gradient-based white-box adversarial attacks. By applying imperceptible perturbations to visual inputs via Projected Gradient Descent (PGD), Auto-PGD (APGD), or Carlini & Wagner (CW) methods, an attacker can manipulate the visual embeddings generated by the model. This manipulation disrupts the cross-modal alignment structure, causing the…

Large Language Models (LLMs), including Llama 2, Mistral, and Vicuna, are susceptible to a white-box adversarial attack that circumvents safety alignment mechanisms (such as RLHF). The vulnerability exists due to the models' susceptibility to intrinsic optimization of adversarial suffixes using Exponentiated Gradient Descent (EGD). Unlike previous methods that rely on inefficient discrete token searches (e.g., Greedy Coordinate Gradient) or standard projected gradient descent, this attack…

Large Language Models (LLMs) are vulnerable to an adaptive black-box jailbreaking framework named MAJIC (Markovian Adaptive Jailbreaking via Iterative Composition). This vulnerability allows an attacker to bypass safety alignment mechanisms by modeling the selection and composition of prompt disguise strategies as a Markov chain. Unlike static attacks, MAJIC initializes a transition matrix using a proxy model to determine the probability of a specific strategy succeeding after a prior strategy…

A universal prompt injection vulnerability, termed "Involuntary Jailbreak," affects multiple large language models. The attack uses a single prompt that instructs the model to learn a pattern from abstract string operators (X and Y). The model is then asked to generate its own examples of questions that should be refused (harmful questions) and provide detailed, non-refusal answers to them, in order to satisfy the learned operator logic. This reframes the generation of harmful content as a…

Safety alignment degradation occurs in instruction-tuned Large Language Models (LLMs), specifically Llama-2-7B, Llama-3.2-1B, Qwen2.5, and Phi-3, during the fine-tuning process on benign downstream datasets (e.g., Dolly, Alpaca). This vulnerability results from suboptimal optimization configurations—specifically aggressive learning rates, small batch sizes, and insufficient gradient accumulation—which cause the model parameters to diverge from the pre-trained safety optimization landscape (the…

Large language models that support a developer role in their API are vulnerable to a jailbreaking attack that leverages malicious developer messages. An attacker can craft a developer message that overrides the model's safety alignment by setting a permissive persona, providing explicit instructions to bypass refusals, and using few-shot examples of harmful query-response pairs. This technique, named D-Attack, is effective on its own. A more advanced variant, DH-CoT, enhances the attack by…

Large Language Models (LLMs) exposed via public APIs are vulnerable to model fingerprinting attacks where an attacker can identify the exact backend model family and version (e.g., distinguishing Mistral-7B-v0.1 from v0.3) by analyzing response patterns. While traditional fingerprinting relies on manual query curation, this vulnerability is exacerbated by Reinforcement Learning (RL) based query optimization. An attacker can train an RL agent (specifically using Proximal Policy Optimization) to…

The Magic-Token-Guided Co-Training (MTC) framework for Large Language Models (LLMs) introduces a mechanism where distinct behavioral modes are activated via hardcoded system-level strings known as "magic tokens." A specific vulnerability exists in the implementation of the "negative" (neg) behavior mode, which is explicitly trained to generate unfiltered, risk-prone, and harmful content for internal red-teaming. The framework relies on the secrecy of the magic token (e.g., a random string like…