Impact
Security research involving model-output integrity and reliability
10 matching entries out of 539 in this category
Lower-trust tool content can assert facts beyond its authority and distort an agent's decisions. PIPES screens response units against source provenance and expected field meaning.
Source: arXiv
OpenClaw is vulnerable to persistent memory poisoning, allowing an attacker to manipulate the agent's long-term memory store (MEMORY.md) via prompt injection. Because the autonomous agent continuously integrates this memory file as context for all subsequent reasoning and task planning, injected payloads act as durable behavioral constraints. This allows an attacker to persistently alter the agent's core policy, manipulate tool selection, and hijack future sessions without any further…
Source: arXiv
The Model Context Protocol (MCP) architecture lacks a semantic verification mechanism to enforce consistency between a tool's documented behavior (exposed to the Large Language Model via JSON schemas) and its actual executable logic. This design gap allows MCP Servers to present benign, read-only, or limited-scope descriptions to the LLM agent while implementing undocumented, privileged, or state-mutating functionality in the underlying code. An attacker can exploit this description–code…
Source: arXiv
Large Language Models (LLMs) enabled with Function Calling (FC) capabilities are vulnerable to adversarial query rewriting and semantic manipulation. Standard FC models, typically trained via Supervised Fine-Tuning (SFT) on static datasets, fail to generalize against adversarial inputs that deviate from fixed distribution patterns. An attacker can exploit this by crafting queries that are semantically similar to valid requests but engineered to induce "bad cases," such as incorrect tool…
Source: arXiv
Large Language Models (LLMs), specifically GPT-4o, GPT-4o-mini, LLaMA-2-13B, Mistral-7B, and Phi-3.5-mini, are vulnerable to Man-in-the-Middle (MitM) adversarial prompt injections that undermine factual recall. Termed the "$\chi$mera" (Chimera) attack framework, this vulnerability exists when an attacker intercepts and modifies user queries (e.g., via malicious browser extensions, compromised frontends, or proxy middleware) before they reach the victim model. By appending adversarial…
Source: arXiv
A vulnerability exists in Large Language Models (LLMs) and multi-label text classification systems that allows for Textual Dynamic Outputs Attacks (TDOA). This technique enables hard-label black-box attacks against systems with variable or generative output spaces (where the number of labels or specific label tokens are not fixed). The attack functions by training a surrogate model on clustered coarse-grained labels derived from the victim model's fine-grained dynamic outputs. It subsequently…
Source: arXiv
Large Language Model (LLM) agents capable of invoking external APIs are vulnerable to intent integrity violations. When an agent receives natural language instructions that are ambiguous, underspecified, or contain values not supported by the underlying API schema, the agent frequently fails to preserve user intent. Instead of rejecting the request or asking for clarification, the model may hallucinate parameter values, map unsupported requests to unsafe defaults, or execute actions on…
Source: arXiv
Alibaba Cloud PAI-Judge and PAI-Judge-Plus are vulnerable to a composite adversarial attack that exploits attention mechanism limitations in Large Language Models (LLMs). An authenticated attacker can manipulate automated evaluation outcomes by appending a long, irrelevant text suffix (approximately 1000 to 2000+ characters) to a response containing adversarial perturbations. This "long-suffix" strategy overwhelms the judge model's context window, causing the attention mechanism to degrade and…
Source: arXiv
A vulnerability exists in Large Language Model (LLM) routing systems (control planes) that allows for the manipulation of inference flow via adversarial input sequences. LLM routers, which dynamically direct user queries to either "weak" (cheaper) or "strong" (expensive) models based on predicted query complexity, can be bypassed by appending specific, pre-optimized token sequences known as "confounder gadgets." These gadgets artificially inflate the router's complexity score for an input…
Source: arXiv
A vulnerability exists in Large Language Model (LLM)-based time series forecasting architectures, specifically affecting models such as TimeGPT, LLMTime, and TimeLLM. These models are susceptible to a gradient-free, black-box adversarial attack method termed Directional Gradient Approximation (DGA). An attacker can inject imperceptible perturbations into the historical time series input window (lookback window) to manipulate the model's output. By treating the model as a black box and…
Source: arXiv