Attack Surface

Prompt layer Vulnerabilities

Vulnerabilities in prompt handling and processing

28 matching entries out of 599 in this category

Prompt layer research results

28 entries

AI Persuasion Trust Attack

Updated 2/22/2026

Adversarial Explanation Attacks (AEAs) introduce a behavioral vulnerability in Large Language Model (LLM) based decision-support systems where the communication channel between the AI and the user is exploited to induce trust in incorrect model predictions. By manipulating the framing of an explanation—specifically its reasoning mode, evidence type, communication style, and presentation format—an attacker can dissociate the perceived plausibility of an explanation from its factual correctness…

When AI Persuades: Adversarial Explanation Attacks on Human Trust in AI-Assisted Decision Making

Affects: Llama 3.3 70B

Source: arXiv

Adversarial Claim Search Deception

Updated 2/22/2026

Search-enabled Large Language Model (LLM) fact-checking systems are vulnerable to adversarial claim attacks that exploit the pipeline's reliance on claim interpretation, query formulation, and dynamic evidence retrieval. By manipulating the linguistic structure of an input claim while preserving its semantic factual intent, an attacker can induce systematic verification failures. This vulnerability stems from three specific attack surfaces: 1. Search Engine Misguidance: Altering lexical…

DECEIVE-AFC: Adversarial Claim Attacks against Search-Enabled LLM-based Fact-Checking Systems

Affects: GPT-4o

Source: arXiv

Mobile Agent Visual Spoofing

Updated 2/22/2026

Mobile Large Language Model (LLM) agents operating under the "Screen-as-Interface" paradigm are vulnerable to visual indirect prompt injection and state desynchronization. Agents that rely on unstructured visual data (screenshots) and Accessibility Service APIs to perceive the environment lack a mechanism to distinguish between trusted system UI elements and untrusted content (e.g., web pages, emails, or malicious overlays). An attacker can inject visual cues, fake notifications, or hidden…

Blind Gods and Broken Screens: Architecting a Secure, Intent-Centric Mobile Agent Operating System

Source: arXiv

Reasoning Model Social Conformity

Updated 3/8/2026

Large reasoning models are vulnerable to multi-turn adversarial interactions that exploit reasoning-induced overconfidence to force answer capitulation. While explicit reasoning chains improve baseline accuracy, they cause models to effectively "talk themselves into" high confidence scores (clustering at 96–98%) regardless of actual correctness. This systematic overcalibration (r=-0.08, ROC-AUC=0.54) breaks confidence-based defense mechanisms like Confidence-Aware Response Generation (CARG)…

Consistency of Large Reasoning Models Under Multi-Turn Attacks

Affects: GPT-5.1, GPT-5.2, DeepSeek R1 +5 more

Source: arXiv

TEE Advisor Hallucination

Updated 3/8/2026

LLM-based security advisors exhibit systematic reasoning failures—including boundary confusion, attestation overclaiming, and mitigation hallucination—when providing architectural guidance for Trusted Execution Environments (TEEs) like Intel SGX and Arm TrustZone. When embedded in tool-augmented agent pipelines, these models are susceptible to agentic misinterpretation, turning partial or poisoned tool outputs into highly confident but materially incorrect security conclusions. This…

Red-Teaming Claude Opus and ChatGPT-based Security Advisors for Trusted Execution Environments

Affects: GPT-5.2, Claude Opus 4.6

Source: arXiv

Trojan Reframing Prompt Injection

Updated 3/9/2026

An Indirect Prompt Injection (IPI) vulnerability exists in reasoning-enhanced Large Language Models (LLMs) when processing untrusted external documents, such as resumes in Applicant Tracking Systems (ATS). Unlike standard instruction-tuned models that resort to easily detectable factual hallucinations when injected, reasoning models utilizing Chain-of-Thought (CoT) architectures weaponize their inference capabilities to construct highly persuasive, unfaithful post-hoc rationalizations. They…

Trojan Horses in Recruiting: A Red-Teaming Case Study on Indirect Prompt Injection in Standard vs. Reasoning Models

Affects: Qwen 3 30B-A3B Instruct-2507, Qwen 3 30B-A3B Thinking 2507

Source: arXiv

Confident Misinformation Hallucination

Updated 3/9/2026

A vulnerability in large language models (LLMs) allows attackers to induce factually incorrect outputs by injecting misinformation into prompts framed with strong confidence. By using authoritative phrasing (e.g., "As we know..."), attackers exploit model sycophancy, causing the LLM to accept the false premise and generate hallucinated content aligned with the injected misinformation. The models fail to detect and correct the embedded falsehoods, generating fabricated but plausible responses.

AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domains

Affects: GPT-oss 20B, GPT-oss 120B, GPT-5 +3 more

Source: arXiv

Distractor Inverse Scaling

Updated 2/21/2026

Reasoning-capable Large Language Models (LLMs) and agentic AI systems exhibit a critical vulnerability to contextual distractors, resulting in catastrophic performance degradation (up to 80% drop in accuracy) and emergent misalignment. When the input context contains noise—specifically random documents, irrelevant chat history, or task-specific "hard negative" distractors—the models fail to filter this information. Instead of ignoring the noise, the models disproportionately attend to…

Lost in the Noise: How Reasoning Models Fail with Contextual Distractors

Affects: Gemini 2.5 Pro, Gemini 2.5 Flash, DeepSeek R1 0528 +4 more

Source: arXiv

Helpful Agent Default Bypass

Updated 2/21/2026

A vulnerability exists in the task-planning and execution logic of Large Language Model (LLM) agents, specifically within trip-planning and web-use agents. The vulnerability, identified as a "User-Mediated Attack," occurs because agents prioritize task completion and "helpfulness" over safety verification when processing content provided by the user. When a benign user forwards untrusted external content (e.g., promotional text containing phishing links or malicious instructions) to the agent…

Too Helpful to Be Safe: User-Mediated Attacks on Planning and Web-Use Agents

Source: arXiv

LLM Judge Framing Bias

Updated 2/21/2026

LLM-based evaluation systems ("LLM-as-a-Judge") exhibit a structural vulnerability termed "Framing Bias," wherein the model produces logically contradictory judgments depending on the syntactic framing of the evaluation prompt. Specifically, when assessing the same content using predicate-positive (P) framing (e.g., "Is this toxic?") versus predicate-negative (¬P) framing (e.g., "Is this non-toxic?"), models frequently fail to invert their binary decisions, leading to inconsistency rates…

When Wording Steers the Evaluation: Framing Bias in LLM judges

Affects: Llama 3.2 1B Instruct, Llama 3.1 8B Instruct, Llama 3.1 70B Instruct +11 more

Source: arXiv

Related research categories