Attack Type

Hallucination Research

Exploits causing models to generate false information

53 matching entries

Hallucination research results

53 entries

Selective refusal gaps in visual question answering

Published Sep 4, 2026Analyzed Sep 9, 2026Source: arXiv

KoNA measures whether vision-language models answer valid image questions while refusing unsafe components or correcting unsupported premises. Its 9,300 question-answer pairs include mixed and fully answerable controls.

Knowing What Not to Answer: Selective Non-Compliance in Vision-Language Models

Evaluated models: InternVL3 2B Instruct, InternVL3-78B-Instruct, Qwen 2.5 VL 3B Instruct+5 more

InternVL3 2B Instruct
InternVL3-78B-Instruct
Qwen 2.5 VL 3B Instruct
Qwen 2.5 VL 72B Instruct
GPT-5
Gemini 2.5 Flash
InternVL3-2B-KoNA
Qwen2.5-VL-3B-KoNA

Cascading Agent False Consensus

Published Mar 4, 2026Analyzed Mar 9, 2026Source: arXiv

Multi-Agent Systems based on Large Language Models (LLM-MAS) are vulnerable to systemic Consensus Corruption via cascading error amplification. Because mainstream collaborative architectures rely on recursive context reuse without atomic-level provenance tracking, a single atomic falsehood injected into the system is repeatedly cited and reused within the multi-agent interaction chain. This structural exposure causes the error to deterministically compound across the communication graph…

From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration

Evaluated models: GPT-4o

TEE Advisor Hallucination

Published Feb 23, 2026Analyzed Mar 8, 2026Source: arXiv

LLM-based security advisors exhibit systematic reasoning failures—including boundary confusion, attestation overclaiming, and mitigation hallucination—when providing architectural guidance for Trusted Execution Environments (TEEs) like Intel SGX and Arm TrustZone. When embedded in tool-augmented agent pipelines, these models are susceptible to agentic misinterpretation, turning partial or poisoned tool outputs into highly confident but materially incorrect security conclusions. This…

Red-Teaming Claude Opus and ChatGPT-based Security Advisors for Trusted Execution Environments

Evaluated models: GPT-5.2, Claude Opus 4.6

Trojan Reframing Prompt Injection

Published Feb 19, 2026Analyzed Mar 9, 2026Source: arXiv

An Indirect Prompt Injection (IPI) vulnerability exists in reasoning-enhanced Large Language Models (LLMs) when processing untrusted external documents, such as resumes in Applicant Tracking Systems (ATS). Unlike standard instruction-tuned models that resort to easily detectable factual hallucinations when injected, reasoning models utilizing Chain-of-Thought (CoT) architectures weaponize their inference capabilities to construct highly persuasive, unfaithful post-hoc rationalizations. They…

Trojan Horses in Recruiting: A Red-Teaming Case Study on Indirect Prompt Injection in Standard vs. Reasoning Models

Evaluated models: Qwen 3 30B-A3B Instruct-2507, Qwen 3 30B-A3B Thinking 2507

Reasoning Model Social Conformity

Published Feb 13, 2026Analyzed Mar 8, 2026Source: arXiv

Large reasoning models are vulnerable to multi-turn adversarial interactions that exploit reasoning-induced overconfidence to force answer capitulation. While explicit reasoning chains improve baseline accuracy, they cause models to effectively "talk themselves into" high confidence scores (clustering at 96–98%) regardless of actual correctness. This systematic overcalibration (r=-0.08, ROC-AUC=0.54) breaks confidence-based defense mechanisms like Confidence-Aware Response Generation (CARG)…

Consistency of Large Reasoning Models Under Multi-Turn Attacks

Evaluated models: GPT-5.1, GPT-5.2, DeepSeek R1+5 more

GPT-5.1
GPT-5.2
DeepSeek R1
Grok 4.1
Grok 3
Gemini 2.5 Pro
GPT-oss 120B
GPT-4o

Mobile Agent Visual Spoofing

Published Feb 11, 2026Analyzed Feb 22, 2026Source: arXiv

Mobile Large Language Model (LLM) agents operating under the "Screen-as-Interface" paradigm are vulnerable to visual indirect prompt injection and state desynchronization. Agents that rely on unstructured visual data (screenshots) and Accessibility Service APIs to perceive the environment lack a mechanism to distinguish between trusted system UI elements and untrusted content (e.g., web pages, emails, or malicious overlays). An attacker can inject visual cues, fake notifications, or hidden…

Blind Gods and Broken Screens: Architecting a Secure, Intent-Centric Mobile Agent Operating System

Evaluated models: Not reported

LLM Scaling Fidelity Paradox

Published Feb 10, 2026Analyzed Feb 21, 2026Source: arXiv

A vulnerability exists in Large Language Model (LLM) context compression architectures (specifically compressor-decoder setups) characterized as the "Size-Fidelity Paradox." When scaling up the parameter count of the compressor model (e.g., beyond 4B parameters in Qwen-3 and LLaMA-3.2 families), the system exhibits a degradation in faithful preservation of the source text, despite improvements in standard training loss and perplexity metrics. This degradation manifests through two primary…

When Less is More: The LLM Scaling Paradox in Context Compression

Evaluated models: Qwen 3 8B, Qwen 3 32B, Llama 3.2 11B+1 more

Qwen 3 8B
Qwen 3 32B
Llama 3.2 11B
Llama 3.2 90B

Self-Evolution Alignment Blindspots

Published Feb 10, 2026Analyzed Mar 8, 2026Source: arXiv

Closed-loop, self-evolving Large Language Model (LLM) multi-agent systems (MAS) are vulnerable to irreversible safety erosion and alignment failure. When agents recursively optimize and update their policies using only synthetic data derived from internal interactions—without continuous external human grounding—the system naturally minimizes interaction energy and optimizes for internal conversational consistency. This isolation causes a progressive drift away from initial anthropic safety…

The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societies

Evaluated models: GPT-3.5 Turbo, Qwen 3 8B

Context-Robust RAG Poisoning

Published Feb 6, 2026Analyzed Feb 22, 2026Source: arXiv

Retrieval-Augmented Generation (RAG) systems are vulnerable to a robust corpus poisoning attack known as "Confundo." This vulnerability arises from the lack of pipeline awareness in standard RAG implementations, specifically regarding document ingestion (tokenization and chunking) and query variations. An attacker can exploit this by fine-tuning a Large Language Model (LLM) to function as a poison generator. Unlike traditional adversarial examples which are brittle, Confundo generates poison…

Confundo: Learning to Generate Robust Poison for Practical RAG Systems

Evaluated models: Llama 3 8B, Gemini Pro

Chunky Post-Training Miscalibration

Published Feb 5, 2026Analyzed Feb 21, 2026Source: arXiv

Large Language Models (LLMs) exhibit a vulnerability termed "chunky post-training," where the model learns spurious correlations between incidental prompt features (e.g., formatting styles, specific vocabulary, sentence structure) and specific behavioral modes (e.g., refusal, code generation, rebuttal) present in distinct chunks of post-training data. This results in behavioral mis-routing during inference, where benign inputs sharing surface-level features with restricted or specialized…

Chunky Post-Training: Data Driven Failures of Generalization

Evaluated models: Claude Haiku 4.5, Claude Sonnet 4.5, Claude Opus 4.5+5 more

Claude Haiku 4.5
Claude Sonnet 4.5
Claude Opus 4.5
GPT-5.1
Grok 4.1 Mini
Tülu 3 8B
Tülu 3 70B
Tülu 3 405B

Related research categories