Attack Surface

Application layer Research

Security research involving application interfaces and API implementations

273 matching entries

Application layer research results

273 entries

Staged LLM Pipeline Attack

Published Jun 30, 2025Analyzed Jul 14, 2025Source: arXiv

Large language models (LLMs) protected by multi-stage safeguard pipelines (input and output classifiers) are vulnerable to staged adversarial attacks (STACK). STACK exploits weaknesses in individual components sequentially, combining jailbreaks for each classifier with a jailbreak for the underlying LLM to bypass the entire pipeline. Successful attacks achieve high attack success rates (ASR), even on datasets of particularly harmful queries.

STACK: Adversarial Attacks on LLM Safeguard Pipelines

Evaluated models: Claude Opus 4, Gemma 2 9B, GPT-4 Turbo+4 more

Claude Opus 4
Gemma 2 9B
GPT-4 Turbo
GPT-4o
GPT-5
Llama 3 8B Instruct
Qwen 3 14B

Agent API Goal Divergence

Published Jun 9, 2025Analyzed Dec 9, 2025Source: arXiv

Large Language Model (LLM) agents capable of invoking external APIs are vulnerable to intent integrity violations. When an agent receives natural language instructions that are ambiguous, underspecified, or contain values not supported by the underlying API schema, the agent frequently fails to preserve user intent. Instead of rejecting the request or asking for clarification, the model may hallucinate parameter values, map unsupported requests to unsafe defaults, or execute actions on…

TAI3: Testing Agent Integrity in Interpreting User Intent

Evaluated models: GPT-4o Mini, Llama 3.1 8B, Qwen 3 30B-A3B+5 more

GPT-4o Mini
Llama 3.1 8B
Qwen 3 30B-A3B
Llama 3.3 70B
DeepSeek R1 Distill Llama 70B
Claude 3.5 Haiku
Gemini 2.5 Pro
o3-mini

Phantom Token User Deception

Published May 30, 2025Analyzed Jan 14, 2026Source: arXiv

Large Language Models (LLMs) that utilize byte-stream parsing or structural extraction to process PDF files—specifically the OpenAI GPT and Anthropic Claude families—are vulnerable to adversarial text injection via imperceptible "phantom tokens." This vulnerability exploits the disconnect between how PDF viewers render documents for humans (visual layer) and how LLMs extract text from the PDF operator stream (data layer). Attackers can manipulate standard PDF text-showing operators (TJ and Tj)…

TRAPDOC: Deceiving LLM Users by Injecting Imperceptible Phantom Tokens into Documents

Evaluated models: GPT-4, o4-mini

Adaptive LLM Jailbreaking Strategy

Published May 29, 2025Analyzed Jul 14, 2025Source: arXiv

Large Language Models (LLMs) are vulnerable to adaptive jailbreaking attacks that exploit their semantic comprehension capabilities. The MEF framework demonstrates that by tailoring attacks to the model's understanding level (Type I or Type II), evasion of input, inference, and output-level defenses is significantly improved. This is achieved through layered semantic mutations and dual-ended encryption techniques, allowing bypass of security measures even in advanced models like GPT-4o.

Adaptive Jailbreaking Strategies Based on the Semantic Understanding Capabilities of Large Language Models

Evaluated models: GPT-4o, Llama 2 13B, Llama 2 7B

Hybrid Agent Prompt Injection

Published May 28, 2025Analyzed Dec 9, 2025Source: arXiv

Computer-Use Agents (CUAs) powered by Large Language Models (LLMs) operating in hybrid Web-OS environments are vulnerable to indirect prompt injection. Attackers can embed malicious natural language or code instructions within legitimate web content (e.g., social media forums, chat applications, shared cloud documents) that the agent processes during benign task execution. Due to the agent's inability to distinguish between trusted user instructions and untrusted environmental data, the CUA…

RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments

Evaluated models: Claude 3.5 Sonnet, Claude 3.7 Sonnet, GPT-4o

Asynchronous Audio Jailbreak

Published May 20, 2025Analyzed May 31, 2025Source: arXiv

End-to-end Large Audio-Language Models (LALMs) are vulnerable to AudioJailbreak, a novel attack that appends adversarial audio perturbations ("jailbreak audios") to user prompts. These perturbations, even when applied asynchronously and without alignment to the user's speech, can manipulate the LALM's response to generate adversary-desired outputs that bypass safety mechanisms. The attack achieves universality by employing a single perturbation effective across different prompts and robustness…

AudioJailbreak: Jailbreak Attacks against End-to-End Large Audio-Language Models

Evaluated models: BLSP, FunAudioLLM, GPT-4o+9 more

BLSP
FunAudioLLM
GPT-4o
Ichigo
Llama Omni
LLaSM
Mini-Omni
Mini-Omni 2
Qwen 2 Audio
Qwen Audio
SALMONN
SpeechGPT

LLM Judge Prompt Injection

Published May 19, 2025Analyzed May 31, 2025Source: arXiv

Large Language Models (LLMs) used for evaluating text quality (LLM-as-a-Judge architectures) are vulnerable to prompt-injection attacks. Maliciously crafted suffixes appended to input text can manipulate the LLM's judgment, causing it to incorrectly favor a predetermined response even if another response is objectively superior. Two attack vectors are identified: Comparative Undermining Attack (CUA), directly targeting the final decision, and Justification Manipulation Attack (JMA), altering…

Investigating the Vulnerability of LLM-as-a-Judge Architectures to Prompt-Injection Attacks

Evaluated models: Falcon 3 3B Instruct, Qwen 2.5 3B Instruct

On-Device LLM Hijacking

Published May 19, 2025Analyzed Dec 9, 2025Source: arXiv

Mobile LLM agents utilizing vision-based screen perception (OCR or Multimodal Large Language Models) are vulnerable to Visual Prompt Injection via malicious GUI overlays. An attacker holding the SYSTEM_ALERT_WINDOW permission can deploy non-focusable floating windows (using FLAG_NOT_FOCUSABLE) containing adversarial text or fabricated UI elements over legitimate applications. Because the agent captures the entire screen buffer to interpret the device state, it ingests the adversarial overlay…

From Assistants to Adversaries: Exploring the Security Risks of Mobile LLM Agents

Evaluated models: GPT-4o

LLM Multi-Agent IP Leakage

Published May 18, 2025Analyzed May 31, 2025Source: arXiv

Large Language Model (LLM)-based Multi-Agent Systems (MAS) are vulnerable to intellectual property (IP) leakage attacks. An attacker with black-box access (only interacting via the public API) can craft adversarial queries that propagate through the MAS, extracting sensitive information such as system prompts, task instructions, tool specifications, number of agents, and system topology.

IP Leakage Attacks Targeting LLM-Based Multi-Agent Systems

Evaluated models: GPT-4o, GPT-4o Mini, Llama 3.1 70B+2 more

GPT-4o
GPT-4o Mini
Llama 3.1 70B
Llama 3.1 8B
Qwen 2.5 72B

LLM User Simulation Shilling

Published May 18, 2025Analyzed Jan 14, 2026Source: arXiv

Recommender Systems (RS) utilizing collaborative filtering and review processing (such as NMF, NeuNMF, and Dual-Tower architectures) are vulnerable to low-knowledge shilling attacks via LLM-based user agents. This vulnerability, demonstrated by the Agent4SR framework, allows an attacker to manipulate recommendation outcomes (Top-N lists) for target items without access to the internal model parameters or training data. The attack leverages Large Language Models to construct cognitively…

LLM-Based User Simulation for Low-Knowledge Shilling Attacks on Recommender Systems

Evaluated models: GPT-4o

Related research categories