Attack Surface

Application layer Research

Security research involving application interfaces and API implementations

273 matching entries

Application layer research results

273 entries

Controlled-Release Guard Bypass

Published Oct 2, 2025Analyzed Dec 30, 2025Source: arXiv

A vulnerability termed "Controlled-Release Prompting" allows attackers to bypass lightweight input filters (prompt guards) deployed in front of Large Language Models (LLMs). The attack exploits the computational resource asymmetry between the resource-constrained guard model and the highly capable target model. Attackers encode malicious instructions using obfuscation techniques—such as substitution ciphers (Timed-Release) or verbose character descriptions (Spaced-Release)—that require…

Bypassing Prompt Guards in Production with Controlled-Release Prompting

Evaluated models: Gemini 2.5 Flash, Gemini 2.5 Pro, DeepSeek R1+2 more

Gemini 2.5 Flash
Gemini 2.5 Pro
DeepSeek R1
Grok 3
GPT-5 Mini

Code Agent Executable Jailbreaks

Published Oct 1, 2025Analyzed Oct 13, 2025Source: arXiv

AI code agents are vulnerable to jailbreaking attacks that cause them to generate or complete malicious code. The vulnerability is significantly amplified when a base Large Language Model (LLM) is integrated into an agentic framework that uses multi-step planning and tool-use. Initial safety refusals by the LLM are frequently overturned during subsequent planning or self-correction steps within the agent's reasoning loop.

Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks

Evaluated models: Claude 3.7 Sonnet, DeepSeek R1, Dolphin Mistral 24B Venice+6 more

Claude 3.7 Sonnet
DeepSeek R1
Dolphin Mistral 24B Venice
GPT-4.1
Llama 3 8B
Llama 3.1 70B
Mistral Large 2.1
o1
Qwen 3 235B-A22B

Chained Tool-Use Injections

Published Sep 30, 2025Analyzed Oct 13, 2025Source: arXiv

A vulnerability exists in tool-enabled Large Language Model (LLM) agents, termed Sequential Tool Attack Chaining (STAC), where a sequence of individually benign tool calls can be orchestrated to achieve a malicious outcome. An attacker can guide an agent through a multi-turn interaction, with each step appearing harmless in isolation. Safety mechanisms that evaluate individual prompts or actions fail to detect the threat because the malicious intent is distributed across the sequence and only…

STAC: When Innocent Tools Form Dangerous Chains to Jailbreak LLM Agents

Evaluated models: GPT-4.1, GPT-4.1 Mini, Llama 3.1 405B Instruct+4 more

GPT-4.1
GPT-4.1 Mini
Llama 3.1 405B Instruct
Llama 3.3 70B Instruct
Mistral Large
Mistral Small
Qwen 3 32B

Self-Amplifying Memory Poisoning

Published Sep 29, 2025Analyzed Jan 14, 2026Source: arXiv

Large Language Model (LLM) agents utilizing long-term memory or Retrieval-Augmented Generation (RAG) are vulnerable to context-dependent memory injection attacks. Unlike traditional prompt injections that are overtly malicious, this vulnerability involves injecting records that appear benign and coherent in isolation—thereby bypassing standard perplexity filters and static content moderation (e.g., LlamaGuard). These records contain "sleeping" malicious logic that is only activated when…

A-memguard: A proactive defense framework for llm-based agent memory

Evaluated models: GPT-4o, Llama 3.1 8B

Search Agents Vulnerable to Unreliable Results

Published Sep 28, 2025Analyzed Oct 13, 2025Source: arXiv

LLM-based search agents are vulnerable to manipulation via unreliable search results. An attacker can craft a website containing malicious content (e.g., misinformation, harmful instructions, or indirect prompt injections) that is indexed by search engines. When an agent retrieves and processes this page in response to a benign user query, it may uncritically accept the malicious content as factual and incorporate it into its final response. This allows the agent to be used as a vector for…

SafeSearch: Automated Red-Teaming for the Safety of LLM-Based Search Agents

Evaluated models: DeepSeek R1, Gemini 2.5 Flash, Gemini 2.5 Pro+11 more

DeepSeek R1
Gemini 2.5 Flash
Gemini 2.5 Pro
Gemma 3 27B IT
GPT-4.1
GPT-4.1 Mini
GPT-5
GPT-5 Mini
GPT-oss 120B
Kimi K2
o4-mini
Qwen 3 235B-A22B
Qwen 3 32B
Qwen 3 8B

Emergent Agentic Vulnerabilities

Published Sep 21, 2025Analyzed Jan 14, 2026Source: arXiv

GPT-OSS-20B exhibits "agentic-only" vulnerabilities where safety guardrails effective in standalone model inference fail when the model operates within an agentic execution loop. These vulnerabilities emerge when the model is deployed in a multi-step agentic architecture (e.g., utilizing LangGraph, tool usage, and memory retention). Attackers can bypass safety filters by employing context-aware iterative refinement attacks, which incorporate the full agentic state—including tool outputs…

Mind the Gap: Comparing Model-vs Agentic-Level Red Teaming with Action-Graph Observability on GPT-OSS-20B

Evaluated models: Not reported

Multi-Agent Compositional Leak

Published Sep 16, 2025Analyzed Jan 14, 2026Source: arXiv

Multi-agent Large Language Model (LLM) systems are vulnerable to compositional privacy leakage, a flaw where sensitive information is exposed through the aggregation of individually benign responses from distinct agents. In distributed architectures where data is siloed (e.g., distinct agents handling HR, Finance, and IT logs), individual agents lack a global view of the user’s accumulated knowledge or the sensitive attributes derivable from cross-agent data combinations. An attacker can…

The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration

Evaluated models: Qwen 3 32B, Gemini 2.5 Pro, GPT-5

GUI Agent Dark Pattern Blindness

Published Sep 12, 2025Analyzed Dec 9, 2025Source: arXiv

Large Language Model (LLM)-powered GUI agents exhibit a vulnerability to deceptive interface designs (dark patterns) due to goal-driven optimization and procedural myopia. When executing natural language instructions on web interfaces, these agents consistently prioritize minimizing steps and achieving task completion over user safety or privacy. Agents frequently recognize manipulative elements—such as pre-selected consent checkboxes, hidden costs, or trick questions—in their internal…

Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight

Evaluated models: GPT-4o, Claude 3.7 Sonnet, DeepSeek V3+1 more

GPT-4o
Claude 3.7 Sonnet
DeepSeek V3
Gemini 2.0 Flash

Typos Undermine Watermarks

Published Sep 11, 2025Analyzed Feb 22, 2026Source: arXiv

Large Language Model (LLM) inference-time watermarking schemes are vulnerable to evasion via character-level perturbations that disrupt the model's tokenizer. Unlike token-level attacks (e.g., synonym replacement), character-level edits—such as homoglyph substitutions, zero-width character insertions, and typos—force the tokenizer to segment a single semantic unit into multiple sub-word tokens. This fragmentation alters the context window used by the watermarking hashing function (e.g., the…

Character-Level Perturbations Disrupt LLM Watermarks

Evaluated models: Llama 3 8B

Multimodal Prompt Injection

Published Sep 7, 2025Analyzed Dec 9, 2025Source: arXiv

Large Language Models (LLMs), including GPT-4o, LLaMA-3, and GPT-3.5-Turbo, are vulnerable to multimodal prompt injection attacks. These models fail to distinguish between system-level instructions and user-provided content within the context window. Attackers can exploit this by embedding malicious instructions in direct text, indirect sources (such as third-party webpages or PDFs), or visual inputs (images). Successful exploitation results in the model prioritizing the injected adversarial…

Multimodal Prompt Injection Attacks: Risks and Defenses for Modern LLMs

Evaluated models: GPT-3.5, GPT-4o, Llama 3 8B+1 more

GPT-3.5
GPT-4o
Llama 3 8B
Mistral Large 24B

Related research categories