Skip to main content
LLM Security Database
Skip to research search
Last analyzed Sep 9, 2026

Language Model Security Database

985 research findings · 1119 evaluated models

Research findings

985 entries

Matches every word across titles, descriptions, sources, affected systems, and models.

Agentic Binary Reverse-Engineering Capability Gaps

Published Analyzed Source: arXiv

SRE-Bench evaluates whether cybersecurity agents can recover the behavior of realistic binary-only software without relying on source-code memorization. The authors construct 19 private programs, 44 anti-analysis primitives, 262 binary instances, and 1,572 deterministic grading tasks covering security-relevant reverse-engineering scenarios.

The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark
Evaluated models: GPT-5.6 Sol, Claude-Opus-5, GPT-5.5+2 more
  • GPT-5.6 Sol
  • Claude-Opus-5
  • GPT-5.5
  • Grok 4.5
  • GLM 5.2

Historical reasoning-envelope isolation failures

Published Analyzed Source: arXiv

The authors report historical isolation failures in client-held encrypted reasoning blocks across compatible provider API contexts.

Stealing Reasoning Traces from Proprietary LLM APIs
Evaluated models: Fable 5, Opus 4.8, Sonnet 5+15 more
  • Fable 5
  • Opus 4.8
  • Sonnet 5
  • Sonnet 4.6
  • Sonnet 4.5
  • Haiku 4.5
  • GPT-5.6 Sol
  • GPT-5.6-terra
  • GPT-5.6 Luna
  • GPT-5
  • GPT-5 Mini
  • o4-mini
  • Gemini 3.1 Pro
  • Gemini 3 Pro
  • Gemini Robotics 1.6
  • Gemini 3.5 Flash
  • Gemini 3 Flash
  • Gemini 3.1 Flash Lite

Trajectory Poisoning in Self-Evolving Agent Skills

Published Analyzed Source: arXiv

Self-evolving agent-skill systems may promote recurring, attacker-controlled execution records into persistent reusable instructions. The paper evaluates whether poisoned but task-plausible trajectories survive aggregation and appear in generated skill artifacts, using inert canary behaviors and two structurally different skill-evolution pipelines.

When Experience Becomes Instruction: Trajectory Poisoning in Self-Evolving Agent Skill Systems
Evaluated models: GPT-5.4, MiniMax M2.5, DeepSeek V3.2+3 more
  • GPT-5.4
  • MiniMax M2.5
  • DeepSeek V3.2
  • DeepSeek V4-Pro
  • Qwen3.5-35B-A3B
  • Qwen3.5-122B-A10B

Dynamic Security Testing for Third-Party Agent Skills

Published Analyzed Source: arXiv

SkillSentry evaluates third-party agent skills by constructing source-grounded decoy environments and comparing matched executions with and without the tested skill. The method requires completed, observable, skill-attributed side effects rather than treating suspicious text, ordinary privileged operations, or unexecuted paths as proven malicious behavior.

SkillSentry: Adaptive Honey Worlds for Dynamic Safety Testing of Agent Skills
Evaluated models: DeepSeek V4-Pro

Backdoored PEFT adapters and limits of weight-only detection

Published Analyzed Source: arXiv

Third-party fine-tuning adapters may contain backdoors. Z-PEFT screens adapter weights using spectral features, evaluated on PADBench's 13,300 adapters.

Z-PEFT: Zero-shot Backdoor Detection in Parameter-Efficient Fine-Tuning via Canonical Spectral Signatures
Evaluated models: Llama 2 7B, Llama 2 13B, Qwen 1.5 7B+2 more
  • Llama 2 7B
  • Llama 2 13B
  • Qwen 1.5 7B
  • Flan-T5 XL
  • RoBERTa Base

Repository-Level Coding Agent Vulnerability Detection Gaps

Published Analyzed Source: arXiv

VulnGym measures whether coding agents can locate and explain repository-level security vulnerabilities from realistic advisory and source-code context. The benchmark contains 184 reviewed advisories, 408 line-annotated vulnerability entries, and 23 repositories, with separate end-to-end detection and oracle-conditioned localization tasks.

VulnGym: Benchmarking Coding Agents for Repository-Level Vulnerability Detection
Evaluated models: DeepSeek V4 Flash, GLM 5.2, MiniMax-M3+4 more
  • DeepSeek V4 Flash
  • GLM 5.2
  • MiniMax-M3
  • Qwen3.5-2B
  • Qwen3.5-4B
  • Qwen3.5-9B
  • Qwen3.5-27B

Persistent Agent Memory Poisoning and Incomplete Repair

Published Analyzed Source: arXiv

MemSecBench follows malicious agent-memory content from initial write through persistence, retrieval, action selection, execution, and attempted selective repair. Its controlled Write–Execute–Forget protocol evaluates 310 human-reviewed cases across two harnesses, four memory backends, three model backends, and seven evidence-gated lifecycle checkpoints.

MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair
Evaluated models: GPT-5.5, DeepSeek V4-Pro, MiniMax-M3

Workspace Agent Runtime Safety Failures Across Risk Carriers

Published Analyzed Source: arXiv

AgentS4D measures unsafe actions and state changes across complete workspace-agent executions rather than treating task completion or isolated model responses as safety evidence. Its 328 sandboxed cases introduce risky content through user requests, documents, web resources, tools, third-party skills, and persistent memory, then compare the same cases across four agent harnesses and five model backends.

AgentS4D: Benchmarking Runtime Risks across the Execution Lifecycle of LLM-Based Workspace Agents
Evaluated models: GPT-5.5, Gemini 3.1 Pro, DeepSeek V4-Pro+2 more
  • GPT-5.5
  • Gemini 3.1 Pro
  • DeepSeek V4-Pro
  • MiniMax-M3
  • Qwen3.7-Plus

Research methodology

Entries summarize publicly available primary-source security research. Model names reflect only systems explicitly evaluated by the cited paper, and measurements are research-reported unless independent verification is stated.

Discovery and publication are separate: new papers enter a candidate queue, then require human review and explicit approval before publication. Updates follow that review process, not a guaranteed daily publishing schedule. “Last analyzed” is the newest analysis date among published entries; it does not mean that all recent research has been covered.