Impact

Integrity Research

Security research involving model-output integrity and reliability

530 matching entries

Integrity research results

530 entries

Trajectory Poisoning in Self-Evolving Agent Skills

Published 8/6/2026

Analyzed 8/13/2026

Self-evolving agent-skill systems may promote recurring, attacker-controlled execution records into persistent reusable instructions. The paper evaluates whether poisoned but task-plausible trajectories survive aggregation and appear in generated skill artifacts, using inert canary behaviors and two structurally different skill-evolution pipelines.

When Experience Becomes Instruction: Trajectory Poisoning in Self-Evolving Agent Skill Systems

Evaluated models: GPT-5.4, MiniMax M2.5, DeepSeek V3.2 +3 more

Source: arXiv

Dynamic Security Testing for Third-Party Agent Skills

Published 8/4/2026

Analyzed 8/13/2026

SkillSentry evaluates third-party agent skills by constructing source-grounded decoy environments and comparing matched executions with and without the tested skill. The method requires completed, observable, skill-attributed side effects rather than treating suspicious text, ordinary privileged operations, or unexecuted paths as proven malicious behavior.

SkillSentry: Adaptive Honey Worlds for Dynamic Safety Testing of Agent Skills

Evaluated models: DeepSeek V4-Pro

Source: arXiv

Repository-Level Coding Agent Vulnerability Detection Gaps

Published 8/3/2026

Analyzed 8/13/2026

VulnGym measures whether coding agents can locate and explain repository-level security vulnerabilities from realistic advisory and source-code context. The benchmark contains 184 reviewed advisories, 408 line-annotated vulnerability entries, and 23 repositories, with separate end-to-end detection and oracle-conditioned localization tasks.

VulnGym: Benchmarking Coding Agents for Repository-Level Vulnerability Detection

Evaluated models: DeepSeek V4 Flash, GLM 5.2, MiniMax-M3 +4 more

Source: arXiv

Persistent Agent Memory Poisoning and Incomplete Repair

Published 7/29/2026

Analyzed 8/13/2026

MemSecBench follows malicious agent-memory content from initial write through persistence, retrieval, action selection, execution, and attempted selective repair. Its controlled Write–Execute–Forget protocol evaluates 310 human-reviewed cases across two harnesses, four memory backends, three model backends, and seven evidence-gated lifecycle checkpoints.

MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair

Evaluated models: GPT-5.5, DeepSeek V4-Pro, MiniMax-M3

Source: arXiv

Workspace Agent Runtime Safety Failures Across Risk Carriers

Published 7/29/2026

Analyzed 8/13/2026

AgentS4D measures unsafe actions and state changes across complete workspace-agent executions rather than treating task completion or isolated model responses as safety evidence. Its 328 sandboxed cases introduce risky content through user requests, documents, web resources, tools, third-party skills, and persistent memory, then compare the same cases across four agent harnesses and five model backends.

AgentS4D: Benchmarking Runtime Risks across the Execution Lifecycle of LLM-Based Workspace Agents

Evaluated models: GPT-5.5, Gemini 3.1 Pro, DeepSeek V4-Pro +2 more

Source: arXiv

Incomplete MCP Tool-Call Attack Detection

Published 7/28/2026

Analyzed 8/13/2026

The MTGuard study evaluates unsafe Model Context Protocol tool calls originating from compromised server data, host-side execution changes, and malicious user-controlled resources. Its hybrid monitor combines pre-execution parameter inspection, behavioral observation, and post-execution result verification across browser-automation and financial-analysis agents.

Hybrid Analysis for Secure MCP Tool Use in LLM Agents

Evaluated models: GPT-5.6 Luna, DeepSeek V4 Flash, DeepSeek V4-Pro

Source: arXiv

Long-Context Enterprise Agent Policy Adherence Failures

Published 7/28/2026

Analyzed 8/13/2026

HANDBOOK.md measures whether an agent can apply detailed organizational rules while completing realistic, multi-step enterprise tasks. The vendor-authored benchmark includes 65 resettable MCP-backed workflows, policy documents of 20 to 124 pages, and 824 deterministic rubric checks covering required decisions, prohibited actions, and final environment state.

HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following

Evaluated models: Claude Fable 5, GPT-5.6 Sol, Claude Opus 4.8 +16 more

Source: arXiv

Malicious GitHub Issue Injection Against Coding Agents

Published 7/22/2026

Analyzed 8/13/2026

IssueTrojanBench studies indirect prompt injection when a coding agent processes an apparently ordinary software-development issue or related artifact. Starting with six legitimate seed issues from two Python repositories, the authors construct 696 adversarial issue variants spanning four unsafe-action families and six delivery formats, then execute those variants across six agent-model configurations.

IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests

Evaluated models: GPT-5.3 Codex, GPT-5.4, Claude Sonnet 4.6

Source: arXiv

Third-Party Agent Skill Supply-Chain Execution Risks

Published 7/22/2026

Analyzed 8/13/2026

OpenSkillRisk evaluates whether agent harnesses safely handle third-party skills that introduce risky behavior through otherwise plausible, benign tasks. The benchmark assembles 263 risky skills from public agent-skill ecosystems and tests three CLI-agent harnesses against seven risk categories using isolated task workspaces, mocked external services, and execution-level evidence.

OpenSkillRisk: Benchmarking Agent Safety When Using Real-World Risky Third-Party Skills

Evaluated models: GPT-5.1 Codex Mini, GPT-5.3 Codex, GPT-5.4 +10 more

Source: arXiv

Lifecycle poisoning of reusable agent skills

Published 7/15/2026

Analyzed 7/21/2026

The paper presents SkillSec-Eval, a controlled evaluation of attacks against reusable agent skills across repository admission, semantic retrieval, planner selection, runtime execution, and updates. It reports that malicious metadata, retrieval manipulation, unsafe workflow composition, and poisoned updates can cause agents to retrieve, select, or execute unintended skills. These are paper-reported benchmark results, not independently verified vulnerabilities in a named production product.

Agent Skill Security: Threat Models, Attacks, Defenses, and Evaluation

Evaluated models: all-MiniLM-L6-v2, Gemini 3.1 Pro, Gemini 1.5 Flash

Source: arXiv

Related research categories