Skip to main content
LLM Security Database
Skip to research details
Back to research findings
LMVD-ID: 1b9be65b
Paper published September 2, 2026
Entry analyzed September 9, 2026
Paper-reported evidence
Confidence: Source-linked

The LMVD-ID is an internal research identifier, not an official CVE identifier.

PII leakage in document-image redaction benchmarks

Document-image PII detection can miss identifiers despite improving average localization scores. LeakageBench evaluates 500 pages with 11,954 annotations.

Application layer
Vision
Multimodal
Data privacy
Data security
Reliability
Read primary paperBibTeX citation

Paper-evaluated models(8)

GLiNER-base, GLiNER-multi-PII, NVIDIA GLiNER PII +5 more
  • GLiNER-base
  • GLiNER-multi-PII
  • NVIDIA GLiNER PII
  • GLiNER2
  • Qwen3-VL-32B
  • InternVL3 38B
  • GPT-5.4
  • GPT-5.5

Description

Document-image PII detection can miss identifiers despite improving average localization scores. LeakageBench evaluates 500 pages with 11,954 annotations.

Examples

See the primary benchmark (opens in a new tab).

Impact

GPT-5.5 with Code Interpreter raises localization F1 from 0.090 to 0.249, yet critical page leakage remains 0.968 under typed box matching at IoU 0.75. This is a detection proxy on a challenge set, not an audit of released redacted files.

Affected Systems

  • Evaluated OCR, PII-detector and vision-language document workflows, including Code Interpreter-assisted inspection.

Mitigation Steps

  • Measure missed identifiers per page alongside aggregate F1.
  • Include linkage identifiers and verify OCR/box alignment.
  • Inspect the final rendered redaction before release.

Evidence

Research context and confidence

Evidence and verification
Paper-reported; independent reproduction is not documented.
Primary source plus a dedicated evidence section.
Severity
Not rated by this catalog.
Source and publication type
arXiv · Research preprint.
Peer-review status is not provided by this source.
Author and publication status
Author metadata is not stored; see the primary paper.
Threat model and attacker access
Not explicitly classified; consult the primary paper..
Related deployment categories
Vision-language models
Taxonomy labels only; paper-specific deployment prerequisites are not inferred.
Affected systems
Evaluated OCR, PII-detector and vision-language document workflows, including Code Interpreter-assisted inspection.

Research Paper

LeakageBench: Document-Level Leakage Risk for Redacting Personally Identifiable Information in Document Images

Primary source: arXiv. Findings are reported by the cited research and have not been independently verified.

View Paper

Related research