The LMVD-ID is an internal research identifier, not an official CVE identifier.
Boilerplate safety tuning causes false refusals on benign requests
Controlled safety-tuning experiments link boilerplate refusal statements to unnecessary refusals of benign requests. Request-specific rationales improve benign compliance, with benchmark-dependent safety tradeoffs.
Paper-evaluated models(12)
Llama 3.1 8B, Mistral 7B v0.3, Gemma 2 9B +9 more
- Llama 3.1 8B
- Mistral 7B v0.3
- Gemma 2 9B
- Qwen 2.5 7B
- Gemma 2 2B
- Qwen 2.5 3B
- Llama 3.1 70B
- Qwen 2.5 72B
- Llama 3.1 8B Instruct
- Mistral 7B Instruct v0.3
- Gemma 2 9B IT
- Qwen 2.5 7B Instruct
Description
Controlled safety-tuning experiments link boilerplate refusal statements to unnecessary refusals of benign requests. Request-specific rationales improve benign compliance, with benchmark-dependent safety tradeoffs.
Examples
See the primary study (opens in a new tab).
Impact
For Llama-3.1-8B, rationale-only versus statement-only training raises XSTest-Safe compliance from 0.48 to 0.71, while MaliciousInstruct harmful compliance rises from 0.02 to 0.06. The principal experiment uses 256 safety examples and automated judgments with human agreement checks. Results do not establish universally safer tuning.
Affected Systems
- Safety fine-tuning pipelines; four principal base models, additional sizes, and released instruction-tuned comparisons are evaluated separately.
Mitigation Steps
- Check harmful and benign compliance together for each model.
- Prefer request-specific rationales over generic refusal boilerplate when evaluation supports the change.
- Audit automated judgments for partial refusals and deflections.
Evidence
Research context and confidence
- Evidence and verification
- Paper-reported; independent reproduction is not documented.
- Primary source plus a dedicated evidence section.
- Severity
- Not rated by this catalog.
- Source and publication type
- arXiv · Research preprint.
- Peer-review status is not provided by this source.
- Author and publication status
- Author metadata is not stored; see the primary paper.
- Threat model and attacker access
- Not explicitly classified; consult the primary paper..
- Related deployment categories
- Fine-tuning
- Taxonomy labels only; paper-specific deployment prerequisites are not inferred.
- Affected systems
- Safety fine-tuning pipelines; four principal base models, additional sizes, and released instruction-tuned comparisons are evaluated separately.
Research Paper
Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models
Primary source: arXiv. Findings are reported by the cited research and have not been independently verified.
View PaperRelated research
- Abliteration Cripples Math
Published December 1, 2025 · model-layer, jailbreak, fine-tuning
- Twin Prompt Jailbreak
Published June 1, 2025 · model-layer, jailbreak, whitebox
- LLM Lower Layer Freeze Jailbreak
Published February 1, 2025 · model-layer, jailbreak, fine-tuning