Skip to main content
LLM Security Database
Skip to research details
Back to research findings

Semantic Confusion Jailbreak

Published
Analyzed
Paper-reported evidence
Primary source linked
Read primary paper
Cite & share
Source BibTeX

Citation metadata is maintained by the primary source and may reflect a later revision.

Paper-evaluated models(5)

GPT-4o, Midjourney, Stable Diffusion +2 more
  • GPT-4o
  • Midjourney
  • Stable Diffusion
  • Stable Diffusion v1.4
  • Stable Diffusion v2.1
On this page

Description

The Antelope attack exploits vulnerabilities in Text-to-Image (T2I) models' safety filters by crafting adversarial prompts. These prompts, while appearing benign, induce the generation of NSFW images by leveraging semantic similarity between harmless and harmful concepts. The attack involves replacing explicit terms in an original prompt with seemingly innocuous alternatives and appending carefully selected suffix tokens. This manipulation bypasses both text-based and image-based filters, generating sensitive content while maintaining a high degree of semantic alignment with the original intent to evade detection.

Examples

Specific examples of adversarial prompts generated by Antelope are presented in the paper's Figure 6 and are omitted here due to their NSFW nature. See arXiv:2412.08156 (opens in a new tab) for details.

Impact

Successful Antelope attacks lead to the generation and dissemination of NSFW content, even in systems with safety filters in place. This compromises the intended safety mechanisms and potentially exposes users to harmful or offensive material. The transferability of the attack to online black-box services further increases the impact.

Affected Systems

A wide range of T2I models vulnerable to prompt injection, including but not limited to:

  • Stable Diffusion (various versions)
  • Midjourney
  • Leonardo.AI
  • Other models employing similar safety filtering mechanisms.

Mitigation Steps

  • Enhance prompt filtering: Implement more robust text-based filters capable of detecting subtle semantic manipulations.
  • Improve image-based filtering: Strengthen image recognition capabilities to identify NSFW content generated from apparently safe prompts.
  • Increase model robustness: Train the T2I models with more diverse and adversarial data to increase their resilience against prompt injection attacks.
  • Regular model updates: Continuously update and refine the model's safety mechanisms to keep pace with evolving attack techniques.
  • Diverse defensive mechanisms: Employ a multi-layered defense strategy that combines various methods, making exploitation significantly harder.

Research context and provenance

Research topics
Jailbreak
Black-box
Application layer
Prompt layer
Vision
Safety
Integrity
Catalog identifier
LMVD-a8264644
Internal research identifier, not an official CVE identifier.
Evidence and verification
Paper-reported; independent reproduction is not documented.
Primary research source linked.
Severity
Not rated by this catalog.
Source and publication type
arXiv · Research preprint.
Peer-review status is not provided by this source.
Author and publication status
Author metadata is not stored; see the primary paper.
Threat model and attacker access
Black-box model, service, or application access.
Related deployment categories
Vision-language models
Taxonomy labels only; paper-specific deployment prerequisites are not inferred.
Affected systems
A wide range of T2I models vulnerable to prompt injection, including but not limited to: Stable Diffusion (various versions) Midjourney Leonardo.AI Other models employing similar safety filtering mechanisms.

Research Paper

Antelope: Potent and Concealed Jailbreak Attack Strategy

Primary source: arXiv. Findings are reported by the cited research and have not been independently verified.

View Paper

Evidence

This entry is based on a primary research source. Its findings are paper-reported; independent reproduction and verification are not claimed.

https://arxiv.org/abs/2412.08156

Related research