Skip to main content
LLM Security Database
Skip to research details
Back to research findings

Flowchart-based LVLM Jailbreak Attack

Published
Analyzed
Paper-reported evidence
Primary source linked
Read primary paper
Cite & share
Source BibTeX

Citation metadata is maintained by the primary source and may reflect a later revision.

Paper-evaluated models(7)

Claude 3.5 Sonnet 20240620, Gemini 1.5 Flash, GPT-4o 2024-08-06 +4 more
  • Claude 3.5 Sonnet 20240620
  • Gemini 1.5 Flash
  • GPT-4o 2024-08-06
  • GPT-4o Mini 2024-07-18
  • InternVL2.5 8B
  • LLaVA NeXT 8B
  • Qwen2-VL 7B Instruct
On this page

Description

FC-Attack leverages automatically generated flowcharts containing step-by-step descriptions derived or rephrased from harmful queries, combined with a benign textual prompt, to jailbreak Large Vision-Language Models (LVLMs). The vulnerability lies in the model's susceptibility to visual prompts containing harmful information within the flowcharts, thus bypassing safety alignment mechanisms.

Examples

See repository, specifically instances where flowcharts, generated from harmful queries within the Advbench dataset, successfully induce harmful outputs from models such as Gemini-1.5, Llava-Next, Qwen2-VL, and InternVL-2.5. A key example would be Figure 3 in the paper.

Impact

Enables attackers to bypass safety mechanisms of LVLMs, leading to the generation of harmful content, including but not limited to hate speech, instructions for illegal activities, or the sharing of misinformation, thus exposing end-users to potentially illegal or harmful content.

Affected Systems

Large Vision-Language Models (LVLMs), specifically:

  • Gemini 1.5 Flash
  • LLaVA NeXT 8B
  • Qwen 2 VL 7B Instruct
  • InternVL 2.5 8B
  • GPT-4o Mini 2024-07-18
  • GPT-4o 2024-08-06
  • Claude 3.5 Sonnet 20240620 (The degree of impact can vary based on model and the specific flowcharts used as part of the prompt attack).

Mitigation Steps

  • Utilize AdaShield-A or similar defenses that employ adaptive shield prompting, but be aware of the potential for reduced model utility.
  • Improve detection of textual information in visual content, as current detection methods are shown to not be effective.
  • Consider the impact that different font styles have on the ASR.

Research context and provenance

Research topics
Model layer
Application layer
Prompt layer
Vision
Multimodal
Fine-tuning
Black-box
AI agents
Catalog identifier
LMVD-cd01fb40
Internal research identifier, not an official CVE identifier.
Evidence and verification
Paper-reported; independent reproduction is not documented.
Primary research source linked.
Severity
Not rated by this catalog.
Source and publication type
arXiv · Research preprint.
Peer-review status is not provided by this source.
Author and publication status
Author metadata is not stored; see the primary paper.
Threat model and attacker access
Black-box model, service, or application access.
Related deployment categories
Vision-language models; Fine-tuning; Agent workflows; Model APIs
Taxonomy labels only; paper-specific deployment prerequisites are not inferred.
Affected systems
Large Vision-Language Models (LVLMs), specifically: Gemini 1.5 Flash LLaVA NeXT 8B Qwen 2 VL 7B Instruct InternVL 2.5 8B GPT-4o Mini 2024-07-18 GPT-4o 2024-08-06 Claude 3.5 Sonnet 20240620 (The degree of impact can…

Research Paper

FC-Attack: Jailbreaking Large Vision-Language Models via Auto-Generated Flowcharts

Primary source: arXiv. Findings are reported by the cited research and have not been independently verified.

View Paper

Evidence

This entry is based on a primary research source. Its findings are paper-reported; independent reproduction and verification are not claimed.

https://arxiv.org/abs/2502.21059

Related research