Typographic VLM Jailbreak

Description

Large Vision-Language Models (VLMs) are vulnerable to jailbreaking attacks via typographically rendered visual prompts. The vulnerability stems from the VLM's ability to process and interpret image-based text, bypassing safety mechanisms designed for text-only prompts. Malicious actors can encode harmful instructions into images, which are then processed by the VLM's visual module and subsequently interpreted by the language model, resulting in the generation of unsafe and policy-violating responses.

Examples

See the paper "Figstep: Jailbreaking large vision-language models via typographic visual prompts (opens in a new tab)" for multiple examples. The attack involves generating images with harmful instructions rendered typographically. These images, paired with a benign text prompt, are input to the VLM. The VLM then generates text that fulfills the harmful instructions encoded in the image. Specific examples are shown in Figures 4, 6, and 10 of the paper.

Impact

Successful exploitation allows malicious actors to bypass safety filters and elicit unsafe responses from VLMs, including but not limited to generation of harmful content, instructions for illegal activities, dissemination of misinformation, and bypassing content moderation systems.

Affected Systems

Various open-source and closed-source VLMs, including but not limited to LLaVA-v1.5, MiniGPT-4, CogVLM, and GPT-4V are susceptible to this attack method. The vulnerability is not limited to specific model architectures.

Mitigation Steps

• Implement robust cross-modal safety alignment mechanisms.
• Develop and deploy more sophisticated content filtering techniques that analyze both textual and visual inputs.
• Research and implement defenses that detect and mitigate typographic attacks.
• Employ multiple layers of security, including pre-processing of visual inputs to detect and neutralize harmful content.
• Regularly update and fine-tune models to improve their resistance to novel attacks.