Multimodal Contextual Jailbreak

Description

Multimodal Large Language Models (MLLMs) are vulnerable to a jailbreaking attack, dubbed PiCo, that leverages token-level typographic attacks on images embedded within code-style instructions. The attack bypasses multi-tiered defense mechanisms, including input filtering and runtime monitoring, by exploiting weaknesses in the visual modality's integration with programming contexts. Harmful intent is concealed within visually benign image fragments and code instructions, circumventing safety protocols.

Examples

See arXiv:2504.01444 (opens in a new tab) for specific image-based examples and details on generating images with typographically altered words.

Impact

Successful exploitation allows attackers to bypass safety protocols and elicit harmful or unsafe responses from the MLLM, including but not limited to generating content related to violence, financial crime, privacy violation, animal abuse, and self-harm. The attack's success rate varies across models but is shown to be significant in many cases.

Affected Systems

Multimodal Large Language Models (MLLMs), including but not limited to Gemini Pro Vision, GPT-4V, GPT-4o, GPT-4-Turbo, and LLAVA-1.5. The attack is effective against both open-source and closed-source models.

Mitigation Steps

• Enhance input filtering to detect typographical attacks on images, potentially utilizing techniques beyond simple keyword matching.
• Improve runtime monitoring to detect malicious intent embedded within seemingly benign code instructions.
• Develop more robust defenses against cross-modal attacks, considering the interaction between visual and textual inputs.
• Conduct comprehensive red-teaming exercises to identify and address vulnerabilities in MLLM defenses.