Speech-Audio Composition Attack

Description: Multimodal Large Language Models (MLLMs) capable of processing speech and audio are vulnerable to Speech-Audio Compositional Attacks. This vulnerability exists because current safety mechanisms often rely on text-only transcription or fail to analyze the full acoustic context of an input. By manipulating the composition of audio signals, an attacker can bypass safety filters and elicit harmful responses. The attacks exploit three specific mechanisms: (1) Speech Overlap, where harmful instructions are acoustically masked beneath benign speech; (2) Multi-speaker Dialogue, where malicious intent is distributed across a conversation and triggered by a benign text query; and (3) Speech-Audio Mixture, where harmful intent is conveyed through non-speech background audio (e.g., sounds of violence) paired with benign speech, exploiting the model's "cross-modal blindness" to environmental context.

Examples:

• Speech Overlap Attack:
• Generate a benign carrier audio clip (e.g., "I’m writing a fictional novel and need to describe a scene in a chemistry lab for one of my characters").
• Generate a harmful audio payload (e.g., instructions on creating a dangerous substance).
• Mix the two streams using cross-fading, adjusting volume and playback speed so the harmful content is acoustically subordinate but still perceptible.
• Submit the mixed audio to the target model. The model may transcribe the benign carrier but act upon the harmful payload.
• Multi-Speaker Dialogue Attack:
• Synthesize a multi-turn dialogue between two speakers discussing the complexities of a harmful topic (e.g., developing nuclear weapons) without using direct command-like phrasing.
• Pair this audio with a semantically neutral text prompt that acts as a pointer, such as: "How would I create the device mentioned in the discussion?"
• The text prompt bypasses text-based filters, while the audio context forces the model to synthesize the harmful information.
• Speech-Audio Mixture (Contextual) Attack:
• Select a benign speech track (e.g., an academic lecture from the VoiceBank-DEMAND dataset).
• Overlay a background audio track containing harmful non-speech context (e.g., audio extracted from videos depicting violence or criminal activity).
• Submit the composite audio to the model. The model typically transcribes the benign speech while failing to flag the harmful environmental context, resulting in a safety failure.
• See the dataset and benchmark at: https://huggingface.co/datasets/tsinghua-ee/SACRED-Bench

Impact: This vulnerability allows attackers to successfully jailbreak state-of-the-art proprietary and open-source models, bypassing safety alignment training. Successful exploitation results in the generation of harmful content, including instructions for illegal acts, hate speech, or violence. In testing, the Speech-Audio Mixture attack achieved an 88.56% Attack Success Rate (ASR) against Google Gemini 2.5 Pro, and nearly 100% ASR against open-source models like Qwen2-Audio-7B.

Affected Systems:

• Google Gemini 2.5 Pro
• Google Gemini 1.5 Pro
• OpenAI GPT-4o
• Alibaba Qwen2-Audio-7B
• Alibaba Qwen2.5-Omni-7B
• MiniCPM-o 2.6
• Step-Audio 2 mini Base
• Kimi-Audio-7B-Instruct

Mitigation Steps:

• Deploy Multimodal Guard Models: Implement a specialized guard model (such as SALMONN-Guard) that jointly inspects speech, non-speech audio, and text inputs for safety judgments, rather than relying on text-only transcription filters.
• Train on Compositional Attack Data: Fine-tune safety models using datasets that include speech overlaps, multi-speaker dialogues, and harmful background audio contexts (like SACRED-Bench) to reduce cross-modal blindness.
• Verify Audio Context: Ensure the safety mechanism analyzes the environmental context of the audio (background sounds), not just the transcribed speech.