Architecture Components

Audio Vulnerabilities

Security issues in speech and audio-enabled AI systems

8 matching entries out of 19 in this category

Audio research results

8 entries

Inaudible Ultrasonic LLM Jailbreak

Updated 4/10/2026

Speech-driven Large Language Models (LLMs) and end-to-end Large Audio-Language Models (LALMs) are vulnerable to inaudible near-ultrasonic prompt injections, a framework dubbed Sirens' Whisper (SWhisper). By exploiting the non-linear response of commodity microphones, attackers can encode structured, phonetically optimized adversarial prompts into the 17–22 kHz near-ultrasonic band. Using regularized channel-inversion pre-compensation, the attacker shapes the waveform to account for microphone…

Sirens' Whisper: Inaudible Near-Ultrasonic Jailbreaks of Speech-Driven LLMs

Affects: GLM-4 Voice, Qwen Omni Turbo, Llama 3.1 8B Instruct +5 more

Source: arXiv

Voice Agent Behavioral Bypass

Updated 2/21/2026

Audio Large Language Models (ALLMs) integrated into voice agent systems for high-stakes domains (banking, IT support, logistics) are vulnerable to multimodal adversarial attacks via spoken interaction. Adversaries can exploit the model's inherent compliance and contextual awareness through multi-turn dialogue to bypass authentication safeguards, escalate privileges (e.g., unauthorized credit limit increases), exfiltrate sensitive Personally Identifiable Information (PII), and poison…

Aegis: Towards Governance, Integrity, and Security of AI Voice Agents

Affects: GPT-4o, GPT-4o Mini, Gemini 1.5 Pro +4 more

Source: arXiv

Speech-Audio Composition Attack

Updated 12/30/2025

Multimodal Large Language Models (MLLMs) capable of processing speech and audio are vulnerable to Speech-Audio Compositional Attacks. This vulnerability exists because current safety mechanisms often rely on text-only transcription or fail to analyze the full acoustic context of an input. By manipulating the composition of audio signals, an attacker can bypass safety filters and elicit harmful responses. The attacks exploit three specific mechanisms: (1) Speech Overlap, where harmful…

Speech-Audio Compositional Attacks on Multimodal LLMs and Their Defense with SALMONN-Guard

Affects: Qwen2-Audio 7B, Qwen 2.5 Omni 7B, Step-Audio 2 Mini Base +6 more

Source: arXiv

Benign Audio Jailbreak

Updated 12/9/2025

Audio-Language Models (ALMs) including Qwen2.5-Omni (3B and 7B) and Phi-4-Multimodal are vulnerable to "WhisperInject," a two-stage adversarial audio attack that bypasses safety guardrails. The vulnerability allows an attacker to inject imperceptible perturbations into benign audio inputs (e.g., a query about the weather) that force the model to generate specific harmful content. The attack utilizes a novel optimization method, Reinforcement Learning with Projected Gradient Descent (RL-PGD)…

When Good Sounds Go Adversarial: Jailbreaking Audio-Language Models with Benign Inputs

Affects: Qwen 2.5 Omni 3B, Qwen 2.5 Omni 7B, Phi-4 Multimodal +2 more

Source: arXiv

Airborne Audio LLM Hijack

Updated 1/14/2026

Audio-based Large Language Models (ALLMs), specifically Qwen2-Audio, are vulnerable to over-the-air adversarial audio attacks. An attacker with white-box access can generate robust adversarial audio perturbations using gradient-based optimization combined with audio augmentation techniques (specifically SpecAugment, translation, and additive noise). These perturbations, when played through a speaker in the physical environment, manipulate the ALLM processing the audio via a microphone. This…

Attacker's Noise Can Manipulate Your Audio-based LLM in the Real World

Source: arXiv

Fine-Grained Speech LLM Control

Updated 12/9/2025

Speech-LLMs Qwen2-Audio (7B-Instruct) and Granite-Speech (3.2-8b) are vulnerable to universal acoustic adversarial attacks. An attacker can optimize a fixed, input-agnostic audio segment (approximately 3.2 seconds in length) via gradient-based optimization on the model's frozen weights. When this adversarial segment is prepended to any arbitrary user speech input, it manipulates the model's latent representation, effectively overriding system prompts and generation behavior. This vulnerability…

Universal Acoustic Adversarial Attacks for Flexible Control of Speech-LLMs

Affects: Qwen 2 7B

Source: arXiv

Multi-Accent Audio Jailbreak

Updated 4/12/2025

Multilingual and multi-accent audio inputs, combined with acoustic adversarial perturbations (reverberation, echo, whisper effects), can bypass safety mechanisms in Large Audio Language Models (LALMs), causing them to generate unsafe or harmful outputs. The vulnerability is amplified by the interaction between acoustic and linguistic variations, particularly in languages with less training data.

Multilingual and Multi-Accent Jailbreaking of Audio LLMs

Affects: DIVA Llama 3 v0 8B, MERaLion AudioLLM, MiniCPM-o 2.6 +2 more

Source: arXiv

Adversarial Speech Jailbreak

Updated 12/29/2024

Multimodal Large Language Models (LLMs) processing speech input are vulnerable to adversarial attacks. Imperceptible perturbations added to audio input can cause the model to generate unsafe or harmful text responses, overriding built-in safety mechanisms. The attacks are effective even with limited knowledge of the model's internal workings, demonstrating transferability across different models.

SpeechGuard: Exploring the adversarial robustness of multimodal large language models

Affects: Flan-T5 XL, Llama 7B, Llama 2 13B Chat +2 more

Source: arXiv

Related research categories