Updated 12/29/2024
Multimodal Large Language Models (LLMs) processing speech input are vulnerable to adversarial attacks. Imperceptible perturbations added to audio input can cause the model to generate unsafe or harmful text responses, overriding built-in safety mechanisms. The attacks are effective even with limited knowledge of the model's internal workings, demonstrating transferability across different models.
SpeechGuard: Exploring the adversarial robustness of multimodal large language models
Affects: Flan-T5 XL, Llama 7B, Llama 2 13B Chat +2 more
Source: arXiv