Ensemble Black-box Jailbreak

Description: Large Language Models (LLMs) are vulnerable to transferable ensemble black-box jailbreak attacks. The vulnerability allows an attacker to bypass safety mechanisms and elicit undesired or harmful responses from the LLM by using an ensemble of LLM-as-attacker methods that optimize malicious prompts, adaptively adjusting resources based on prompt difficulty, and strategically modifying prompt semantics to evade detection.

Examples: See arXiv:2405.18540 (This is a placeholder and should be replaced with the actual arXiv link if available. Otherwise, provide concrete examples from the paper.)

Impact: Successful exploitation could lead to LLMs generating responses that violate their intended safety constraints, providing harmful instructions, biased information, or revealing sensitive data. The transferability of the attack allows its use across multiple LLMs.

Affected Systems: Multiple large language models (LLMs). Specific models affected are not explicitly listed in the research but include Gemma-2B-IT, Gemma2-9B-IT (targets) and Llama3-8B-Instruct, GLM-4-Plus, GLM-4-Flash, Qwen-Max-Latest, and DeepSeek-V2.5 (judges).

Mitigation Steps:

• Implement robust prompt filtering mechanisms that go beyond simple keyword matching.
• Develop defense strategies that are resistant to ensemble attacks.
• Improve or diversify the internal embedding representations within the LLMs to make them less vulnerable to semantic manipulation.
• Employ more sophisticated mechanisms for detecting and mitigating adversarial prompt crafting techniques.
• Regularly audit and update the LLMs’ safety mechanisms against new attack vectors.