Universal Jailbreak Prompt Generator

Description: Large Language Models (LLMs) are vulnerable to robust jailbreak prompts generated by the ArrAttack framework. ArrAttack uses a two-stage process: a robustness judgment model trained to identify prompts that bypass existing LLM safety mechanisms, and a robust jailbreak prompt generation model that leverages this information to create highly effective attacks. This allows attackers to bypass multiple defense mechanisms, including perplexity-based detection, input preprocessing, and re-tokenization methods.

Examples: See the paper for specific examples of successful jailbreak prompts generated by ArrAttack against various LLMs and defense mechanisms. Examples include prompts designed to elicit instructions on bomb-making and methods to conceal criminal activity; these prompts successfully bypassed several defense mechanisms.

Impact: Successful exploitation of this vulnerability allows attackers to elicit harmful or unintended content from LLMs, circumventing built-in safety measures. This can lead to the generation of illegal content, malicious code, misinformation, or other harmful outputs. The impact is amplified by the transferability of ArrAttack across various LLMs and defense strategies.

Affected Systems: All LLMs susceptible to rewriting-based attacks, particularly those employing defenses that do not explicitly account for the adversarial prompt generation techniques described in the ArrAttack paper. Specific models mentioned in the research include but are not limited to GPT-4, Claude-3, Llama2-7b-chat, Vicuna-7b, and Guanaco-7b.

Mitigation Steps:

• Improve Defense Mechanisms: Develop and deploy more robust LLM safety mechanisms capable of identifying and neutralizing the types of adversarial prompts generated by ArrAttack. Consider defenses that incorporate techniques beyond simple input preprocessing and perplexity analysis.
• Regular Model Updates: Frequently update and retrain LLMs with updated datasets specifically to address the evolving landscape of jailbreak attacks.
• Monitoring and Detection: Implement systems for actively monitoring LLM outputs and detecting patterns indicative of successful jailbreak attempts.
• Adversarial Training: Incorporate techniques of adversarial training into the LLM development process to improve model robustness to a wider array of prompts.