Semantic Mirror Jailbreak

Description

Large Language Models (LLMs) are vulnerable to a novel semantic mirror jailbreak attack. This attack leverages a genetic algorithm to generate jailbreak prompts that are semantically similar to benign prompts, evading defenses based on semantic similarity metrics. The attack achieves this by optimizing for both semantic similarity to the original question and the ability to elicit harmful responses.

Examples

Due to the dynamic nature of the attack and the need for a specific prompt generation process using a genetic algorithm, concrete examples are not easily provided. See arXiv:2402.14872 (opens in a new tab) for the genetic-algorithm process used to generate the attack prompts.

Impact

Successful exploitation allows attackers to bypass LLM safety mechanisms and elicit harmful or sensitive information, including hate speech, illegal instructions, and other undesired outputs. The evasion of semantic similarity-based defenses increases the attack's effectiveness and makes it significantly harder to mitigate.

Affected Systems

Open-source LLMs, including Llama-2, Vicuna, and Guanaco tested in the research paper. The vulnerability is likely to affect other LLMs employing similar safety mechanisms.

Mitigation Steps

• Implement more robust safety mechanisms beyond simple semantic similarity checks, possibly incorporating contextual analysis and intention detection.
• Develop and deploy more sophisticated detection mechanisms able to identify the subtle manipulations used by the attack even when semantically similar to benign prompts.
• Regularly update LLM safety models and filters to address new threats. Use adversarial training to strengthen resistance to this style of attack .
• Rate-limit queries with high semantic similarity, especially if from the same source.