Improved LLM Jailbreak Transferability

Description

Large Language Models (LLMs) employing gradient-based optimization for jailbreaking defense are vulnerable to enhanced transferability attacks due to superfluous constraints in their objective functions. Specifically, the "response pattern constraint" (forcing a specific initial response phrase) and the "token tail constraint" (penalizing variations in the response beyond a fixed prefix) limit the search space and reduce the effectiveness of attacks across different models. Removing these constraints significantly increases the success rate of attacks transferred to target models.

Examples

See the paper's GitHub repository: https://github.com/thu-coai/TransferAttack (opens in a new tab)

Impact

Successful exploitation allows attackers to bypass LLM safety mechanisms with higher success rates across a wider range of models than previously possible with gradient-based methods, leading to the generation of unsafe or malicious content. This affects the reliability of safety measures implemented in LLMs.

Affected Systems

LLMs utilizing gradient-based optimization for safety mechanisms and employing "response pattern" and "token tail" constraints in their objective functions. Specifically, the paper demonstrates this vulnerability on Llama-3-8B-Instruct, Llama-2-7B-Chat, and several other models (see paper for details).

Mitigation Steps

• Re-evaluate the objective function used in gradient-based LLM safety mechanisms. Remove or significantly relax constraints on the response pattern beyond the essential elements needed for safety.
• Relax or remove constraints on the “token tail,” allowing for more variability in output while maintaining core safety restrictions.
• Develop more robust safety mechanisms that are less susceptible to manipulation via gradient-based attacks. Consider methods beyond simple token-level prediction.