GCG Suffix Data Exfiltration

Description: A Cross-Prompt Injection Attack (XPIA) can be amplified by appending a Greedy Coordinate Gradient (GCG) suffix to the malicious injection. This increases the likelihood that a Large Language Model (LLM) will execute the injected instruction, even in the presence of a user's primary instruction, leading to data exfiltration. The success rate of the attack depends on the LLM's complexity; medium-complexity models show increased vulnerability.

Examples: See the white paper for experimental setup and results. The attack involves crafting a malicious injection targeting a specific function (e.g., a network request) and embedding it within third-party data presented to the LLM alongside a user prompt. Appending a GCG suffix to the injection significantly increases the probability of the LLM executing the injection. Specific examples from the dataset are not publicly available.

Impact: Successful exploitation leads to data exfiltration from the user's context, potentially exposing sensitive information such as credentials or Personal Identifiable Information (PII). The financial impact of a successful attack can be significant.

Affected Systems: LLMs vulnerable to XPIA and susceptible to manipulation by GCG suffixes. Specifically, the paper tested Phi-3-mini, GPT-3.5, and GPT-4, showing varying degrees of vulnerability. Other LLMs with similar architecture or training may also be affected.

Mitigation Steps:

• Improved Prompt Filtering: Implement more robust prompt filtering techniques to detect malicious injections, particularly those incorporating GCG suffixes.
• Model Complexity: Consider using more complex LLMs, as they exhibit higher resistance to this attack vector.
• GCG Suffix Detection: Develop methods to specifically identify and neutralize GCG suffixes.
• Function Call Sanitization: Sanitize and validate function calls generated by the LLM before execution, to prevent the misuse of tools that provide external access (e.g., network requests).
• Defense Variation: Implement different defense strategies for LLMs of varying complexities, recognizing that the effectiveness of certain defenses (such as prompt filtering) may vary drastically based on model complexity.