Language Model Security Database

PathSeeker demonstrates a novel black-box jailbreak attack against Large Language Models (LLMs) that utilizes multi-agent reinforcement learning. The attack iteratively modifies input prompts based on model responses, leveraging a reward mechanism focused on vocabulary expansion in the LLM's output to circumvent safety mechanisms and elicit harmful responses. This technique bypasses existing safety filters by encouraging the model to relax its constraints, rather than directly targeting…

Large Language Models (LLMs) are vulnerable to a novel attack paradigm, "jailbreak-tuning," which combines data poisoning with jailbreaking techniques to bypass existing safety safeguards. This allows malicious actors to fine-tune LLMs to reliably generate harmful outputs, even when trained on mostly benign data. The vulnerability is amplified in larger LLMs, which are more susceptible to learning harmful behaviors from even minimal exposure to poisoned data.

LLM-based autonomous agents are vulnerable to malfunction amplification attacks. These attacks exploit the inherent instability of agents by inducing repetitive or irrelevant actions through various methods including prompt injection and adversarial perturbations, leading to agent malfunction and task failure. The attacks do not rely on overtly harmful actions, making them harder to detect with standard LLM safety mechanisms.

Large Language Models (LLMs) exhibit increased vulnerability to adversarial prompts employing code-switching techniques, where multiple languages are interwoven within a single query. This vulnerability stems from an unintended correlation between the resource availability of the languages used in the prompt and the LLM's safety alignment. LLMs trained on imbalanced multilingual data are more susceptible to attacks leveraging low-resource languages, resulting in a higher rate of unsafe or…

AutoBreach exploits the vulnerability of Large Language Models (LLMs) to wordplay-based adversarial prompts. By leveraging an LLM to generate diverse wordplay mapping rules and employing a two-stage optimization strategy, AutoBreach crafts prompts that bypass LLM safety mechanisms and elicit harmful or unintended responses, even without modifying system prompts. The vulnerability lies in the LLM's susceptibility to semantic manipulation through cleverly disguised inputs.

Leading safety-aligned Large Language Models (LLMs) are vulnerable to simple adaptive jailbreaking attacks. These attacks utilize manually crafted prompt templates, combined with random search on a suffix to maximize the log-probability of a target token indicating compliance (e.g., "Sure"). The attacks are adaptive, as the prompt template and target token are customized for specific models. Furthermore, some models are vulnerable to transfer attacks (using successful prompts from one LLM on…