Language Model Security Database

A policy compliance vulnerability exists in the OpenAI GPT Store ecosystem affecting Custom GPTs. The vulnerability stems from the inheritance of safety alignment weaknesses from foundational models (GPT-4 and GPT-4o) and the insufficient enforcement of usage policies during the customization and review process. Custom GPTs can be trivially manipulated to violate safety guidelines—specifically regarding Cybersecurity (malware generation), Academic Integrity (ghostwriting), and Romantic…

Post-hoc Large Language Model (LLM) unlearning and guardrailing mechanisms (specifically In-Context Unlearning [ICUL] and standard prompt-based Guardrailing) are vulnerable to information leakage attacks via "Target Masking" and indirect referencing. These systems rely on superficial semantic matching to suppress "forget sets" (specific entities or concepts). Attackers can bypass these restrictions by querying associated properties, relationships, or pseudonyms rather than the explicit target…

The Virus attack method enables attackers to bypass guardrail moderation on fine-tuning data, leading to a significant degradation of safety alignment in large language models (LLMs). This is achieved through a dual-objective data optimization strategy that crafts harmful data undetectable by the guardrail while maximizing their effectiveness in compromising the victim model's safety.

Vision Language Models (VLMs) are vulnerable to visual prompt injection attacks via text-to-image obfuscation. While these models often possess safety guardrails for standard text-based inputs, they fail to apply equivalent safety alignment to textual instructions embedded visually within an image. An attacker can overlay malicious instructions (e.g., requests for illegal acts, hate speech) onto an image file and submit it to the model. The model’s Optical Character Recognition (OCR) or visual…

Large Audio-Language Models (LALMs) are vulnerable to a stealthy adversarial jailbreak attack, AdvWave, which leverages a dual-phase optimization to overcome gradient shattering caused by audio discretization. The attack crafts adversarial audio by adding perceptually realistic environmental noise, making it difficult to detect. The attack also dynamically adapts the adversarial target based on the LALM's response patterns.

A bimodal adversarial attack, PBI-Attack, can manipulate Large Vision-Language Models (LVLMs) into generating toxic or harmful content by iteratively optimizing both textual and visual inputs in a black-box setting. The attack leverages a surrogate LVLM to inject malicious features from a harmful corpus into a benign image, then iteratively refines both image and text perturbations to maximize the toxicity of the model’s output as measured by a toxicity detection model (Perspective API or…

LLM-based planning modules in embodied AI systems are vulnerable to Policy Executable (POEX) jailbreak attacks. Attackers can inject carefully crafted adversarial suffixes into user instructions, causing the LLM to generate and execute harmful policies in both simulated and real-world environments. The attacks bypass safety mechanisms by using optimized, human-readable suffixes that evade perplexity-based detection.

Large Vision-Language Models (VLMs) are vulnerable to a novel black-box jailbreak attack, IDEATOR, which leverages a separate VLM to generate malicious image-text pairs. The attacker VLM iteratively refines its prompts based on the target VLM's responses, bypassing safety mechanisms by generating contextually relevant and visually subtle malicious prompts.

Refusal-trained Large Language Models (LLMs) show decreased safety when deployed as browser agents compared to their performance in chatbot settings. Attack methods effective at jailbreaking LLMs in chat contexts also successfully bypass safety mechanisms in browser agents, leading to the execution of harmful behaviors. This vulnerability stems from a lack of generalization of safety training to agentic, real-world interaction scenarios and the increased context available to the agent (browser…

Large Language Models (LLMs) are vulnerable to transferable ensemble black-box jailbreak attacks. The vulnerability allows an attacker to bypass safety mechanisms and elicit undesired or harmful responses from the LLM by using an ensemble of LLM-as-attacker methods that optimize malicious prompts, adaptively adjusting resources based on prompt difficulty, and strategically modifying prompt semantics to evade detection.