Impact

Reliability Vulnerabilities

Issues affecting model consistency and dependability

3 matching entries out of 187 in this category

Reliability research results

3 entries

LLM Interpreter Resource Exhaustion

Updated 8/16/2025

Large Language Models (LLMs) equipped with native code interpreters are vulnerable to Denial of Service (DoS) via resource exhaustion. An attacker can craft a single prompt that causes the interpreter to execute code that depletes CPU, memory, or disk resources. The vulnerability is particularly pronounced when a resource-intensive task is framed within a plausibly benign or socially-engineered context ("indirect prompts"), which significantly lowers the model's likelihood of refusal compared…

Running in CIRCLE? A Simple Benchmark for LLM Code Interpreter Security

Affects: Gemini 2.0 Flash, Gemini 2.5 Flash, Gemini 2.5 Pro +5 more

Source: arXiv

LLM Judge Subversion

Updated 12/9/2025

Alibaba Cloud PAI-Judge and PAI-Judge-Plus are vulnerable to a composite adversarial attack that exploits attention mechanism limitations in Large Language Models (LLMs). An authenticated attacker can manipulate automated evaluation outcomes by appending a long, irrelevant text suffix (approximately 1000 to 2000+ characters) to a response containing adversarial perturbations. This "long-suffix" strategy overwhelms the judge model's context window, causing the attention mechanism to degrade and…

LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judge

Affects: GPT-4o, Llama 3.1 8B, Llama 3.3 70B +3 more

Source: arXiv

Flowchart-based LVLM Jailbreak Attack

Updated 3/19/2025

FC-Attack leverages automatically generated flowcharts containing step-by-step descriptions derived or rephrased from harmful queries, combined with a benign textual prompt, to jailbreak Large Vision-Language Models (LVLMs). The vulnerability lies in the model's susceptibility to visual prompts containing harmful information within the flowcharts, thus bypassing safety alignment mechanisms.

FC-Attack: Jailbreaking Large Vision-Language Models via Auto-Generated Flowcharts

Affects: Claude 3.5 Sonnet 20240620, Gemini 1.5 Flash, GPT-4o 2024-08-06 +4 more

Source: arXiv

Related research categories