Impact

Reliability Research

Issues affecting model consistency and dependability

7 matching entries out of 203 in this category

Reliability research results

7 entries

LLM Judge Manipulation

Published 3/1/2026

Analyzed 4/10/2026

Generative reward models deployed as LLM-as-a-Judge (LaaJ) evaluators contain a logic bypass vulnerability where superficial "master key" inputs trigger false positive rewards regardless of actual response quality. Instead of evaluating the candidate's output, large judge models are inadvertently triggered by specific token sequences to solve the prompt independently. This allows malicious actors or policy models undergoing reinforcement learning to consistently game the reward signal by…

Security in LLM-as-a-Judge: A Comprehensive SoK

Evaluated models: GPT-4o, o1, Qwen 2.5 72B Instruct +1 more

Source: arXiv

LLM Safety Geometry Fragility

Published 2/1/2026

Analyzed 2/21/2026

Large Language Models (LLMs) aligned via standard preference-based optimization methods (e.g., DPO, RLHF) are vulnerable to safety degradation due to optimization-induced fragility. The vulnerability arises from sharp minima in the alignment loss landscape, specifically within a small, localized subspace of safety-critical parameters (approximately 0.5% of neurons account for >80% of worst-case alignment loss). Standard alignment algorithms enforce uniform constraints or fail to control the…

Revisiting Robustness for LLM Safety Alignment via Selective Geometry Control

Evaluated models: Llama 3 8B, Llama 3.2 3B, Qwen 2.5 7B

Source: arXiv

Unified Robustness Gap

Published 2/1/2026

Analyzed 3/9/2026

Text scoring models, including dense retrievers, rerankers, and reward models, are vulnerable to score manipulation attacks via search-based discrete perturbations and content injection. An attacker can systematically modify candidate texts using rudimentary string manipulations, gradient-guided token swaps (e.g., HotFlip), masked language modeling (MLM) swaps, or query/sentence injections to spuriously increase model scores. This structural failure condition allows an irrelevant passage or a…

Unifying Adversarial Robustness and Training Across Text Scoring Models

Evaluated models: E5 BERT-base, Qwen 3 0.6B, Llama 3.2 3B Instruct +2 more

Source: arXiv

Voice Agent Behavioral Bypass

Published 2/1/2026

Analyzed 2/21/2026

Audio Large Language Models (ALLMs) integrated into voice agent systems for high-stakes domains (banking, IT support, logistics) are vulnerable to multimodal adversarial attacks via spoken interaction. Adversaries can exploit the model's inherent compliance and contextual awareness through multi-turn dialogue to bypass authentication safeguards, escalate privileges (e.g., unauthorized credit limit increases), exfiltrate sensitive Personally Identifiable Information (PII), and poison…

Aegis: Towards Governance, Integrity, and Security of AI Voice Agents

Evaluated models: GPT-4o, GPT-4o Mini, Gemini 1.5 Pro +4 more

Source: arXiv

Quantization Contrastive Jailbreak

Published 1/1/2026

Analyzed 1/14/2026

A malicious model supply chain vulnerability exists involving a technique termed Adversarial Contrastive Learning (ACL) for Large Language Model (LLM) quantization attacks. This vulnerability allows an attacker to publish a model that appears benign and preserves high utility in full precision (e.g., BF16 or FP32) but exhibits malicious behaviors—such as jailbreak, over-refusal, or advertisement injection—immediately upon zero-shot quantization (e.g., INT8, FP4, or NF4).

Adversarial Contrastive Learning for LLM Quantization Attacks

Evaluated models: Qwen 2.5 1.5B Instruct, Qwen 2.5 3B Instruct, Llama 3.2 1B Instruct +1 more

Source: arXiv

BarkPlug Data Poisoning Attack

Published 12/1/2024

Analyzed 3/19/2025

A poisoning attack against a Retrieval-Augmented Generation (RAG) system that manipulates the retriever component by injecting a poisoned document into the data used by the embedding model. This poisoned document contains modified and incorrect information. When activated, the system retrieves the poisoned document and uses it to generate misleading, biased, and unfaithful responses to user queries.

Poison Attacks and Adversarial Prompts Against an Informed University Virtual Assistant

Evaluated models: Barkplug V.2

Source: arXiv

LLM Data Poisoning Jailbreak

Published 8/1/2024

Analyzed 12/29/2024

Large Language Models (LLMs) are vulnerable to a novel attack paradigm, "jailbreak-tuning," which combines data poisoning with jailbreaking techniques to bypass existing safety safeguards. This allows malicious actors to fine-tune LLMs to reliably generate harmful outputs, even when trained on mostly benign data. The vulnerability is amplified in larger LLMs, which are more susceptible to learning harmful behaviors from even minimal exposure to poisoned data.

Data Poisoning in LLMs: Jailbreak-Tuning and Scaling Laws

Evaluated models: GPT-3.5 (GPT-3.5-turbo-0125), GPT-4, GPT-4o +3 more

Source: arXiv

Related research categories