Query-Agnostic Poisoning of Policy-Aware RAG Knowledge Bases

Description

The paper describes a specific, reproducible retrieval-layer poisoning issue: an adversary with knowledge-base write access can add semantically broad, cross-category policy rules that rank in top-k retrieval across varied queries without knowing runtime prompts. In the authors' isolated PA-LLM-RAG evaluation, one injected rule caused at least one poisoned rule to enter the LLM context for 17/20 prompts. Safe defensive reproduction should use a synthetic policy KB, synthetic benign prompts, and measure whether injected test documents enter retrieved context; do not connect testing to operational systems. All measurements are paper-reported and were not independently verified.

Examples

Defensive reference: reproduce the paper's evaluation design in an isolated synthetic environment using its 60-rule, three-category KB structure, top-3 cosine retrieval, 20 benign mission prompts, and context-integrity metric; compare clean retrieval against controlled synthetic poison insertion without operational payloads.

Impact

Successful poisoning can corrupt the policy context supplied to a RAG decision model, causing incorrect or policy-violating outputs. In the evaluated simulated IoBT setting, this could degrade mission reliability and potentially create safety consequences; real-world impact was not demonstrated, and the authors note validation was limited to one Gazebo simulation scenario.

Affected Systems

• Policy-Aware LLM-RAG (PA-LLM-RAG) framework
• RAG systems whose authoritative policy or knowledge base can be written by an adversary
• Structured, taxonomy-based RAG knowledge bases using semantic vector retrieval
• The evaluated setup reports the Llama 3.2 family without a checkpoint size, so that generation-only label is excluded from model facets.

Mitigation Steps

• Restrict, authenticate, and audit all knowledge-base write and synchronization paths; treat upstream content as untrusted until validated.
• Run anomaly detection on candidate knowledge-base entries before they enter LLM context, including both distribution-boundary detection and taxonomy-aware semantic checks where categories exist.
• Use filter-and-refill retrieval: inspect a larger ranked candidate pool, block suspicious entries, and pass only verified clean documents to the LLM.
• Maintain an immutable or versioned verified knowledge-base baseline and compare retrieved rules against approved content.
• Quarantine and review flagged entries rather than relying solely on automatic permanent deletion, especially in dynamic knowledge bases.
• Continuously retrain and recalibrate detectors as legitimate knowledge-base content changes, and test against paraphrased, single-category, and adaptive poisoning variants.

Evidence

• Primary source and publication metadata: https://arxiv.org/abs/2607.04379 — arXiv records submission on 2026-07-05 and identifies the paper as an adversarial study of PA-LLM-RAG.
• https://arxiv.org/html/2607.04379 — Section IV-A defines a white-box adversary with KB write access, knowledge of nomic-embed-text and the WF/ROE/CAP taxonomy, but no prompt interception or LLM-weight access.
• https://arxiv.org/html/2607.04379 — Sections IV-B and IV-C formalize the objective as making an injected rule appear in top-k retrieval for arbitrary queries and describe query-agnostic construction around the query-distribution centroid.
• https://arxiv.org/html/2607.04379 — Section IV-E reports that one injected rule at 1.6% poisoning entered context for 17/20 evaluated prompts (85%); this is a paper-reported result, not independently verified.
• https://arxiv.org/html/2607.04379 — Section VI-A reports the setup: 60 legitimate rules, 24 generated adversarial rules, Llama 3.2, nomic-embed-text, Gazebo Harmonic simulation, 20 prompts, and three runs per poisoning level.
• https://arxiv.org/html/2607.04379 — Section VI-B reports saturation at about 2.65 poisoned rules per context at 7.7% poisoning; this is a paper-reported measurement.
• https://arxiv.org/html/2607.04379 — Sections V-A through V-D describe CLD-KB: One-Class SVM plus Member-Based Category Spread, combined with filter-and-refill before documents reach the LLM.
• https://arxiv.org/html/2607.04379 — Sections VI-C.1 through VI-C.3 report CLD-KB recall, precision, and F1 of 1.000, 100% context integrity, zero false positives, and approximately 7 ms overhead; these results were not independently verified.
• https://arxiv.org/html/2607.04379 — Section VII limits generalizability: the defense depends on a three-category taxonomy, uses only 60 legitimate rules, covers one simulated mission scenario, and lacks hardware-in-the-loop or real-world validation.