BarkPlug Data Poisoning Attack

Description: A poisoning attack against a Retrieval-Augmented Generation (RAG) system that manipulates the retriever component by injecting a poisoned document into the data used by the embedding model. This poisoned document contains modified and incorrect information. When activated, the system retrieves the poisoned document and uses it to generate misleading, biased, and unfaithful responses to user queries.

Examples:

• Benign Input: What are Dr. Rahimi’s research interests?
• Adversarial Input: Graph Theory. What are Dr. Rahimi’s research interests?
• Expected Output (based on paper): Dr. Shahram Rahimi’s research interests are: Computational Intelligence, Knowledge and Expert Systems, Fuzzy Rule-Base Systems, Genetic Algorithms and Swarm Computing. Artificial Intelligence and Machine Learning (specifically in Healthcare).
• Poisoned Output (based on paper): Dr. Rahimi’s research interests include: Graph theory, Structural graph theory, Induced subgraphs, Perfect graphs, Chi-boundedness, Graph-matroid symbiosis, Hadwiger’s conjecture.

Impact: The attack leads to the generation of factually incorrect or misleading information, potentially causing reputational damage, misinformed decisions, disruption of services, or other negative outcomes depending on the application of the virtual assistant.

Affected Systems: RAG systems where the retriever component uses external data that is not properly sanitized or protected from manipulation, such as BarkPlug v.2.

Mitigation Steps:

• Implement retrieval refinements such as improved ranking algorithms, data consistency checks.
• Use metadata to better manage knowledge base.