Zero-Shot Embedding Leak

Description

A Universal Zero-shot Embedding Inversion vulnerability exists in vector databases and embedding-based retrieval systems. The flaw allows an attacker to reconstruct original plaintext documents from their vector embeddings without requiring access to the original training data or training an embedding-specific inversion model. The attack, identified as "ZSinvert," leverages a multi-stage adversarial decoding process: (1) a cosine-similarity guided beam search using a Large Language Model (LLM) to generate candidate text sequences that maximize similarity to the target embedding, followed by (2) a universal, offline-trained correction model that refines the text for lexical accuracy. This method is effective across diverse encoder architectures (BERT, T5, Qwen) and remains effective against defenses employing Gaussian noise perturbation up to $\sigma=0.01$.

Examples

The following examples demonstrate the recovery of sensitive information from the Enron Email Dataset using the gte-Qwen encoder via the ZSinvert method.

Example 1: Internal Corporate Communication

• Target Embedding Source (Original): "Subject: Congratulations! Body: Congratulations on your promotion to MD! In addition to being a great personal achievement, your promotion helps to raise the"
• Inverted Text (Reconstructed): "Dear congratulations regarding promotion + Personal MD-mentioned - Great “Congratulations!!! On Achiecing an MD and Boost in Promising Directions!” Your Success Meams much!!"
• Observation: The attack successfully recovers the context ("promotion"), the specific role ("MD"), and the sentiment.

Example 2: Sensitive Business Context

• Target Embedding Source (Original): "Subject: Re: St. Lucie County Body: I’m printing as we speak. I’m so excited about that picture making money!"
• Inverted Text (Reconstructed): "Congratulations St Lucie printing. yes my picture making the money so exciting now!!!"
• Observation: The attack recovers specific locations ("St. Lucie"), actions ("printing"), and intent ("picture making money").

Impact

This vulnerability results in a critical loss of confidentiality for data stored in vector databases. An attacker who gains access to the embedding vectors (e.g., via a database dump, side-channel leak, or compromised API) can reconstruct the underlying sensitive plaintext, including PII, trade secrets, and private communications. The attack achieves a leakage rate above 80% on sensitive datasets and maintains high semantic fidelity (Cosine Similarity > 90) even when the encoder is treated as a black box.

Affected Systems

• Vector Databases storing text embeddings (e.g., used in RAG pipelines).
• Systems utilizing dense text retrieval models, specifically including but not limited to:
• Contriever (BERT-based)
• GTE (General Text Embeddings)
• GTR (Generalizable T5-based Retriever)
• LLM-based embedders (e.g., GTE-Qwen2-1.5B-instruct)

Mitigation Steps

• Data Classification: Treat embedding vectors with the equivalent security classification level as the raw plaintext documents they represent.
• Access Control: Do not expose embedding vectors to untrusted third-party services or public APIs. Apply strict Access Control Lists (ACLs) to vector databases.
• Ineffective Defenses: Do not rely on adding Gaussian noise (up to $\sigma=0.01$) to embeddings as a defense, as ZSinvert retains high inversion accuracy at noise levels that preserve retrieval utility.