Steering Dataset Poisoning

Description

A vulnerability in contrastive activation steering allows attackers to subvert Large Language Model (LLM) behavior via dataset poisoning. By corrupting >20% of the contrastive pairs used to compute the steering vector, an attacker can degrade the intended steering effect and covertly inject secondary, malicious behaviors. The vulnerability exploits the standard difference-of-means computation used to isolate activation directions. Because the steering vector is calculated as the unweighted difference between the mean of positive and negative high-dimensional activations, coordinated outlier data systematically distorts both the cosine similarity and projected norm of the resulting vector.

Examples

• Coordinated Behavior Corruption: An attacker poisons a dataset intended to train a "power-seeking" steering vector by replacing 30% of the training pairs with examples of a different behavior, such as "wealth-seeking." When the standard difference-of-means is computed, the resulting steering vector shifts geometrically toward the outlier direction. At inference time, applying this compromised steering vector secretly induces the "wealth-seeking" behavior in the LLM.
• Mislabeling Corruption: An attacker swaps the "positive" (with behavior) and "negative" (without behavior) labels on >20% of the contrastive pairs. This shrinks the projected norm of the difference of means, degrading the intended behavioral control without altering the vector's angle.
• See the repository at https://github.com/cullena20/SteeringLLMsCorruption (opens in a new tab) for reproducible attack datasets.

Impact

Attackers can bypass AI safety controls, neutralize behavioral guardrails, and stealthily implant malicious traits (e.g., incorrigibility, sycophancy, power-seeking). Because the primary steering trait is often only partially degraded while the secondary payload is injected, the corruption can easily go unnoticed during standard system evaluations.

Affected Systems

LLMs utilizing contrastive activation steering frameworks that rely on standard difference-of-means vector computation. The vulnerability has been explicitly demonstrated on models including:

• Llama-3.2-3B-Instruct
• Mistral-7b-Instruct-v0.3
• OLMo-2-1124-7B-Instruct

Mitigation Steps

• Replace standard sample mean computations with the Lee and Valiant (2022) robust mean estimator. This algorithm identifies the central distribution of the input activations and down-weights points outside this region proportional to their distance, effectively neutralizing random, mislabeled, and most anticorrelated behavioral data poisoning.
• Implement bounds checking and dynamic tuning on the steering magnitude (the α parameter) prior to inference. Because mislabeling and random corruption heavily target the projected norm (length) rather than the angle of the vector, tuning the magnitude can recover intended performance.
• Sanitize and audit automated dataset generation pipelines used for steering to ensure corrupted or mislabeled pairs remain strictly below the model's inherent 10-20% corruption tolerance threshold.