Skip to main content
LLM Security Database
Skip to research details
Back to research findings
LMVD-ID: b76b9647
Paper published May 1, 2024
Entry analyzed December 28, 2024
Paper-reported evidence
Confidence: Source-linked

The LMVD-ID is an internal research identifier, not an official CVE identifier.

Adaptive Sparse Jailbreak

A vulnerability in several open-source Large Language Models (LLMs) allows for efficient jailbreaking via Adaptive Dense-to-Sparse Constrained Optimization (ADC). This attack uses a continuous optimization method…

Jailbreak
Model layer
Black-box
Safety
Integrity
BibTeX citation

Paper-evaluated models(6)

GPT-3.5 Turbo, GPT-4, Llama2-chat-7B +3 more
  • GPT-3.5 Turbo
  • GPT-4
  • Llama2-chat-7B
  • Vicuna 7B v1.5
  • Zephyr 7B Beta
  • Zephyr 7B R2D2

Description

A vulnerability in several open-source Large Language Models (LLMs) allows for efficient jailbreaking via Adaptive Dense-to-Sparse Constrained Optimization (ADC). This attack uses a continuous optimization method, progressively increasing sparsity to generate adversarial token sequences that bypass safety measures and elicit harmful responses. The attack is more effective and efficient than prior token-level methods.

Examples

Specific examples of successful jailbreaks using the ADC method are not publicly available in this paper, but will be released in the associated code repository.

Impact

Successful exploitation allows an attacker to circumvent the safety mechanisms of affected LLMs, causing them to generate harmful, discriminatory, violent, or otherwise undesirable content, posing risks to users, organizations, and society.

Affected Systems

The vulnerability affects multiple open-source LLMs including, but not limited to: Llama2-chat-7B, Vicuna-v1.5-7B, Zephyr-7bβ, and Zephyr 7B R2D2. The paper suggests this method can also affect closed-source models, but no specific results are displayed.

Mitigation Steps

  • No specific mitigation steps are provided in the research paper. Further investigation is required to determine effective mitigations.

Research context and confidence

Evidence and verification
Paper-reported; independent reproduction is not documented.
Primary research source linked.
Severity
Not rated by this catalog.
Source and publication type
arXiv · Research preprint.
Peer-review status is not provided by this source.
Author and publication status
Author metadata is not stored; see the primary paper.
Threat model and attacker access
Black-box model, service, or application access.
Related deployment categories
No related deployment category is classified.
Taxonomy labels only; paper-specific deployment prerequisites are not inferred.
Affected systems
The vulnerability affects multiple open-source LLMs including, but not limited to: Llama2-chat-7B, Vicuna-v1.5-7B, Zephyr-7bβ, and Zephyr 7B R2D2. The paper suggests this method can also affect closed-source models…

Research Paper

Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimization

Primary source: arXiv. Findings are reported by the cited research and have not been independently verified.

View Paper

Evidence

This entry is based on a primary research source. Its findings are paper-reported; independent reproduction and verification are not claimed.

https://arxiv.org/abs/2405.09113

Related research