Skip to main content
LLM Security Database
Skip to research details
Back to research findings

Logit-Based LLM Jailbreak

Published
Analyzed
Paper-reported evidence
Primary source linked
Read primary paper
Cite & share
Source BibTeX

Citation metadata is maintained by the primary source and may reflect a later revision.

Paper-evaluated models(5)

Gemma 7B IT, Llama 2 13B Chat, Llama 2 7B Chat +2 more
  • Gemma 7B IT
  • Llama 2 13B Chat
  • Llama 2 7B Chat
  • Llama 3 8B Instruct
  • Mistral 7B Instruct
On this page

Description

A vulnerability exists in several large language models (LLMs) allowing attackers to manipulate the models' output logits, biasing the probability distribution toward the generation of harmful content. The attack does not involve modifying the input prompt, but rather directly manipulates the internal probability scores assigned to output tokens during the generation process. By strategically increasing the logits of tokens forming a harmful response while decreasing those belonging to safety filters, an adversary can cause the LLM to produce undesired outputs.

Examples

See the paper "Lockpicking LLMs: A Logit-Based Jailbreak Using Token-level Manipulation" for detailed examples and algorithms. The attack requires white-box access (knowledge of the model's internal logits) and exploits the limited variety of refusal patterns observed in LLMs. Example attack prompts and resulting harmful outputs are included in the appendix of the linked paper.

Impact

Successful exploitation allows attackers to bypass LLMs' built-in safety mechanisms, resulting in the generation of harmful content, including but not limited to: hate speech, instructions for illegal activities, misinformation and disinformation campaigns, and personally identifiable information (PII) leaks. The impact is dependent on the specific model and how it is deployed.

Affected Systems

Multiple open-source LLMs, including Llama-2-7B-Chat, Llama-2-13B-Chat, Mistral-7B-Instruct, Gemma-7B-IT, and Llama-3-8B-Instruct were demonstrated to be vulnerable. Other LLMs with similar architectures may also be affected.

Mitigation Steps

  • Increase the diversity and complexity of denial patterns used by the model when rejecting harmful prompts.
  • Develop and implement robust internal mechanisms to detect and counteract logit manipulation attempts.
  • Implement stronger output filters designed to identify and block harmful content even after logit manipulation.
  • Regularly update and retrain models to account for emerging attack vectors.

Research context and provenance

Research topics
Model layer
Jailbreak
White-box
Safety
Integrity
Catalog identifier
LMVD-e13bc97d
Internal research identifier, not an official CVE identifier.
Evidence and verification
Paper-reported; independent reproduction is not documented.
Primary research source linked.
Severity
Not rated by this catalog.
Source and publication type
arXiv · Research preprint.
Peer-review status is not provided by this source.
Author and publication status
Author metadata is not stored; see the primary paper.
Threat model and attacker access
White-box access to model or deployment internals.
Related deployment categories
No related deployment category is classified.
Taxonomy labels only; paper-specific deployment prerequisites are not inferred.
Affected systems
Multiple open-source LLMs, including Llama-2-7B-Chat, Llama-2-13B-Chat, Mistral-7B-Instruct, Gemma-7B-IT, and Llama-3-8B-Instruct were demonstrated to be vulnerable. Other LLMs with similar architectures may also be…

Research Paper

Lockpicking LLMs: A Logit-Based Jailbreak Using Token-level Manipulation

Primary source: arXiv. Findings are reported by the cited research and have not been independently verified.

View Paper

Evidence

This entry is based on a primary research source. Its findings are paper-reported; independent reproduction and verification are not claimed.

https://arxiv.org/abs/2405.13068

Related research