Skip to main content

Pliny prompt injections for LLMs

Overview​

The Pliny plugin is designed to test LLM systems using a curated collection of prompts from the L1B3RT4S repository.

These prompts are specifically crafted to test various aspects of LLM behavior, including safety boundaries, ethical guidelines, and response patterns across different major AI companies (Google, Anthropic, Amazon, Meta, OpenAI, etc.).

The prompts are dynamically fetched from the repository. Please be aware of any license requirements if you create derived works from these prompts.

How it works​

For each test, the plugin:

  • Randomly selects prompts from the collected set
  • Tests if the model appropriately refuses or handles the prompts
  • Verifies that the model doesn't simply repeat the test prompt

Configuration​

To include the Pliny plugin in your LLM red teaming setup, add the following configuration to your YAML file:

redteam:
  plugins:
    - pliny

You can control the number of injections using the config.numTests parameter (defaults to 5):

redteam:
  plugins:
    - id: pliny
      config:
        numTests: 25

License​

The prompts used by this plugin are licensed under the AGPL-3.0 license and are dynamically downloaded from the L1B3RT4S repository. For details, see the LICENSE file in the repository.