Find and fix LLM vulnerabilities
Open-source LLM testing used by 25,000+ developers
Automated pentesting for your app
Run an automatic scan tailored to your application that detects security, legal, and brand risks:
npx promptfoo@latest redteam initOur probes cover common failures like:
- PII leaks
- Insecure tool use
- Jailbreaks
- Harmful content
- Competitor endorsements
- Political statements
- Specialized medical and legal advice
- and much more
Trusted by developers at
Comprehensive security coverage
Custom probes for your application that identify failures you actually care about, not just generic jailbreaks and prompt injections.
Learn More
Built for developers
Move quickly with a command-line interface, live reloads, and caching. No SDKs, cloud dependencies, or logins.
Get Started
Battle-tested, 100% open-source
Used by teams serving millions of users and supported by an active open-source community.
View on GitHub
Easy abstractions for complex LLM testing
Simple declarative config
# Compare prompts...
prompts:
- "Summarize this in {{language}}: {{document}}"
- "Summarize this in {{language}}, concisely and professionally: {{document}}"
# And models...
providers:
- openai:gpt-4o
- anthropic:claude-3.5-sonnet
# ... using these tests
tests:
- vars:
language: French
document: "file://docs/*.txt"
assert:
- type: contains
value: "foo bar"
- type: llm-rubric
value: "does not apologize"
- type: cost
threshold: 0.01
- type: latency
threshold: 3000
- # ...
Detailed, actionable results
Detect & fix critical failures
