# promptfoo

## Docs

- [Api Reference](/docs/api-reference/)
- [Configuration](/docs/category/configuration/)
- [Guides](/docs/category/guides/)
- [Integrations](/docs/category/integrations/)
- [Red Teaming](/docs/category/red-teaming/)
- [Usage](/docs/category/usage/)
- [Caching](/docs/configuration/caching/)
- [Chat](/docs/configuration/chat/)
- [Datasets](/docs/configuration/datasets/)
- [Expected Outputs](/docs/configuration/expected-outputs/)
- [Classifier](/docs/configuration/expected-outputs/classifier/)
- [Deterministic](/docs/configuration/expected-outputs/deterministic/)
- [Guardrails](/docs/configuration/expected-outputs/guardrails/)
- [Javascript](/docs/configuration/expected-outputs/javascript/)
- [Model Graded](/docs/configuration/expected-outputs/model-graded/)
- [Answer Relevance](/docs/configuration/expected-outputs/model-graded/answer-relevance/)
- [Context Faithfulness](/docs/configuration/expected-outputs/model-graded/context-faithfulness/)
- [Context Recall](/docs/configuration/expected-outputs/model-graded/context-recall/)
- [Context Relevance](/docs/configuration/expected-outputs/model-graded/context-relevance/)
- [Factuality](/docs/configuration/expected-outputs/model-graded/factuality/)
- [G Eval](/docs/configuration/expected-outputs/model-graded/g-eval/)
- [Llm Rubric](/docs/configuration/expected-outputs/model-graded/llm-rubric/)
- [Model Graded Closedqa](/docs/configuration/expected-outputs/model-graded/model-graded-closedqa/)
- [Select Best](/docs/configuration/expected-outputs/model-graded/select-best/)
- [Moderation](/docs/configuration/expected-outputs/moderation/)
- [Python](/docs/configuration/expected-outputs/python/)
- [Similar](/docs/configuration/expected-outputs/similar/)
- [Guide](/docs/configuration/guide/)
- [Parameters](/docs/configuration/parameters/)
- [Reference](/docs/configuration/reference/)
- [Scenarios](/docs/configuration/scenarios/)
- [Telemetry](/docs/configuration/telemetry/)
- [Testing Llm Chains](/docs/configuration/testing-llm-chains/)
- [Contributing](/docs/contributing/)
- [Enterprise](/docs/enterprise/)
- [Authentication](/docs/enterprise/authentication/)
- [Findings](/docs/enterprise/findings/)
- [Red Teams](/docs/enterprise/red-teams/)
- [Service Accounts](/docs/enterprise/service-accounts/)
- [Teams](/docs/enterprise/teams/)
- [Webhooks](/docs/enterprise/webhooks/)
- [Faq](/docs/faq/)
- [Getting Started](/docs/getting-started/)
- [Azure Vs Openai](/docs/guides/azure-vs-openai/)
- [Chatbase Redteam](/docs/guides/chatbase-redteam/)
- [Choosing Best Gpt Model](/docs/guides/choosing-best-gpt-model/)
- [Claude Vs Gpt](/docs/guides/claude-vs-gpt/)
- [Cohere Command R Benchmark](/docs/guides/cohere-command-r-benchmark/)
- [Compare Llama2 Vs Gpt](/docs/guides/compare-llama2-vs-gpt/)
- [Dbrx Benchmark](/docs/guides/dbrx-benchmark/)
- [Deepseek Benchmark](/docs/guides/deepseek-benchmark/)
- [Evaling With Harmbench](/docs/guides/evaling-with-harmbench/)
- [Evaluate Json](/docs/guides/evaluate-json/)
- [Evaluate Llm Temperature](/docs/guides/evaluate-llm-temperature/)
- [Evaluate Openai Assistants](/docs/guides/evaluate-openai-assistants/)
- [Evaluate Rag](/docs/guides/evaluate-rag/)
- [Evaluate Replicate Lifeboat](/docs/guides/evaluate-replicate-lifeboat/)
- [Factuality Eval](/docs/guides/factuality-eval/)
- [Gemini Vs Gpt](/docs/guides/gemini-vs-gpt/)
- [Gemma Vs Llama](/docs/guides/gemma-vs-llama/)
- [Gemma Vs Mistral](/docs/guides/gemma-vs-mistral/)
- [Gpt 3.5 Vs Gpt 4](/docs/guides/gpt-3.5-vs-gpt-4/)
- [Gpt 4 Vs Gpt 4o](/docs/guides/gpt-4-vs-gpt-4o/)
- [Gpt Vs O1](/docs/guides/gpt-vs-o1/)
- [Langchain Prompttemplate](/docs/guides/langchain-prompttemplate/)
- [Llama2 Uncensored Benchmark Ollama](/docs/guides/llama2-uncensored-benchmark-ollama/)
- [Llm Redteaming](/docs/guides/llm-redteaming/)
- [Mistral Vs Llama](/docs/guides/mistral-vs-llama/)
- [Mixtral Vs Gpt](/docs/guides/mixtral-vs-gpt/)
- [Multimodal Red Team](/docs/guides/multimodal-red-team/)
- [Phi Vs Llama](/docs/guides/phi-vs-llama/)
- [Prevent Llm Hallucations](/docs/guides/prevent-llm-hallucations/)
- [Qwen Benchmark](/docs/guides/qwen-benchmark/)
- [Sandboxed Code Evals](/docs/guides/sandboxed-code-evals/)
- [Testing Guardrails](/docs/guides/testing-guardrails/)
- [Text To Sql Evaluation](/docs/guides/text-to-sql-evaluation/)
- [Installation](/docs/installation/)
- [Burp](/docs/integrations/burp/)
- [Ci Cd](/docs/integrations/ci-cd/)
- [Circle Ci](/docs/integrations/circle-ci/)
- [Github Action](/docs/integrations/github-action/)
- [Gitlab Ci](/docs/integrations/gitlab-ci/)
- [Google Sheets](/docs/integrations/google-sheets/)
- [Helicone](/docs/integrations/helicone/)
- [Jenkins](/docs/integrations/jenkins/)
- [Jest](/docs/integrations/jest/)
- [Langfuse](/docs/integrations/langfuse/)
- [Mocha Chai](/docs/integrations/mocha-chai/)
- [Portkey](/docs/integrations/portkey/)
- [Python](/docs/integrations/python/)
- [Intro](/docs/intro/)
- [Model Audit](/docs/model-audit/)
- [Scanners](/docs/model-audit/scanners/)
- [Usage](/docs/model-audit/usage/)
- [Providers](/docs/providers/)
- [Adaline](/docs/providers/adaline/)
- [Ai21](/docs/providers/ai21/)
- [Alibaba](/docs/providers/alibaba/)
- [Anthropic](/docs/providers/anthropic/)
- [Aws Bedrock](/docs/providers/aws-bedrock/)
- [Azure](/docs/providers/azure/)
- [Browser](/docs/providers/browser/)
- [Cloudera](/docs/providers/cloudera/)
- [Cloudflare Ai](/docs/providers/cloudflare-ai/)
- [Cohere](/docs/providers/cohere/)
- [Custom Api](/docs/providers/custom-api/)
- [Custom Script](/docs/providers/custom-script/)
- [Databricks](/docs/providers/databricks/)
- [Deepseek](/docs/providers/deepseek/)
- [Echo](/docs/providers/echo/)
- [F5](/docs/providers/f5/)
- [Fal](/docs/providers/fal/)
- [Fireworks](/docs/providers/fireworks/)
- [Github](/docs/providers/github/)
- [Go](/docs/providers/go/)
- [Google](/docs/providers/google/)
- [Groq](/docs/providers/groq/)
- [Http](/docs/providers/http/)
- [Huggingface](/docs/providers/huggingface/)
- [Hyperbolic](/docs/providers/hyperbolic/)
- [Ibm Bam](/docs/providers/ibm-bam/)
- [Jfrog](/docs/providers/jfrog/)
- [Lambdalabs](/docs/providers/lambdalabs/)
- [Litellm](/docs/providers/litellm/)
- [Llama.cpp](/docs/providers/llama.cpp/)
- [Llamafile](/docs/providers/llamafile/)
- [Localai](/docs/providers/localai/)
- [Manual Input](/docs/providers/manual-input/)
- [Mistral](/docs/providers/mistral/)
- [Ollama](/docs/providers/ollama/)
- [Openai](/docs/providers/openai/)
- [Openllm](/docs/providers/openllm/)
- [Openrouter](/docs/providers/openrouter/)
- [Perplexity](/docs/providers/perplexity/)
- [Python](/docs/providers/python/)
- [Replicate](/docs/providers/replicate/)
- [Sagemaker](/docs/providers/sagemaker/)
- [Sequence](/docs/providers/sequence/)
- [Simulated User](/docs/providers/simulated-user/)
- [Text Generation Webui](/docs/providers/text-generation-webui/)
- [Togetherai](/docs/providers/togetherai/)
- [Vertex](/docs/providers/vertex/)
- [Vllm](/docs/providers/vllm/)
- [Voyage](/docs/providers/voyage/)
- [Watsonx](/docs/providers/watsonx/)
- [Webhook](/docs/providers/webhook/)
- [Websocket](/docs/providers/websocket/)
- [Xai](/docs/providers/xai/)
- [Red Team](/docs/red-team/)
- [Agents](/docs/red-team/agents/)
- [Architecture](/docs/red-team/architecture/)
- [Configuration](/docs/red-team/configuration/)
- [Foundation Models](/docs/red-team/foundation-models/)
- [Guardrails](/docs/red-team/guardrails/)
- [Llm Vulnerability Types](/docs/red-team/llm-vulnerability-types/)
- [Owasp Llm Top 10](/docs/red-team/owasp-llm-top-10/)
- [Plugins](/docs/red-team/plugins/)
- [Ascii Smuggling](/docs/red-team/plugins/ascii-smuggling/)
- [Beavertails](/docs/red-team/plugins/beavertails/)
- [Bfla](/docs/red-team/plugins/bfla/)
- [Bola](/docs/red-team/plugins/bola/)
- [Competitors](/docs/red-team/plugins/competitors/)
- [Context Compliance Attack](/docs/red-team/plugins/context-compliance-attack/)
- [Contracts](/docs/red-team/plugins/contracts/)
- [Cross Session Leak](/docs/red-team/plugins/cross-session-leak/)
- [Custom](/docs/red-team/plugins/custom/)
- [Cyberseceval](/docs/red-team/plugins/cyberseceval/)
- [Debug Access](/docs/red-team/plugins/debug-access/)
- [Divergent Repetition](/docs/red-team/plugins/divergent-repetition/)
- [Excessive Agency](/docs/red-team/plugins/excessive-agency/)
- [Hallucination](/docs/red-team/plugins/hallucination/)
- [Harmbench](/docs/red-team/plugins/harmbench/)
- [Harmful](/docs/red-team/plugins/harmful/)
- [Hijacking](/docs/red-team/plugins/hijacking/)
- [Imitation](/docs/red-team/plugins/imitation/)
- [Indirect Prompt Injection](/docs/red-team/plugins/indirect-prompt-injection/)
- [Intent](/docs/red-team/plugins/intent/)
- [Overreliance](/docs/red-team/plugins/overreliance/)
- [Pii](/docs/red-team/plugins/pii/)
- [Pliny](/docs/red-team/plugins/pliny/)
- [Policy](/docs/red-team/plugins/policy/)
- [Politics](/docs/red-team/plugins/politics/)
- [Prompt Extraction](/docs/red-team/plugins/prompt-extraction/)
- [Rag Document Exfiltration](/docs/red-team/plugins/rag-document-exfiltration/)
- [Rag Poisoning](/docs/red-team/plugins/rag-poisoning/)
- [Rbac](/docs/red-team/plugins/rbac/)
- [Reasoning Dos](/docs/red-team/plugins/reasoning-dos/)
- [Religion](/docs/red-team/plugins/religion/)
- [Shell Injection](/docs/red-team/plugins/shell-injection/)
- [Sql Injection](/docs/red-team/plugins/sql-injection/)
- [Ssrf](/docs/red-team/plugins/ssrf/)
- [System Prompt Override](/docs/red-team/plugins/system-prompt-override/)
- [Tool Discovery Multi Turn](/docs/red-team/plugins/tool-discovery-multi-turn/)
- [Tool Discovery](/docs/red-team/plugins/tool-discovery/)
- [Unsafebench](/docs/red-team/plugins/unsafebench/)
- [Quickstart](/docs/red-team/quickstart/)
- [Rag](/docs/red-team/rag/)
- [Strategies](/docs/red-team/strategies/)
- [Audio](/docs/red-team/strategies/audio/)
- [Base64](/docs/red-team/strategies/base64/)
- [Basic](/docs/red-team/strategies/basic/)
- [Best Of N](/docs/red-team/strategies/best-of-n/)
- [Citation](/docs/red-team/strategies/citation/)
- [Composite Jailbreaks](/docs/red-team/strategies/composite-jailbreaks/)
- [Custom](/docs/red-team/strategies/custom/)
- [Gcg](/docs/red-team/strategies/gcg/)
- [Goat](/docs/red-team/strategies/goat/)
- [Hex](/docs/red-team/strategies/hex/)
- [Image](/docs/red-team/strategies/image/)
- [Iterative](/docs/red-team/strategies/iterative/)
- [Leetspeak](/docs/red-team/strategies/leetspeak/)
- [Likert](/docs/red-team/strategies/likert/)
- [Math Prompt](/docs/red-team/strategies/math-prompt/)
- [Multi Turn](/docs/red-team/strategies/multi-turn/)
- [Multilingual](/docs/red-team/strategies/multilingual/)
- [Pandamonium](/docs/red-team/strategies/pandamonium/)
- [Prompt Injection](/docs/red-team/strategies/prompt-injection/)
- [Retry](/docs/red-team/strategies/retry/)
- [Rot13](/docs/red-team/strategies/rot13/)
- [Tree](/docs/red-team/strategies/tree/)
- [Attack Generation](/docs/red-team/troubleshooting/attack-generation/)
- [Connecting To Targets](/docs/red-team/troubleshooting/connecting-to-targets/)
- [False Positives](/docs/red-team/troubleshooting/false-positives/)
- [Inference Limit](/docs/red-team/troubleshooting/inference-limit/)
- [Multi Turn Sessions](/docs/red-team/troubleshooting/multi-turn-sessions/)
- [Multiple Response Types](/docs/red-team/troubleshooting/multiple-response-types/)
- [Overview](/docs/red-team/troubleshooting/overview/)
- [Remote Generation](/docs/red-team/troubleshooting/remote-generation/)
- [Command Line](/docs/usage/command-line/)
- [Node Package](/docs/usage/node-package/)
- [Self Hosting](/docs/usage/self-hosting/)
- [Sharing](/docs/usage/sharing/)
- [Troubleshooting](/docs/usage/troubleshooting/)
- [Web Ui](/docs/usage/web-ui/)