Overview
Complete guide to configuring promptfoo for LLM evaluation. Learn prompts, providers, test cases, assertions, and advanced features with examples.
Reference
Look up Promptfoo configuration fields for prompts, providers, tests, assertions, transforms, hooks, and eval options. See types and examples for each field.
Tool Calling
Configure tool definitions that work across OpenAI, Anthropic, AWS Bedrock, Google, and other LLM providers
Prompts
Configure prompts for LLM evaluation including text prompts, chat conversations, file-based prompts, and dynamic prompt generation with variables.
Test Cases
Configure test cases for LLM evaluation with variables, assertions, CSV data, and dynamic generation. Learn inline tests, external files, and media support.
Output Formats
Configure output formats for LLM evaluation results. Export to HTML, JSON, CSV, and YAML formats for analysis, reporting, and data processing.
Chat threads
Configure chat conversations and multi-turn threads for LLM evaluation. Learn conversation history, multi-shot prompts, and chat flow testing.
Dataset generation
Generate test cases from prompts or extend datasets with promptfoo generate dataset. Configure personas and review generated cases before running evals.
Scenarios
Configure scenarios to group test data with evaluation tests. Learn how to organize and run multiple test combinations efficiently in promptfoo.