Skip to main content

Anthropic

This provider supports the Anthropic Claude series of models.

Note: Anthropic models can also be accessed through Amazon Bedrock. For information on using Anthropic models via Bedrock, please refer to our AWS Bedrock documentation.

Setup

To use Anthropic, you need to set the ANTHROPIC_API_KEY environment variable or specify the apiKey in the provider configuration.

Create Anthropic API keys here.

Example of setting the environment variable:

export ANTHROPIC_API_KEY=your_api_key_here

Models

The anthropic provider supports the following models via the messages API:

Model IDDescription
anthropic:messages:claude-3-5-sonnet-20241022 (claude-3-5-sonnet-latest)Latest Claude 3.5 Sonnet model
anthropic:messages:claude-3-5-sonnet-20240620Previous Claude 3.5 Sonnet model
anthropic:messages:claude-3-5-haiku-20241022 (claude-3-5-haiku-latest)Latest Claude 3.5 Haiku model
anthropic:messages:claude-3-5-haiku-20241022 (claude-3-5-haiku-latest)Latest Claude 3.5 Haiku model
anthropic:messages:claude-3-opus-20240229 (claude-3-opus-latest)Claude 3 Opus model
anthropic:messages:claude-3-sonnet-20240229Claude 3 Sonnet model
anthropic:messages:claude-3-haiku-20240307Claude 3 Haiku model

Cross-Platform Model Availability

Claude models are available across multiple platforms. Here's how the model names map across different providers:

ModelAnthropic APIAWS Bedrock (documentation)GCP Vertex AI (documentation)
Claude 3.5 Sonnetclaude-3-5-sonnet-20241022 (claude-3-5-sonnet-latest)anthropic.claude-3-5-sonnet-20241022-v2:0claude-3-5-sonnet-v2@20241022
Claude 3.5 Haikuclaude-3-5-haiku-20241022 (claude-3-5-haiku-latest)anthropic.claude-3-5-haiku-20241022-v1:0claude-3-5-haiku@20241022
Claude 3 Opusclaude-3-opus-20240229 (claude-3-opus-latest)anthropic.claude-3-opus-20240229-v1:0claude-3-opus@20240229
Claude 3 Sonnetclaude-3-sonnet-20240229anthropic.claude-3-sonnet-20240229-v1:0claude-3-sonnet@20240229
Claude 3 Haikuclaude-3-haiku-20240307anthropic.claude-3-haiku-20240307-v1:0claude-3-haiku@20240307

Supported Parameters

Config PropertyEnvironment VariableDescription
apiKeyANTHROPIC_API_KEYYour API key from Anthropic
apiBaseUrlANTHROPIC_BASE_URLThe base URL for requests to the Anthropic API
temperatureANTHROPIC_TEMPERATUREControls the randomness of the output (default: 0)
max_tokensANTHROPIC_MAX_TOKENSThe maximum length of the generated text (default: 1024)
top_p-Controls nucleus sampling, affecting the randomness of the output
top_k-Only sample from the top K options for each subsequent token
tools-An array of tool or function definitions for the model to call
tool_choice-An object specifying the tool to call
headers-Additional headers to be sent with the API request

Prompt Template

To allow for compatibility with the OpenAI prompt template, the following format is supported:

prompt.json
[
{
"role": "system",
"content": "{{ system_message }}"
},
{
"role": "user",
"content": "{{ question }}"
}
]

If the role system is specified, it will be automatically added to the API request. All user or assistant roles will be automatically converted into the right format for the API request. Currently, only type text is supported.

The system_message and question are example variables that can be set with the var directive.

Options

The Anthropic provider supports several options to customize the behavior of the model. These include:

  • temperature: Controls the randomness of the output.
  • max_tokens: The maximum length of the generated text.
  • top_p: Controls nucleus sampling, affecting the randomness of the output.
  • top_k: Only sample from the top K options for each subsequent token.
  • tools: An array of tool or function definitions for the model to call.
  • tool_choice: An object specifying the tool to call.

Example configuration with options and prompts:

promptfooconfig.yaml
providers:
- id: anthropic:messages:claude-3-5-sonnet-20241022
config:
temperature: 0.0
max_tokens: 512
prompts:
- file://prompt.json

Tool Use

The Anthropic provider supports tool use (or function calling). Here's an example configuration for defining tools:

promptfooconfig.yaml
providers:
- id: anthropic:messages:claude-3-5-sonnet-20241022
config:
tools:
- name: get_weather
description: Get the current weather in a given location
input_schema:
type: object
properties:
location:
type: string
description: The city and state, e.g., San Francisco, CA
unit:
type: string
enum:
- celsius
- fahrenheit
required:
- location

See the Anthropic Tool Use Guide for more information on how to define tools and the tool use example here.

Images / Vision

You can include images in the prompts in Claude 3 models.

See the Claude vision example.

One important note: The Claude API only supports base64 representations of images. This is different from how OpenAI's vision works, as it supports grabbing images from a URL. As a result, if you are trying to compare Claude 3 and OpenAI vision capabilities, you will need to have separate prompts for each.

See the OpenAI vision example to understand the differences.

Prompt Caching

Claude supports prompt caching to optimize API usage and reduce costs for repetitive tasks. This feature caches portions of your prompts to avoid reprocessing identical content in subsequent requests.

Supported on all Claude 3 and 3.5 models. Basic example:

promptfooconfig.yaml
providers:
- id: anthropic:messages:claude-3-5-sonnet-20241022
prompts:
- file://prompts.yaml
prompts.yaml
- role: system
content:
- type: text
text: 'System message'
cache_control:
type: ephemeral
- type: text
text: '{{context}}'
cache_control:
type: ephemeral
- role: user
content: '{{question}}'

Common use cases for caching:

  • System messages and instructions
  • Tool/function definitions
  • Large context documents
  • Frequently used images

See Anthropic's Prompt Caching Guide for more details on requirements, pricing, and best practices.

Citations

Claude can provide detailed citations when answering questions about documents. Basic example:

promptfooconfig.yaml
providers:
- id: anthropic:messages:claude-3-5-sonnet-20241022
prompts:
- file://prompts.yaml
prompts.yaml
- role: user
content:
- type: document
source:
type: text
media_type: text/plain
data: 'Your document text here'
citations:
enabled: true
- type: text
text: 'Your question here'

See Anthropic's Citations Guide for more details.

Model-Graded Tests

Model-graded assertions such as factuality or llm-rubric will automatically use Anthropic as the grading provider if ANTHROPIC_API_KEY is set and OPENAI_API_KEY is not set.

If both API keys are present, OpenAI will be used by default. You can explicitly override the grading provider in your configuration.

Because of how model-graded evals are implemented, the model must support chat-formatted prompts (except for embedding or classification models).

You can override the grading provider in several ways:

  1. For all test cases using defaultTest:
defaultTest:
options:
provider: anthropic:messages:claude-3-5-sonnet-20241022
  1. For individual assertions:
assert:
- type: llm-rubric
value: Do not mention that you are an AI or chat assistant
provider:
id: anthropic:messages:claude-3-5-sonnet-20241022
config:
temperature: 0.0
  1. For specific tests:
tests:
- vars:
question: What is the capital of France?
options:
provider:
id: anthropic:messages:claude-3-5-sonnet-20241022
assert:
- type: llm-rubric
value: Answer should mention Paris

Additional Capabilities

  • Caching: Promptfoo caches previous LLM requests by default.
  • Token Usage Tracking: Provides detailed information on the number of tokens used in each request, aiding in usage monitoring and optimization.
  • Cost Calculation: Calculates the cost of each request based on the number of tokens generated and the specific model used.

See Also

Examples

We provide several example implementations demonstrating Claude's capabilities:

Core Features

Model Comparisons & Evaluations

Cloud Platform Integrations

For more examples and general usage patterns, visit our examples directory on GitHub.