Skip to main content

Azure

The azureopenai provider is an interface to OpenAI through Azure. It behaves the same as the OpenAI provider.

Setup

First, set the AZURE_OPENAI_API_KEY environment variable.

Next, edit the promptfoo configuration file to point to the Azure provider.

  • azureopenai:chat:<deployment name> - uses the given deployment (for chat endpoints such as gpt-35-turbo, gpt-4)
  • azureopenai:completion:<deployment name> - uses the given deployment (for completion endpoints such as gpt-35-instruct)

Also set the apiHost value to point to your endpoint:

providers:
- id: azureopenai:chat:deploymentNameHere
config:
apiHost: 'xxxxxxxx.openai.azure.com'

Additional config parameters are passed like so:

providers:
- id: azureopenai:chat:deploymentNameHere
config:
apiHost: 'xxxxxxxx.openai.azure.com'
temperature: 0.5
max_tokens: 1024
tip

All other OpenAI provider environment variables and configuration properties are supported.

Using client credentials

To use client credentials for authentication with Azure, you need to set the following configuration variables:

providers:
- id: azureopenai:chat:deploymentNameHere
config:
apiHost: 'xxxxxxxx.openai.azure.com'
azureClientId: 'your-azure-client-id'
azureClientSecret: 'your-azure-client-secret'
azureTenantId: 'your-azure-tenant-id'
azureAuthorityHost: 'https://login.microsoftonline.com' # Optional
azureTokenScope: 'https://cognitiveservices.azure.com/.default' # Optional

These credentials will be used to obtain an access token for the Azure OpenAI API.

The azureAuthorityHost defaults to 'https://login.microsoftonline.com' if not specified. The azureTokenScope defaults to 'https://cognitiveservices.azure.com/.default', the scope required to authenticate with Azure Cognitive Services.

You must also install a peer dependency from Azure:

npm i @azure/identity

Model-graded tests

Model-graded assertions such as factuality or llm-rubric use OpenAI by default. If you are using Azure, you must override the grader to point to your Azure deployment.

The easiest way to do this for all your test cases is to add the defaultTest property to your config:

defaultTest:
options:
provider:
id: azureopenai:chat:gpt-4-deployment-name
config:
apiHost: 'xxxxxxx.openai.azure.com'

However, you can also do this for individual assertions:

# ...
assert:
- type: llm-rubric
value: Do not mention that you are an AI or chat assistant
provider:
id: azureopenai:chat:xxxx
config:
apiHost: 'xxxxxxx.openai.azure.com'

Or individual tests:

# ...
tests:
- vars:
# ...
options:
provider:
id: azureopenai:chat:xxxx
config:
apiHost: 'xxxxxxx.openai.azure.com'
assert:
- type: llm-rubric
value: Do not mention that you are an AI or chat assistant

Similarity

The similar assertion type requires an embedding model such as text-embedding-ada-002. Be sure to specify a deployment with an embedding model, not a chat model, when overriding the grader.

AI Services

You may also specify deployment_id and dataSources, used to integrate with the Azure AI Search API.

providers:
- id: azureopenai:chat:deploymentNameHere
config:
apiHost: 'xxxxxxxx.openai.azure.com'
deployment_id: 'abc123'
dataSources:
- type: AzureCognitiveSearch
parameters:
endpoint: '...'
key: '...'
indexName: '...'

(The inconsistency in naming convention between deployment_id and dataSources reflects the actual naming in the Azure API.)

Configuration

These properties can be set under the provider config key`:

General config

NameDescription
apiHostAPI host.
apiBaseUrlBase URL of the API (used instead of host).
apiKeyAPI key.
apiVersionAPI version.

Azure-specific config

NameDescription
azureClientIdAzure identity client ID.
azureClientSecretAzure identity client secret.
azureTenantIdAzure identity tenant ID.
azureAuthorityHostAzure identity authority host.
azureTokenScopeAzure identity token scope.
deployment_idAzure cognitive services deployment ID.
dataSourcesAzure cognitive services parameter for specifying data sources.

OpenAI config:

NameDescription
temperatureControls randomness of the output.
top_pControls nucleus sampling.
frequency_penaltyPenalizes new tokens based on their frequency.
presence_penaltyPenalizes new tokens based on their presence.
best_ofGenerates multiple outputs and chooses the best.
functionsSpecifies functions available for use.
function_callControls automatic function calling.
response_formatSpecifies the format of the response.
stopSpecifies stop sequences for the generation.
passthroughAnything under passthrough will be sent as a top-level request param