Skip to main content

Portkey AI integration

Portkey is an AI observability suite that includes prompt management capabilities.

To reference prompts in Portkey:

  1. Set the PORTKEY_API_KEY environment variable.

  2. Use the portkey:// prefix for your prompts, followed by the Portkey prompt ID. For example:

    prompts:
      - 'portkey://pp-test-promp-669f48'

    providers:
      - openai:gpt-3.5-turbo-0613

    tests:
      - vars:
          topic: ...

Variables from your promptfoo test cases will be automatically plugged into the Portkey prompt as variables. The resulting prompt will be rendered and returned to promptfoo, and used as the prompt for the test case.

Note that promptfoo does not follow the temperature, model, and other parameters set in Portkey. You must set them in the providers configuration yourself.