Skip to main content

Google Vertex

The vertex provider is compatible with Google's Vertex AI offering, which offers access to models such as Gemini and Bison.

You can use it by specifying any of the available stable or latest model versions offered by Vertex AI. These include:

  • vertex:gemini-pro
  • vertex:gemini-ultra
  • vertex:chat-bison
  • vertex:chat-bison@001
  • vertex:chat-bison-32k
  • vertex:chat-bison-32k@001
  • vertex:codechat-bison
  • vertex:codechat-bison@001
  • vertex:codechat-bison-32k
  • vertex:codechat-bison-32k@001
tip

If you are using Google AI Studio, see the google provider.

Supported environment variables:

  • VERTEX_API_KEY (required) - gcloud API token. The easiest way to get an API key is to run gcloud auth print-access-token.
  • VERTEX_PROJECT_ID (required) - gcloud project ID
  • VERTEX_REGION - gcloud region, defaults to us-central1
  • VERTEX_PUBLISHER - model publisher, defaults to google
  • VERTEX_API_HOST - used to override the full Google API host, e.g. for an LLM proxy, defaults to {region}-aiplatform.googleapis.com

The Vertex provider also supports various configuration options such as context, examples, temperature, maxOutputTokens, and more, which can be used to customize the the behavior of the model like so:

providers:
- id: vertex:chat-bison-32k
config:
temperature: 0
maxOutputTokens: 1024