Skip to main content

WatsonX

IBM WatsonX offers a range of enterprise-grade foundation models optimized for various business use cases. This provider supports several powerful models from the Granite and Llama series, along with additional models for code generation, multilingual tasks, vision processing, and more.

Supported Models​

  • Granite Series

    • granite-20b-multilingual
    • granite-34b-code-instruct
    • granite-20b-code-instruct
    • granite-8b-code-instruct
    • granite-3b-code-instruct
    • granite-8b-japanese
    • granite-7b-lab

  • Llama Series

    • llama-3-2-90b-vision-instruct
    • llama-3-2-11b-vision-instruct
    • llama-3-2-3b-instruct
    • llama-3-2-1b-instruct
    • llama-guard-3-11b-vision
    • llama-3-1-70b-instruct
    • llama-3-1-8b-instruct
    • llama3-llava-next-8b-hf
    • llama-3-405b-instruct
    • llama-3-70b-instruct
    • llama-3-8b-instruct

  • Additional Models

    • allam-1-13b-instruct
    • codellama-34b-instruct
    • elyza-japanese-llama-2-7b-instruct
    • flan-t5-xl-3b
    • flan-t5-xxl-11b
    • flan-ul2-20b
    • jais-13b-chat
    • llama2-13b-dpo-v7
    • mistral-large-2
    • mixtral-8x7b-instruct
    • mt0-xxl-13b

Prerequisites​

Before integrating the WatsonX provider, ensure you have the following:

  1. IBM Cloud Account: You will need an IBM Cloud account to obtain API access to WatsonX models.

  2. API Key or Bearer Token, and Project ID:

    • API Key: You can retrieve your API key by logging in to your IBM Cloud Account and navigating to the "API Keys" section.
    • Bearer Token: To obtain a bearer token, follow this guide.
    • Project ID: To find your Project ID, log in to IBM WatsonX Prompt Lab, select your project, and locate the project ID in the provided curl command.

Make sure you have either the API key or bearer token, along with the project ID, before proceeding.

Installation​

To install the WatsonX provider, use the following steps:

  1. Install the necessary dependencies:

    npm install @ibm-cloud/watsonx-ai ibm-cloud-sdk-core

  2. Set up the necessary environment variables:

    export WATSONX_AI_APIKEY=your-ibm-cloud-api-key

    or

    export WATSONX_AI_BEARER_TOKEN=your-ibm-cloud-bearer-token

    then

    export WATSONX_AI_PROJECT_ID=your-ibm-project-id

  3. Alternatively, you can configure the API key and project ID directly in the configuration file.

    providers:
      - id: watsonx:ibm/granite-13b-chat-v2
      config:
        apiKey: your-ibm-cloud-api-key # replace with apiBearerToken if you are using bearer token instead of apikey
        projectId: your-ibm-project-id
        serviceUrl: https://us-south.ml.cloud.ibm.com

Usage Examples​

Once configured, you can use the WatsonX provider to generate text responses based on prompts. Here’s an example of using the Granite 13B Chat V2 model to answer a question:

providers:
  - watsonx:ibm/granite-13b-chat-v2 # for Meta models, use watsonx:meta-llama/llama-3-2-1b-instruct

prompts:
  - "Answer the following question: '{{question}}'"

tests:
  - vars:
      question: 'What is the capital of France?'
    assert:
      - type: contains
        value: 'Paris'