Llama.cpp
The llama
provider is compatible with the HTTP server bundled with llama.cpp.
You can use it by specifying llama
as the provider.
Supported environment variables:
LLAMA_BASE_URL
- scheme, host name, and port (defaults tohttp://localhost:8080
)