LLM Provider Integrations

Provider picks the kitchen; Gateway decides whether the order travels by waiter, API, or carrier pigeon.

1. Motivation

HeavenBase separates where a model is served from how a request is transported. That keeps application intent stable while credentials, endpoints, and Gateway policy vary by environment.

provider owns model hosting, credentials, and base URL.
gateway owns the SDK or proxy transport shape.
model owns Provider-specific identifiers behind one canonical alias.

2. Use the Default Route

One OpenRouter key covers the broadest bundled model set:

export OPENROUTER_API_KEY="sk-or-v1-..."

import heavenbase as hb

llm = hb.LLM(preset="chat", cache=False)
text = llm.chat("Reply with exactly: hb-ok", max_tokens=8)

The default route is preset -> model -> openrouter -> openai. Here openai names the OpenAI-compatible Gateway, not necessarily the model host.

3. Choose a Hosted Provider

Provider id	Credential	Default endpoint family
`openrouter`	`OPENROUTER_API_KEY`	OpenRouter
`openai`	`OPENAI_API_KEY`	OpenAI
`anthropic`	`ANTHROPIC_API_KEY`	Anthropic
`grok`	`XAI_API_KEY`	xAI
`gemini`	`GEMINI_API_KEY`	Google OpenAI compatibility
`deepseek`	`DEEPSEEK_API_KEY`	DeepSeek
`moonshot`, `moonshot-zh`	`MOONSHOT_API_KEY`	Moonshot global or China
`zai`, `zai-zh`	`ZAI_API_KEY`	Z.ai global or China
`minimax`	`MINIMAX_API_KEY`	MiniMax
`cohere`	`COHERE_API_KEY`	Cohere embeddings
`voyage`	`VOYAGE_API_KEY`	Voyage embeddings
`dashscope`	`DASHSCOPE_API_KEY`	Alibaba DashScope

Set the default Provider when most calls should share one host:

export DEEPSEEK_API_KEY="sk-..."
hb cfg set heavenbase.llm.default_provider deepseek
hb cfg set heavenbase.llm.presets.chat.model deepseek-v4-flash

Pin one call when comparing routes:

fast = hb.LLM(model="ds-flash", provider="deepseek", cache=False)
claude = hb.LLM(model="haiku", provider="anthropic", cache=False)
grok = hb.LLM(model="grok", provider="grok", cache=False)

4. Choose a Gateway

Gateway	Transport
`openai`	OpenAI Python SDK against an OpenAI-compatible endpoint
`anthropic`	Official Anthropic SDK and native Messages payloads
`portkey`	Portkey routing, policy, and observability
`litellm`	LiteLLM Provider adapter ecosystem
`bifrost`	Bifrost-compatible Provider routing
`mock`	Deterministic offline adapter

export PORTKEY_API_KEY="..."

llm = hb.LLM(preset="chat", gateway="portkey", cache=False)

Use the native Anthropic Gateway when Claude Messages payloads or Anthropic SDK behavior matter:

llm = hb.LLM(model="haiku", provider="anthropic", gateway="anthropic")

5. Inspect a Route without Spending

The print Apply profile resolves a redacted route view without making a model call.

llm = hb.LLM(
    model="haiku",
    provider="anthropic",
    gateway="anthropic",
)
route = llm.spec.engine.apply(llm.spec, profile="print")

print(route["bindings"]["provider"]["base_url"])

The final URL comes from Provider config, Gateway config, or a call-time base_url=... override. The print profile redacts secrets. Raw spec.to_dict() bindings may contain them and must not be logged.

6. Run Local Providers

Provider id	Default URL	Typical use
`ollama`	`http://localhost:11434/v1`	Local chat, image generation, and OCR
`lmstudio`	`http://localhost:1234/v1`	Local chat and embeddings
`omlx`	`http://localhost:8000/v1`	Apple Silicon OpenAI-compatible serving
`vllm`	`http://localhost:8000/v1`	Self-hosted OpenAI-compatible serving

Packaged local Presets keep their Provider explicit:

local and worker-local use LM Studio with qwen3.6-flash.
embed-local uses LM Studio with embeddinggemma.
imagen-local uses Ollama with z-image-turbo.
ocr-local uses Ollama with glm-ocr.

Override the environment URL when the server runs elsewhere:

export LM_STUDIO_BASE_URL="http://127.0.0.1:1234/v1"

7. Handle Proxies Deliberately

Standard HTTP_PROXY, HTTPS_PROXY, and NO_PROXY values work for many SDK routes. Call-time values make the route visible in application code:

llm = hb.LLM(
    preset="chat",
    http_proxy="http://127.0.0.1:7890",
    https_proxy="http://127.0.0.1:7890",
    no_proxy="localhost,127.0.0.1",
)

Local Provider defaults clear inherited proxies so loopback calls do not accidentally leave the machine. Review materialized routes when VPN or TUN mode changes reachability.

Summary

Provider selects hosting; Gateway selects transport.
Canonical model aliases keep route changes out of application logic.
The redacted print Apply profile provides route evidence without a live request.
Local Presets pin the Provider that their runtime requires.

Further Exploration

Related resources:

First LLM — Configure credentials and run a first call
LLM Overview — Presets and the model catalog
Advanced LLM — Gateway limits, caching, and client exports

Introduction

Quickstart

Core Concepts

Developer

Utilities Toolbox

LLM

Agent Runtime

Community

LLM Provider Integrations

1. Motivation

2. Use the Default Route

3. Choose a Hosted Provider

4. Choose a Gateway

5. Inspect a Route without Spending

6. Run Local Providers

7. Handle Proxies Deliberately

Summary

Further Exploration

​1. Motivation

​2. Use the Default Route

​3. Choose a Hosted Provider

​4. Choose a Gateway

​5. Inspect a Route without Spending

​6. Run Local Providers

​7. Handle Proxies Deliberately

​Summary

​Further Exploration

1. Motivation

2. Use the Default Route

3. Choose a Hosted Provider

4. Choose a Gateway

5. Inspect a Route without Spending

6. Run Local Providers

7. Handle Proxies Deliberately

Summary

Further Exploration