LLM provider integrations

Provider picks the restaurant. Gateway picks how you place the order.

HeavenBase separates where a model is served (provider) from how the request is sent (gateway). Most projects set one API key, keep the default openrouter provider, and never touch gateway config until they need Portkey, LiteLLM, or native Anthropic.

1. Default Route

For new projects, set one OpenRouter key and use presets:

export OPENROUTER_API_KEY="sk-or-v1-..."

import heavenbase as hb

llm = hb.LLM(preset="chat", cache=False)
text = llm.chat("Reply with exactly: hb-ok", max_tokens=8)

preset="chat" resolves to deepseek-v4-flash by default. The bundled default provider is openrouter, and the default gateway is openai, which means HeavenBase uses the OpenAI Python SDK against the provider’s OpenAI-compatible endpoint. See First LLM for the full provider catalog and per-key setup examples.

2. Switch Default Provider

Use default_provider when most calls should use one provider:

export OPENROUTER_API_KEY="sk-or-v1-..."
hb cfg set heavenbase.llm.default_provider openrouter

export DEEPSEEK_API_KEY="sk-..."
hb cfg set heavenbase.llm.default_provider deepseek
hb cfg set heavenbase.llm.presets.chat.model deepseek-v4-flash

export ANTHROPIC_API_KEY="sk-ant-..."
hb cfg set heavenbase.llm.default_provider anthropic
hb cfg set heavenbase.llm.presets.chat.model claude-sonnet-5

export ZAI_API_KEY="..."
hb cfg set heavenbase.llm.default_provider zai
hb cfg set heavenbase.llm.presets.chat.model glm-4.7-flash

hb cfg set heavenbase.llm.default_provider lmstudio
hb cfg set heavenbase.llm.presets.chat.model qwen3.6-flash

hb cfg set heavenbase.llm.default_provider omlx
hb cfg set heavenbase.llm.presets.chat.model qwen3.6-flash

The bundled local presets pin provider-specific defaults: local and worker-local use LM Studio with qwen3.6-flash, imagen-local uses Ollama with x/z-image-turbo, and ocr-local uses Ollama with glm-ocr. The omlx provider is available for direct OpenAI-compatible calls, LiteLLM (openai/<model>), Portkey custom-host routing, and Bifrost OpenAI-compatible provider routing. Use an explicit provider for one call when you are comparing routes:

fast = hb.LLM(model="ds-flash", provider="deepseek", cache=False)
claude = hb.LLM(model="haiku", provider="anthropic", cache=False)
glm = hb.LLM(model="glm-flash", provider="openrouter", cache=False)

3. Gateway Examples

Use gateway="portkey" when you want routing policy, observability, or gateway-side controls:

export PORTKEY_API_KEY="..."

import heavenbase as hb

llm = hb.LLM(preset="chat", gateway="portkey", cache=False)

For short live route checks, this is a cheap deterministic path:

llm = hb.LLM(preset="chat", gateway="portkey", cache=False, temperature=0)

Gateway	Use when
`openai`	You want direct OpenAI-compatible SDK calls. This is the default.
`anthropic`	You want the official Anthropic SDK and native Messages payloads.
`portkey`	You want routing, policy, observability, or hosted gateway behavior.
`litellm`	You already standardize provider routing through LiteLLM model names.
`bifrost`	You run a Bifrost-compatible OpenAI endpoint.
`mock`	You need offline deterministic tests.

Gateway setup walkthroughs live in First LLM §5.2 and Advanced LLM.

4. Anthropic Gateway Decision

HeavenBase supports two Anthropic routes. Use the OpenAI SDK compatibility endpoint for quick Claude checks and comparisons with other OpenAI-compatible providers:

import heavenbase as hb

llm = hb.LLM(model="haiku", provider="anthropic", gateway="openai", cache=False)
print(llm.spec.materialize()["base_url"])

Use the native gateway when you want the official Anthropic Python SDK and the Claude Messages format:

llm = hb.LLM(model="haiku", provider="anthropic", gateway="anthropic", cache=False)
print(llm.spec.materialize()["base_url"])

Current routing is preset -> model -> provider -> gateway, and the final URL comes from provider config, gateway config, or a call-time base_url=... override. The native gateway converts HeavenBase’s internal OpenAI-style message/tool history into Anthropic Messages payloads, maps canonical think and reasoning_effort controls to Anthropic thinking fields, then normalizes the response back to the usual include fields. Proxy support fits that model:

Portkey supports OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages gateway formats.
LiteLLM supports Anthropic through the anthropic/ provider route and /v1/messages passthrough.
Bifrost supports Anthropic through OpenAI-compatible conversion to /v1/messages and through provider-compatible Anthropic SDK endpoints.

Use materialize() when you need route evidence without live Claude spend:

runtime = hb.LLM(model="haiku", provider="anthropic", gateway="anthropic").spec.materialize()
print(runtime["base_url"])

5. GLM and Z.ai Route Checks

For GLM tool-call validation, keep native OpenAI JSON tools as the default. Start with glm-flash through OpenRouter via Portkey when you want a broadly compatible route.

runtime = hb.LLM(model="glm-flash", provider="zai", gateway="portkey").spec.materialize()
print(runtime["model"])

6. Proxy and VPN Notes

Local live checks can use provider keys, base URL overrides, and simple proxy exports from ~/.bashrc, including HTTP_PROXY, HTTPS_PROXY, and NO_PROXY. Use those exports when VPN/TUN mode changes GLM, Anthropic, or OpenRouter reachability. You can also pass proxy settings directly:

llm = hb.LLM(
    preset="chat",
    http_proxy="http://127.0.0.1:7890",
    https_proxy="http://127.0.0.1:7890",
    no_proxy="localhost,127.0.0.1",
)

Further Exploration

Related resources:

First LLM — configure keys and run the first chat.
LLM Overview — presets, models, and the resolution model.
Advanced LLM — gateway depth, caching, and upstream limitations.

​1. Default Route

​2. Switch Default Provider

​3. Gateway Examples

​4. Anthropic Gateway Decision

​5. GLM and Z.ai Route Checks

​6. Proxy and VPN Notes

​Further Exploration

1. Default Route

2. Switch Default Provider

3. Gateway Examples

4. Anthropic Gateway Decision

5. GLM and Z.ai Route Checks

6. Proxy and VPN Notes

Further Exploration