First LLM - HeavenBase

No one wants to see output = client.chat.completions.create(...).choices[0].message.content.

HeavenBase routes LLM calls through an OpenAI-compatible gateway. The default provider is OpenRouter, giving you access to most major models with a single API key. Model choices never appear in your code. You reference presets — normal chat presets choose the model and behavior, while heavenbase.llm.default_provider chooses where that model is served. Explicit provider= arguments and provider pins on a preset still override the default provider.

1. Set an API Key

HeavenBase ships with built-in configs for these providers. Set the relevant environment variables or use hb cfg set:

# Single API key for OpenAI, Anthropic, Google, DeepSeek, and 200+ models.
# Default to using <OPENROUTER_API_KEY> environment variable.
export OPENROUTER_API_KEY="sk-or-v1-..."
hb cfg set heavenbase.llm.default_provider openrouter   # default
hb cfg set heavenbase.llm.providers.openrouter.base_url "https://openrouter.ai/api/v1"  # default
hb cfg set heavenbase.llm.presets.chat.model deepseek-v4-flash   # default model for chat preset

# Standard OpenAI endpoint. Always available for OpenAI-compatible providers.
# Default to using <OPENAI_API_KEY> environment variable.
export OPENAI_API_KEY="sk-..."
hb cfg set heavenbase.llm.default_provider openai
hb cfg set heavenbase.llm.providers.openai.base_url "https://api.openai.com/v1"  # default
hb cfg set heavenbase.llm.presets.chat.model gpt-5.5

# Anthropic offers an OpenAI SDK compatibility endpoint for quick Claude checks.
# Use Portkey/LiteLLM/Bifrost when you need gateway policy or routing.
# Default to using <ANTHROPIC_API_KEY> environment variable.
export ANTHROPIC_API_KEY="sk-ant-..."
hb cfg set heavenbase.llm.default_provider anthropic
hb cfg set heavenbase.llm.providers.anthropic.base_url "https://api.anthropic.com/v1"  # default
hb cfg set heavenbase.llm.presets.chat.model claude-sonnet-5

# Default to using <GEMINI_API_KEY> environment variable.
export GEMINI_API_KEY="AIza..."
hb cfg set heavenbase.llm.default_provider gemini
hb cfg set heavenbase.llm.providers.gemini.base_url "https://generativelanguage.googleapis.com/v1beta/openai"  # default
hb cfg set heavenbase.llm.presets.chat.model gemini-3.5-flash

# xAI exposes an OpenAI-compatible endpoint.
# Default to using <XAI_API_KEY> environment variable.
export XAI_API_KEY="xai-..."
hb cfg set heavenbase.llm.default_provider grok
hb cfg set heavenbase.llm.providers.grok.base_url "https://api.x.ai/v1"  # default
hb cfg set heavenbase.llm.presets.chat.model grok-4.3

# Default to using <DEEPSEEK_API_KEY> environment variable.
export DEEPSEEK_API_KEY="sk-..."
hb cfg set heavenbase.llm.default_provider deepseek
hb cfg set heavenbase.llm.providers.deepseek.base_url "https://api.deepseek.com/v1"  # default
hb cfg set heavenbase.llm.presets.chat.model deepseek-v4-flash

# Global: https://api.moonshot.ai/v1 (moonshot)
# China: https://api.moonshot.cn/v1 (moonshot-zh)
# Default to using <MOONSHOT_API_KEY> environment variable.
export MOONSHOT_API_KEY="sk-..."
hb cfg set heavenbase.llm.default_provider moonshot        # global
# hb cfg set heavenbase.llm.default_provider moonshot-zh   # china
hb cfg set heavenbase.llm.presets.chat.model kimi-k2.6

# Global: https://api.z.ai/api/paas/v4 (zai)
# China: https://open.bigmodel.cn/api/paas/v4 (zai-zh)
# Default to using <ZAI_API_KEY> environment variable.
export ZAI_API_KEY="..."
hb cfg set heavenbase.llm.default_provider zai          # global
# hb cfg set heavenbase.llm.default_provider zai-zh     # china
hb cfg set heavenbase.llm.presets.chat.model glm-4.7-flash

# OpenAI-compatible endpoint at https://api.minimax.io/v1
# Default to using <MINIMAX_API_KEY> environment variable.
export MINIMAX_API_KEY="..."
hb cfg set heavenbase.llm.default_provider minimax
hb cfg set heavenbase.llm.presets.chat.model MiniMax-M2.7

# Default to using <DASHSCOPE_API_KEY> environment variable.
export DASHSCOPE_API_KEY="sk-..."
hb cfg set heavenbase.llm.default_provider dashscope
hb cfg set heavenbase.llm.providers.dashscope.base_url "https://dashscope.aliyuncs.com/compatible-mode/v1"  # default
hb cfg set heavenbase.llm.presets.chat.model qwen3.6-flash

# Local endpoint default at http://localhost:11434/v1. No API key needed.
hb cfg set heavenbase.llm.default_provider ollama
hb cfg set heavenbase.llm.presets.chat.model qwen3.6-flash

# Local endpoint default at http://localhost:1234/v1. No API key needed.
hb cfg set heavenbase.llm.default_provider lmstudio
hb cfg set heavenbase.llm.presets.chat.model qwen3.6-flash

# Local endpoint default at http://localhost:8000/v1. No API key needed.
hb cfg set heavenbase.llm.default_provider omlx
hb cfg set heavenbase.llm.presets.chat.model qwen3.6-flash

# Local endpoint default at http://localhost:8000/v1. No API key needed.
hb cfg set heavenbase.llm.default_provider vllm
hb cfg set heavenbase.llm.presets.chat.model qwen3.6-flash

If you have an older config that pins a preset provider, unset it so default_provider can take effect:

hb cfg unset heavenbase.llm.presets.chat.provider

You can also manually set hb cfg set heavenbase.llm.providers.<provider>.base_url for any OpenAI-compatible provider, and hb cfg set heavenbase.llm.providers.<provider>.api_key if the provider doesn’t use the standard environment variable. Local providers (ollama, lmstudio, omlx, and vllm) accept provider-specific model IDs too; the curated qwen3.6-flash entry maps to each local provider’s configured name. For provider-pinned local defaults, use preset="local" or preset="worker-local" for LM Studio qwen3.6-flash, preset="imagen-local" for Ollama x/z-image-turbo, and preset="ocr-local" for Ollama glm-ocr. Any LiteLLM-compatible provider can be added via config. See the LLM overview for the complete provider and model catalog.

2. CLI Chat

Send a message. By default this uses the chat preset with deepseek-v4-flash on your configured default_provider (openrouter out of the box):

$ hb llm chat "What is a data gateway?"
# A data gateway is a middleware layer that sits between applications
# and their data sources, providing unified access, routing, caching,
# and security policies without coupling clients to specific backends...

Override the preset with --preset:

$ hb llm chat --preset system "Name three data backends"
# SQLite, PostgreSQL, and Elasticsearch.

Options:

--preset / -p: use a named preset (system, chat, reason, coder, local, worker-local, embed, embed-local, imagen-local, ocr-local, …)
--model / -m: override the model (ds-flash, sonnet, gpt, … support canonical model names or defined aliases)
--provider / -b: override the provider (openrouter, openai, anthropic, …)
--verbose / -v: show the resolved LLM spec including model, provider, gateway, args, etc.
--input / -i: read the prompt from a file

The chat preset is used by hb llm chat and hb llm session. The default hb.LLM() (no preset) uses system, a lightweight preset for short orchestration calls. The embed preset controls hb llm embed and uses text-embedding-3-small; set heavenbase.llm.presets.embed.provider when your default provider does not serve embeddings.

3. CLI Embeddings

The embed preset defaults to text-embedding-3-small (1536 dimensions). It uses your default_provider unless you pin an embedding provider. Configuring an embedding provider is similar to chat, just set the embed preset’s provider and model.

export OPENROUTER_API_KEY="sk-or-v1-..."
hb cfg set heavenbase.llm.presets.embed.provider openrouter
hb cfg set heavenbase.llm.presets.embed.model openai/text-embedding-3-small

export OPENAI_API_KEY="sk-..."
hb cfg set heavenbase.llm.presets.embed.provider openai
hb cfg set heavenbase.llm.presets.embed.model text-embedding-3-small

export COHERE_API_KEY="..."
hb cfg set heavenbase.llm.presets.embed.provider cohere
hb cfg set heavenbase.llm.presets.embed.model embed-v4.0

export VOYAGE_API_KEY="pa-..."
hb cfg set heavenbase.llm.presets.embed.provider voyage
hb cfg set heavenbase.llm.presets.embed.model voyage-4-lite

hb cfg set heavenbase.llm.presets.embed.provider ollama
hb cfg set heavenbase.llm.presets.embed.model embeddinggemma

hb cfg set heavenbase.llm.presets.embed.provider lmstudio
hb cfg set heavenbase.llm.presets.embed.model text-embedding-embeddinggemma-300m

hb cfg set heavenbase.llm.presets.embed.provider omlx
hb cfg set heavenbase.llm.presets.embed.model embeddinggemma

It returns the full raw embedding array:

$ hb llm embed "Semantic text to embed"
# [0.0123456789, -0.0098765432, 0.0456789123, ...]

Pass --preview when you only want a compact inspection view. Preview mode shows the first 4 and last 2 values, rounded to 6 decimals:

$ hb llm embed "Semantic text to embed" --preview
# [0.012346, -0.009877, 0.045679, -0.023457, ..., -0.015991, -0.017471]

Pass --json for the full response object including usage. --json --preview keeps the same keys but replaces embeddings with the abbreviated string:

hb llm embed "hello" --json
# {
#     "embeddings": [
#         0.01675415,
#         ...
#         -0.0159912109375,
#         -0.0174713134765625
#     ],
#     "dim": 1536,
#     "usage": {
#         "prompt_tokens": 1,
#         "total_tokens": 1,
#         "cost": 2e-08
#     }
# }

hb llm embed "hello" --json --preview
# {
#     "embeddings": "[0.016754, -0.004312, 0.008901, 0.002211, ..., -0.015991, -0.017471]",
#     "dim": 1536,
#     "usage": {
#         "prompt_tokens": 1,
#         "total_tokens": 1,
#         "cost": 2e-08
#     }
# }

Use --copy / -cp on hb llm chat or hb llm embed when you want the emitted response copied to the clipboard.

4. Interactive Session

Start a multi-turn session:

hb llm session

The session uses the chat preset. Type messages at the >>> prompt. Example:

$ hb llm session
# Session started. Type /help for commands, /bye or /exit to quit.
# >>> What is a data gateway?
# A data gateway is a middleware layer that provides unified access
# to multiple data sources with routing and caching.
# >>> List three use cases
# 1. Multi-database query routing
# 2. Read/write splitting
# 3. API gateway for vector and structured data
# >>> /bye

Slash commands: /help, /save <path>, /load <path>, /clear, /regen <seed>, /back, /tools, /mcp, /exit. For attaching MCP tools in sessions, see the First MCP chapter in the quickstart.

5. Python API

5.1. The `LLM` Class

hb.LLM is the Python API behind the hb llm CLI. Construct one with no arguments to use the system preset, or pass a preset, model, provider, gateway, or request defaults:

import heavenbase as hb

# Default: the system preset.
llm = hb.LLM()

# Common overrides.
reasoning = hb.LLM(preset="reason")  # deepseek-v4-pro

# Route explicitly through Anthropic's native gateway.
claude = hb.LLM(model="sonnet", provider="anthropic", gateway="anthropic")

# Set request defaults on the instance.
chat = hb.LLM(preset="chat", max_tokens=256, temperature=0.0)

# Disable response caching when you need a fresh provider call.
uncached = hb.LLM(cache=False)

The LLM constructor accepts preset, model, provider, gateway, and extra keyword arguments such as temperature, max_tokens, or seed. Extra keyword arguments become provider request defaults. For normal chat presets, the model comes from heavenbase.llm.presets.<name>.model, while the provider comes from heavenbase.llm.default_provider unless the preset explicitly pins a provider.

Then call the method that matches the operation:

llm = hb.LLM(preset="chat")   # Default model: deepseek-v4-flash (DeepSeek through OpenRouter)
text = llm.chat("Reply with exactly: hello world")
print(text)
# hello world

text = llm.chat([
    {"role": "system", "content": "Whatever number the user asks for, reply with negative 42."},
    {"role": "user", "content": "Give me a random number between 0 and 100, output the number only."}
])  # OpenAI-format messages list
print(text)
# -42

llm = hb.LLM(preset="chat")   # Default model: deepseek-v4-flash (DeepSeek through OpenRouter)
for chunk in llm.stream("Explain embeddings in one sentence."):
    print(chunk, end="", flush=True)
print()

llm = hb.LLM(preset="chat")   # Default model: deepseek-v4-flash (DeepSeek through OpenRouter)
resp = llm.chat(
    "What is the answer to life, the universe, and everything?",
    include=["text", "usage", "elapsed"],
)
# More fields are available through `include`; see the LLM reference.

print(resp["text"])
# The answer is 42. In Douglas Adams' *The Hitchhiker's Guide to the Galaxy*, ...
print(resp["usage"])
# {'completion_tokens': 96, 'prompt_tokens': 21, 'total_tokens': 117, 'completion_tokens_details': ..., ...}
print(resp["elapsed"], "s")
# 2.8803 s

llm = hb.LLM(preset="embed")   # Default model: text-embedding-3-small (OpenAI through OpenRouter)
vec = llm.embed("Semantic text")
print(vec[:5])      # first 5 values
print(len(vec))     # dimension (model-dependent, actual result)
print(llm.dim)      # dimension from LLM property, fallback to actual result if not set by the model

vecs = llm.embed(["hello world", "foo bar"])
print(len(vecs), len(vecs[0]))  # 2 1536

llm = hb.LLM(preset="imagen")   # Default model: gpt-5-image-mini (OpenAI through OpenRouter)
img = llm.imagen("The quick brown fox jumps over the lazy dog")
img.save("fox.png")
img.to_pil().show()

local_img = hb.LLM(preset="imagen-local").imagen("A small red fox icon")  # Ollama x/z-image-turbo

5.2. LLM Gateways

A gateway is the transport adapter HeavenBase uses after it resolves the preset, model, and provider. The provider decides where the model is served; the gateway decides how the request is sent. HeavenBase supports six gateway keys:

Gateway	Use when
`openai`	You want the OpenAI Python SDK against an OpenAI-compatible endpoint. This is the default.
`anthropic`	You want the official Anthropic Python SDK and native Messages payloads.
`portkey`	You want Portkey routing, policy, observability, or gateway-side controls.
`bifrost`	You run a Bifrost-compatible gateway and want provider-prefixed model routing.
`litellm`	You already standardize provider routing through LiteLLM’s Python gateway.
`mock`	You need deterministic offline behavior for tests and demos.

heavenbase.llm.default_gateway defaults to openai. It is used only when the call, preset, or provider does not pin a gateway. Set it globally with hb cfg set heavenbase.llm.default_gateway <gateway>, or override it for one instance with hb.LLM(..., gateway="..."). The chat preset resolves to deepseek-v4-flash through the openrouter provider unless you change the preset’s model or provider. Because OpenRouter exposes an OpenAI-compatible API, the default openai gateway can use hb.LLM(preset="chat") directly.

# Default path: OpenRouter serves deepseek-v4-flash through an OpenAI-compatible API.
export OPENROUTER_API_KEY="sk-or-v1-..."

hb cfg set heavenbase.llm.default_gateway openai
hb cfg set heavenbase.llm.default_provider openrouter
hb cfg set heavenbase.llm.providers.openrouter.base_url "https://openrouter.ai/api/v1"

# Native Anthropic uses the Messages API, so switch the chat preset to a Claude model and provider.
pip install anthropic

export ANTHROPIC_API_KEY="sk-ant-..."
hb cfg set heavenbase.llm.default_gateway anthropic
hb cfg set heavenbase.llm.default_provider anthropic
hb cfg set heavenbase.llm.presets.chat.model claude-sonnet-5

# Run Portkey locally. Docker works too:
# docker run --rm -p 8787:8787 portkeyai/gateway:latest
npx -y @portkey-ai/gateway

# Keep the chat preset on OpenRouter, but send requests through Portkey.
export OPENROUTER_API_KEY="sk-or-v1-..."
hb cfg set heavenbase.llm.default_gateway portkey
hb cfg set heavenbase.llm.gateways.portkey.base_url "http://localhost:8787/v1"

# For hosted Portkey, also set PORTKEY_API_KEY and use https://api.portkey.ai/v1.

# Bifrost needs provider configuration before it can route a model.
mkdir -p .bifrost
cat > .bifrost/config.json <<'JSON'
{
    "$schema": "https://www.getbifrost.ai/schema",
    "encryption_key": "env.BIFROST_ENCRYPTION_KEY",
    "client": {"drop_excess_requests": false},
    "providers": {
        "openrouter": {
            "keys": [
                {"name": "openrouter-primary", "value": "env.OPENROUTER_API_KEY", "models": ["*"], "weight": 1.0}
            ]
        }
    },
    "config_store": {"enabled": false}
}
JSON

export OPENROUTER_API_KEY="sk-or-v1-..."
export BIFROST_ENCRYPTION_KEY="0123456789abcdef0123456789abcdef"
npx -y @maximhq/bifrost -app-dir ./.bifrost

# Official Bifrost local default is port 8080; HeavenBase can point at it explicitly.
hb cfg set heavenbase.llm.default_gateway bifrost
hb cfg set heavenbase.llm.gateways.bifrost.base_url "http://localhost:8080/v1"

# LiteLLM is a Python gateway path, not an exported OpenAI SDK client.
pip install litellm

export OPENROUTER_API_KEY="sk-or-v1-..."
hb cfg set heavenbase.llm.default_gateway litellm
hb cfg set heavenbase.llm.default_provider openrouter

# Deterministic local gateway for tests and demos. No network call or API key.
hb cfg set heavenbase.llm.default_gateway mock
hb cfg set heavenbase.llm.default_provider mock
hb cfg set heavenbase.llm.presets.chat.model mock

5.3. Exporting Clients

An OpenAI-compatible LLM instance can export a raw openai.OpenAI or openai.AsyncOpenAI client. The exported client carries the resolved client construction settings: API key, base URL, headers, timeout, and retry policy. It does not go through HeavenBase’s chat, stream, embedding, or image helpers. Use to_client() or to_aclient() for the SDK client. Use to_args() for the resolved request arguments, including the provider-facing model name and request defaults. This split is useful when an external SDK owns the call loop but you still want HeavenBase to own model, provider, gateway, and credential resolution.

import heavenbase as hb

llm = hb.LLM(preset="chat")
client = llm.to_client()            # openai.OpenAI, pre-configured

response = client.chat.completions.create(
    messages=[{"role": "user", "content": "Hello"}],
    **llm.to_args(),                # model plus request defaults
)
print(response.choices[0].message.content)

import asyncio
import heavenbase as hb

llm = hb.LLM(preset="chat")
aclient = llm.to_aclient()          # openai.AsyncOpenAI, pre-configured

async def main():
    response = await aclient.chat.completions.create(
        messages=[{"role": "user", "content": "Hello"}],
        **llm.to_args(),
    )
    print(response.choices[0].message.content)

asyncio.run(main())

import heavenbase as hb
from agents import Agent, OpenAIChatCompletionsModel, ModelSettings

llm = hb.LLM(preset="chat")
args = llm.to_args()                        # {"model": "...", "temperature": ..., ...}
model_name = args.pop("model")

agent = Agent(
    name="my-agent",
    model=OpenAIChatCompletionsModel(
        model=model_name,
        openai_client=llm.to_aclient(),     # AsyncOpenAI with HeavenBase config
    ),
    model_settings=ModelSettings(**args),
)

to_client(), to_aclient(), and to_args() are for OpenAI-compatible client exports: openai, portkey, or bifrost. They raise ValueError for litellm, anthropic, and mock gateways. The OpenAI Agents SDK tab is an integration pattern; install openai-agents before running it.

For the native Anthropic gateway, export an anthropic.Anthropic or anthropic.AsyncAnthropic client instead:

import heavenbase as hb

llm = hb.LLM(model="sonnet", provider="anthropic", gateway="anthropic")
client = llm.to_anthropic_client()      # anthropic.Anthropic, pre-configured
aclient = llm.to_anthropic_aclient()    # anthropic.AsyncAnthropic, pre-configured

OpenAI-compatible chat completions use POST /v1/chat/completions, request fields such as model and messages, and responses shaped around choices[0].message.content plus token usage fields. Anthropic-compatible chat uses POST /v1/messages, request fields such as model, max_tokens, messages, and optional top-level system, and responses shaped around typed content blocks, stop_reason, and usage.input_tokens / usage.output_tokens.

Further Exploration

Related resources:

LLM overview — full presets, model catalog, all built-in providers
CLI reference — complete command reference
First MCP — generic MCP math toolkit demo
HeavenBase MCP — expose a workspace over MCP, let agents handle the rest

​1. Set an API Key

​2. CLI Chat

​3. CLI Embeddings

​4. Interactive Session

​5. Python API

​5.1. The LLM Class

​5.2. LLM Gateways

​5.3. Exporting Clients

​Further Exploration

1. Set an API Key

2. CLI Chat

3. CLI Embeddings

4. Interactive Session

5. Python API

5.1. The `LLM` Class

5.2. LLM Gateways

5.3. Exporting Clients

Further Exploration