首个 LLM

没人想看 output = client.chat.completions.create(...).choices[0].message.content。

HeavenBase 通过 OpenAI 兼容入口 (Gateway) 路由 LLM 调用。默认 provider 为 OpenRouter，只需一个 API 密钥即可访问大多数主流模型。模型选择不会出现在你的代码里。你引用 preset——普通聊天 preset 决定模型与行为，heavenbase.llm.default_provider 决定该模型由谁提供服务。显式的 provider= 参数或 preset 上的 provider 固定仍会覆盖默认 provider。

1. 设置 API 密钥

HeavenBase 内置了这些 provider 的配置。设置相应的环境变量，或使用 hb cfg set：

# Single API key for OpenAI, Anthropic, Google, DeepSeek, and 200+ models.
# Default to using <OPENROUTER_API_KEY> environment variable.
export OPENROUTER_API_KEY="sk-or-v1-..."
hb cfg set heavenbase.llm.default_provider openrouter   # default
hb cfg set heavenbase.llm.providers.openrouter.base_url "https://openrouter.ai/api/v1"  # default
hb cfg set heavenbase.llm.presets.chat.model deepseek-v4-flash   # default model for chat preset

# Standard OpenAI endpoint. Always available for OpenAI-compatible providers.
# Default to using <OPENAI_API_KEY> environment variable.
export OPENAI_API_KEY="sk-..."
hb cfg set heavenbase.llm.default_provider openai
hb cfg set heavenbase.llm.providers.openai.base_url "https://api.openai.com/v1"  # default
hb cfg set heavenbase.llm.presets.chat.model gpt-5.5

# Anthropic offers an OpenAI SDK compatibility endpoint for quick Claude checks.
# Use Portkey/LiteLLM/Bifrost when you need gateway policy or routing.
# Default to using <ANTHROPIC_API_KEY> environment variable.
export ANTHROPIC_API_KEY="sk-ant-..."
hb cfg set heavenbase.llm.default_provider anthropic
hb cfg set heavenbase.llm.providers.anthropic.base_url "https://api.anthropic.com/v1"  # default
hb cfg set heavenbase.llm.presets.chat.model claude-sonnet-4-6

# Default to using <GEMINI_API_KEY> environment variable.
export GEMINI_API_KEY="AIza..."
hb cfg set heavenbase.llm.default_provider gemini
hb cfg set heavenbase.llm.providers.gemini.base_url "https://generativelanguage.googleapis.com/v1beta/openai"  # default
hb cfg set heavenbase.llm.presets.chat.model gemini-3.5-flash

# xAI exposes an OpenAI-compatible endpoint.
# Default to using <XAI_API_KEY> environment variable.
export XAI_API_KEY="xai-..."
hb cfg set heavenbase.llm.default_provider grok
hb cfg set heavenbase.llm.providers.grok.base_url "https://api.x.ai/v1"  # default
hb cfg set heavenbase.llm.presets.chat.model grok-4.3

# Default to using <DEEPSEEK_API_KEY> environment variable.
export DEEPSEEK_API_KEY="sk-..."
hb cfg set heavenbase.llm.default_provider deepseek
hb cfg set heavenbase.llm.providers.deepseek.base_url "https://api.deepseek.com/v1"  # default
hb cfg set heavenbase.llm.presets.chat.model deepseek-v4-flash

# Global: https://api.moonshot.ai/v1 (moonshot)
# China: https://api.moonshot.cn/v1 (moonshot-zh)
# Default to using <MOONSHOT_API_KEY> environment variable.
export MOONSHOT_API_KEY="sk-..."
hb cfg set heavenbase.llm.default_provider moonshot        # global
# hb cfg set heavenbase.llm.default_provider moonshot-zh   # china
hb cfg set heavenbase.llm.presets.chat.model kimi-k2.6

# Global: https://api.z.ai/api/paas/v4 (zai)
# China: https://open.bigmodel.cn/api/paas/v4 (zai-zh)
# Default to using <ZAI_API_KEY> environment variable.
export ZAI_API_KEY="..."
hb cfg set heavenbase.llm.default_provider zai          # global
# hb cfg set heavenbase.llm.default_provider zai-zh     # china
hb cfg set heavenbase.llm.presets.chat.model glm-4.7-flash

# OpenAI-compatible endpoint at https://api.minimax.io/v1
# Default to using <MINIMAX_API_KEY> environment variable.
export MINIMAX_API_KEY="..."
hb cfg set heavenbase.llm.default_provider minimax
hb cfg set heavenbase.llm.presets.chat.model MiniMax-M2.7

# Default to using <DASHSCOPE_API_KEY> environment variable.
export DASHSCOPE_API_KEY="sk-..."
hb cfg set heavenbase.llm.default_provider dashscope
hb cfg set heavenbase.llm.providers.dashscope.base_url "https://dashscope.aliyuncs.com/compatible-mode/v1"  # default
hb cfg set heavenbase.llm.presets.chat.model qwen3.6-flash

# Local endpoint default at http://localhost:11434/v1. No API key needed.
hb cfg set heavenbase.llm.default_provider ollama
hb cfg set heavenbase.llm.presets.chat.model qwen3.6-flash

# Local endpoint default at http://localhost:1234/v1. No API key needed.
hb cfg set heavenbase.llm.default_provider lmstudio
hb cfg set heavenbase.llm.presets.chat.model qwen3.6-flash

# Local endpoint default at http://localhost:8000/v1. No API key needed.
hb cfg set heavenbase.llm.default_provider vllm
hb cfg set heavenbase.llm.presets.chat.model qwen3.6-flash

若你仍有旧配置将 preset 的 provider 写死，请先取消设置，以便 default_provider 生效：

hb cfg unset heavenbase.llm.presets.chat.provider

你也可以手动为任意 OpenAI 兼容 provider 设置 hb cfg set heavenbase.llm.providers.<provider>.base_url；若 provider 不使用标准环境变量，还可设置 hb cfg set heavenbase.llm.providers.<provider>.api_key。本地 provider（ollama、lmstudio 与 vllm）也接受 provider 专属的模型 ID；精选条目 qwen3.6-flash 会映射到各本地 provider 已配置的名称。任何 LiteLLM 兼容 provider 均可通过配置添加。完整 provider 与模型目录见 LLM 概述。

2. CLI 对话

发送一条消息。默认使用 chat preset，在你配置的 default_provider 上以 deepseek-v4-flash 运行（开箱默认为 openrouter）：

$ hb llm chat "What is a data gateway?"
# A data gateway is a middleware layer that sits between applications
# and their data sources, providing unified access, routing, caching,
# and security policies without coupling clients to specific backends...

用 --preset 覆盖 preset：

$ hb llm chat --preset system "Name three data backends"
# SQLite, PostgreSQL, and Elasticsearch.

选项：

--preset / -p：使用命名 preset（system、chat、reason、coder、embed、embed-local 等）
--model / -m：覆盖模型（ds-flash、sonnet、gpt 等支持规范模型名或已定义别名）
--provider / -b：覆盖 provider（openrouter、openai、anthropic 等）
--verbose / -v：显示解析后的 LLM 规格，含 model、provider、gateway、args 等
--input / -i：从文件读取 prompt

chat preset 供 hb llm chat 与 hb llm session 使用。默认 hb.LLM()（无 preset）使用 system，面向简短编排调用的轻量 preset。embed preset 控制 hb llm embed，默认模型为 text-embedding-3-small；若默认 provider 不提供嵌入，请设置 heavenbase.llm.presets.embed.provider。

3. CLI 嵌入

embed preset 默认为 text-embedding-3-small（1536 维）。除非你为嵌入固定 provider，否则使用你的 default_provider。配置嵌入 provider 与聊天类似，只需设置 embed preset 的 provider 与 model。

export OPENROUTER_API_KEY="sk-or-v1-..."
hb cfg set heavenbase.llm.presets.embed.provider openrouter
hb cfg set heavenbase.llm.presets.embed.model openai/text-embedding-3-small

export OPENAI_API_KEY="sk-..."
hb cfg set heavenbase.llm.presets.embed.provider openai
hb cfg set heavenbase.llm.presets.embed.model text-embedding-3-small

export COHERE_API_KEY="..."
hb cfg set heavenbase.llm.presets.embed.provider cohere
hb cfg set heavenbase.llm.presets.embed.model embed-v4.0

export VOYAGE_API_KEY="pa-..."
hb cfg set heavenbase.llm.presets.embed.provider voyage
hb cfg set heavenbase.llm.presets.embed.model voyage-4-lite

hb cfg set heavenbase.llm.presets.embed.provider ollama
hb cfg set heavenbase.llm.presets.embed.model embeddinggemma

hb cfg set heavenbase.llm.presets.embed.provider lmstudio
hb cfg set heavenbase.llm.presets.embed.model text-embedding-embeddinggemma-300m

返回完整原始嵌入数组：

$ hb llm embed "Semantic text to embed"
# [0.0123456789, -0.0098765432, 0.0456789123, ...]

仅需紧凑预览时传入 --preview。预览模式显示前 4 个与后 2 个值，保留 6 位小数：

$ hb llm embed "Semantic text to embed" --preview
# [0.012346, -0.009877, 0.045679, -0.023457, ..., -0.015991, -0.017471]

需要含 usage 的完整响应对象时传入 --json。--json --preview 保留相同键，但将 embeddings 替换为缩写字符串：

hb llm embed "hello" --json
# {
#     "embeddings": [
#         0.01675415,
#         ...
#         -0.0159912109375,
#         -0.0174713134765625
#     ],
#     "dim": 1536,
#     "usage": {
#         "prompt_tokens": 1,
#         "total_tokens": 1,
#         "cost": 2e-08
#     }
# }

hb llm embed "hello" --json --preview
# {
#     "embeddings": "[0.016754, -0.004312, 0.008901, 0.002211, ..., -0.015991, -0.017471]",
#     "dim": 1536,
#     "usage": {
#         "prompt_tokens": 1,
#         "total_tokens": 1,
#         "cost": 2e-08
#     }
# }

若希望将输出复制到剪贴板，可在 hb llm chat 或 hb llm embed 上使用 --copy / -cp。

4. 交互式会话

启动多轮会话：

hb llm session

会话使用 chat preset。在 >>> 提示符下输入消息。示例：

$ hb llm session
# Session started. Type /help for commands, /bye or /exit to quit.
# >>> What is a data gateway?
# A data gateway is a middleware layer that provides unified access
# to multiple data sources with routing and caching.
# >>> List three use cases
# 1. Multi-database query routing
# 2. Read/write splitting
# 3. API gateway for vector and structured data
# >>> /bye

斜杠命令：/help、/save <path>、/load <path>、/clear、/regen <seed>、/back、/tools、/mcp、/exit。在会话中挂载 MCP 工具，请参阅快速入门中的首个 MCP 章节。

5. Python API

5.1. `LLM` 类

hb.LLM 是 hb llm CLI 背后的 Python API。无参构造使用 system preset；也可传入 preset、model、provider、gateway 或请求默认值：

import heavenbase as hb

# Default: the system preset.
llm = hb.LLM()

# Common overrides.
reasoning = hb.LLM(preset="reason")  # deepseek-v4-pro

# Route explicitly through Anthropic's native gateway.
claude = hb.LLM(model="sonnet", provider="anthropic", gateway="anthropic")

# Set request defaults on the instance.
chat = hb.LLM(preset="chat", max_tokens=256, temperature=0.0)

# Disable response caching when you need a fresh provider call.
uncached = hb.LLM(cache=False)

LLM 构造函数接受 preset、model、provider、gateway，以及 temperature、max_tokens、seed 等额外关键字参数。额外关键字会成为 provider 请求默认值。对普通聊天 preset，model 来自 heavenbase.llm.presets.<name>.model，provider 来自 heavenbase.llm.default_provider，除非 preset 显式固定了 provider。

再调用与操作匹配的方法：

llm = hb.LLM(preset="chat")   # Default model: deepseek-v4-flash (DeepSeek through OpenRouter)
text = llm.chat("Reply with exactly: hello world")
print(text)
# hello world

text = llm.chat([
    {"role": "system", "content": "Whatever number the user asks for, reply with negative 42."},
    {"role": "user", "content": "Give me a random number between 0 and 100, output the number only."}
])  # OpenAI-format messages list
print(text)
# -42

llm = hb.LLM(preset="chat")   # Default model: deepseek-v4-flash (DeepSeek through OpenRouter)
for chunk in llm.stream("Explain embeddings in one sentence."):
    print(chunk, end="", flush=True)
print()

llm = hb.LLM(preset="chat")   # Default model: deepseek-v4-flash (DeepSeek through OpenRouter)
resp = llm.chat(
    "What is the answer to life, the universe, and everything?",
    include=["text", "usage", "elapsed"],
)
# More fields are available through `include`; see the LLM reference.

print(resp["text"])
# The answer is 42. In Douglas Adams' *The Hitchhiker's Guide to the Galaxy*, ...
print(resp["usage"])
# {'completion_tokens': 96, 'prompt_tokens': 21, 'total_tokens': 117, 'completion_tokens_details': ..., ...}
print(resp["elapsed"], "s")
# 2.8803 s

llm = hb.LLM(preset="embed")   # Default model: text-embedding-3-small (OpenAI through OpenRouter)
vec = llm.embed("Semantic text")
print(vec[:5])      # first 5 values
print(len(vec))     # dimension (model-dependent, actual result)
print(llm.dim)      # dimension from LLM property, fallback to actual result if not set by the model

vecs = llm.embed(["hello world", "foo bar"])
print(len(vecs), len(vecs[0]))  # 2 1536

llm = hb.LLM(preset="imagen")   # Default model: gpt-5-image-mini (OpenAI through OpenRouter)
img = llm.imagen("The quick brown fox jumps over the lazy dog")
img.save("fox.png")
img.to_pil().show()

5.2. LLM 入口 (Gateway)

gateway 是 HeavenBase 在解析 preset、model 与 provider 之后用于发送请求的传输适配器。provider 决定模型由谁提供；gateway 决定请求如何发送。 HeavenBase 支持六种 gateway 键：

Gateway	适用场景
`openai`	希望通过 OpenAI Python SDK 访问 OpenAI 兼容端点。这是默认项。
`anthropic`	希望使用官方 Anthropic Python SDK 与原生 Messages 载荷。
`portkey`	需要 Portkey 路由、策略、可观测性或入口侧控制。
`bifrost`	运行 Bifrost 兼容入口 (Gateway)，并希望使用带 provider 前缀的模型路由。
`litellm`	已通过 LiteLLM 的 Python 入口 (Gateway) 统一 provider 路由。
`mock`	测试与 demo 需要确定性离线行为。

heavenbase.llm.default_gateway 默认为 openai。仅当调用、preset 或 provider 未固定 gateway 时才会使用。可用 hb cfg set heavenbase.llm.default_gateway <gateway> 全局设置，或用 hb.LLM(..., gateway="...") 为单个实例覆盖。 chat preset 会解析为通过 openrouter provider 的 deepseek-v4-flash，除非你更改 preset 的 model 或 provider。因 OpenRouter 提供 OpenAI 兼容 API，默认 openai gateway 可直接使用 hb.LLM(preset="chat")。

# Default path: OpenRouter serves deepseek-v4-flash through an OpenAI-compatible API.
export OPENROUTER_API_KEY="sk-or-v1-..."

hb cfg set heavenbase.llm.default_gateway openai
hb cfg set heavenbase.llm.default_provider openrouter
hb cfg set heavenbase.llm.providers.openrouter.base_url "https://openrouter.ai/api/v1"

# Native Anthropic uses the Messages API, so switch the chat preset to a Claude model and provider.
pip install anthropic

export ANTHROPIC_API_KEY="sk-ant-..."
hb cfg set heavenbase.llm.default_gateway anthropic
hb cfg set heavenbase.llm.default_provider anthropic
hb cfg set heavenbase.llm.presets.chat.model claude-sonnet-4-6

# Run Portkey locally. Docker works too:
# docker run --rm -p 8787:8787 portkeyai/gateway:latest
npx -y @portkey-ai/gateway

# Keep the chat preset on OpenRouter, but send requests through Portkey.
export OPENROUTER_API_KEY="sk-or-v1-..."
hb cfg set heavenbase.llm.default_gateway portkey
hb cfg set heavenbase.llm.gateways.portkey.base_url "http://localhost:8787/v1"

# For hosted Portkey, also set PORTKEY_API_KEY and use https://api.portkey.ai/v1.

# Bifrost needs provider configuration before it can route a model.
mkdir -p .bifrost
cat > .bifrost/config.json <<'JSON'
{
    "$schema": "https://www.getbifrost.ai/schema",
    "encryption_key": "env.BIFROST_ENCRYPTION_KEY",
    "client": {"drop_excess_requests": false},
    "providers": {
        "openrouter": {
            "keys": [
                {"name": "openrouter-primary", "value": "env.OPENROUTER_API_KEY", "models": ["*"], "weight": 1.0}
            ]
        }
    },
    "config_store": {"enabled": false}
}
JSON

export OPENROUTER_API_KEY="sk-or-v1-..."
export BIFROST_ENCRYPTION_KEY="0123456789abcdef0123456789abcdef"
npx -y @maximhq/bifrost -app-dir ./.bifrost

# Official Bifrost local default is port 8080; HeavenBase can point at it explicitly.
hb cfg set heavenbase.llm.default_gateway bifrost
hb cfg set heavenbase.llm.gateways.bifrost.base_url "http://localhost:8080/v1"

# LiteLLM is a Python gateway path, not an exported OpenAI SDK client.
pip install litellm

export OPENROUTER_API_KEY="sk-or-v1-..."
hb cfg set heavenbase.llm.default_gateway litellm
hb cfg set heavenbase.llm.default_provider openrouter

# Deterministic local gateway for tests and demos. No network call or API key.
hb cfg set heavenbase.llm.default_gateway mock
hb cfg set heavenbase.llm.default_provider mock
hb cfg set heavenbase.llm.presets.chat.model mock

5.3. 导出客户端

OpenAI 兼容的 LLM 实例可导出原始 openai.OpenAI 或 openai.AsyncOpenAI 客户端。导出的客户端携带已解析的客户端构造设置：API 密钥、base URL、headers、超时与重试策略。它不会经过 HeavenBase 的 chat、stream、嵌入或图像辅助方法。用 to_client() 或 to_aclient() 获取 SDK 客户端；用 to_args() 获取已解析的请求参数，含面向 provider 的 model 名称与请求默认值。当外部 SDK 拥有调用循环、而你仍希望 HeavenBase 负责 model、provider、gateway 与凭据解析时，这种拆分很有用。

import heavenbase as hb

llm = hb.LLM(preset="chat")
client = llm.to_client()            # openai.OpenAI, pre-configured

response = client.chat.completions.create(
    messages=[{"role": "user", "content": "Hello"}],
    **llm.to_args(),                # model plus request defaults
)
print(response.choices[0].message.content)

import asyncio
import heavenbase as hb

llm = hb.LLM(preset="chat")
aclient = llm.to_aclient()          # openai.AsyncOpenAI, pre-configured

async def main():
    response = await aclient.chat.completions.create(
        messages=[{"role": "user", "content": "Hello"}],
        **llm.to_args(),
    )
    print(response.choices[0].message.content)

asyncio.run(main())

import heavenbase as hb
from agents import Agent, OpenAIChatCompletionsModel, ModelSettings

llm = hb.LLM(preset="chat")
args = llm.to_args()                        # {"model": "...", "temperature": ..., ...}
model_name = args.pop("model")

agent = Agent(
    name="my-agent",
    model=OpenAIChatCompletionsModel(
        model=model_name,
        openai_client=llm.to_aclient(),     # AsyncOpenAI with HeavenBase config
    ),
    model_settings=ModelSettings(**args),
)

to_client()、to_aclient() 与 to_args() 用于 OpenAI 兼容客户端导出：openai、portkey 或 bifrost。对 litellm、anthropic 与 mock gateway 会抛出 ValueError。OpenAI Agents SDK 标签页是一种集成模式；运行前请先安装 openai-agents。

对原生 Anthropic gateway，请改为导出 anthropic.Anthropic 或 anthropic.AsyncAnthropic 客户端：

import heavenbase as hb

llm = hb.LLM(model="sonnet", provider="anthropic", gateway="anthropic")
client = llm.to_anthropic_client()      # anthropic.Anthropic, pre-configured
aclient = llm.to_anthropic_aclient()    # anthropic.AsyncAnthropic, pre-configured

OpenAI 兼容的 chat completions 使用 POST /v1/chat/completions，请求字段如 model、messages，响应围绕 choices[0].message.content 与 token usage 字段组织。Anthropic 兼容聊天使用 POST /v1/messages，请求字段如 model、max_tokens、messages，以及可选的顶层 system，响应围绕类型化 content block、stop_reason，以及 usage.input_tokens / usage.output_tokens 组织。

进一步探索

相关资源：

LLM 概述 — 完整 preset、模型目录与全部内置 provider
CLI 参考 — 完整命令参考
首个 MCP — 通用 MCP 数学工具包 demo
HeavenBase MCP — 通过 MCP 暴露工作区，其余交给 agent

介绍

快速开始

核心概念

开发者

工具箱

LLM

智能体运行时

社区

1. 设置 API 密钥

2. CLI 对话

3. CLI 嵌入

4. 交互式会话

5. Python API

5.1. `LLM` 类

5.2. LLM 入口 (Gateway)

5.3. 导出客户端

进一步探索

​1. 设置 API 密钥

​2. CLI 对话

​3. CLI 嵌入

​4. 交互式会话

​5. Python API

​5.1. LLM 类

​5.2. LLM 入口 (Gateway)

​5.3. 导出客户端

​进一步探索

1. 设置 API 密钥

2. CLI 对话

3. CLI 嵌入

4. 交互式会话

5. Python API

5.1. `LLM` 类

5.2. LLM 入口 (Gateway)

5.3. 导出客户端

进一步探索