> ## Documentation Index
> Fetch the complete documentation index at: https://ahvn.top/llms.txt
> Use this file to discover all available pages before exploring further.

# File System

> Portable paths with pj and config-driven serialization for local artifacts.

<Note>
  *Don’t trust your memory — write it down.*
</Note>

Use this page when a demo, test, script, or extension needs predictable local files without hand-rolling path handling and serialization details. The File System utilities center on two ideas: **`pj(...)`** for portable paths and a **serialization family** for consistent read/write across JSON, YAML, text, JSONL, jstr, binary, and related formats.

<br />

## 1. Why a Unified File System Utility

Without a unified file system utility, every script and demo re-implements path joining, alias resolution, encoding handling, and serialization. The results are inconsistent: one module uses `os.path.join(...)` with hardcoded relative paths, another uses `pathlib.Path` with `expanduser()`, and a third hardcodes `/` separators that break on Windows. Serialization diverges the same way — one caller writes JSON with `indent=2`, another uses `default=str`, and a third hard-codes `utf-8` encoding.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Anti-pattern: every caller handles paths differently
import json
import os

demo_dir = os.path.join(os.path.expanduser("~"), "heavenbase-demo", "data")
config_path = "../config.yaml"  # breaks depending on working directory
cache_dir = "C:/Users/me/.cache/heavenbase/" if os.name == "nt" else "/home/me/.cache/heavenbase"

with open(os.path.join(demo_dir, "meta.json"), "w", encoding="utf-8") as handle:
    json.dump({"rows": 3}, handle, indent=2, default=str)
```

HeavenBase treats paths and serialization as paired infrastructure. **`pj(...)`** is the single path vocabulary: it expands `~`, resolves environment variables, normalizes separators, optionally returns absolute paths, and resolves caller-provided aliases. The **serialization family** uses config-driven defaults (`CM_HVNB` at `heavenbase.serialize.encoding` and `heavenbase.serialize.indent`) so every caller gets consistent output without repeating those parameters.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import dump_json, pj

# Portable paths and consistent serialization in one flow
project_root = pj(".", abs=True)
cache_file = pj("~/heavenbase-cache", "index.json", abs=True)
run_path = pj("%/runs/latest.json", aliases={"%": "demos/.temp"}, abs=True)

dump_json({"workspace": "docs-demo", "rows": 3}, run_path, sort_keys=True)
```

Both halves matter equally: you build a path with `pj(...)`, then read or write through the matching serialization helper without re-specifying encoding, indentation, or platform details.

<br />

## 2. Two Core Ideas

File System utilities split cleanly into path vocabulary and serialization I/O. Everything else on this page — touching folders, listing files, copying artifacts, folder diagrams — supports those two flows.

### 2.1. `pj(...)` — Portable Paths

`pj(...)` is the canonical path join helper. It joins path parts, expands `~` and environment variables, normalizes separators, optionally returns an absolute path, and can resolve caller-provided aliases.

Think of it as a small path vocabulary. `.` means the current working location, `~` expands to the user home, environment variables expand through the operating system, and aliases such as `%` or `&` can point to a package data folder or resource folder when the caller or config manager provides that alias map.

<Info>
  Utility `pj(...)` handles normal paths and caller aliases. `CM_HVNB.pj(...)` handles HeavenBase package aliases: `%/` for package-local data and `&/` for package resources.
</Info>

<br />

### 2.2. Serialization Family — Config-Driven I/O

The serialization helpers wrap common read/write for the same reason configuration wraps hyperparameters: callers should not pass encoding, indentation, and platform path details through every function.

Each format exposes the same naming pattern:

| Prefix                | Scope                      | Example                     |
| --------------------- | -------------------------- | --------------------------- |
| `loads_*` / `dumps_*` | In-memory strings or bytes | `dumps_json({"rows": 3})`   |
| `load_*` / `dump_*`   | File read/write            | `dump_json(obj, path)`      |
| `save_*`              | Alias of `dump_*`          | `save_txt("# Notes", path)` |
| `append_*`            | Append one record or line  | `append_jsonl(event, path)` |

Text encodings default to `heavenbase.serialize.encoding`, which is `utf-8` in the default config. JSON and YAML indentation default to `heavenbase.serialize.indent`, which is `4`. Missing files return empty defaults (`""`, `{}`, `[]`, or `b""`) unless you pass `strict=True`. This shows the benefit of the config-driven approach: you can change the default encoding or indentation without changing every caller.

JSON helpers use `HbJsonEncoder` and `HbJsonDecoder`, which preserve common HeavenBase values such as datetimes, decimals, tuples, sets, and callable references — so artifacts round-trip without ad-hoc `default=str` handlers.

<br />

## 3. Build a Path

Use `pj(...)` for ordinary local paths:

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import pj

project_root = pj(".", abs=True)
cache_file = pj("~/heavenbase-cache", "index.json", abs=True)
```

Pass aliases when the first path segment is a project-specific shortcut:

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import pj

run_path = pj("%/runs/latest.json", aliases={"%": "demos/.temp"}, abs=True)
asset_path = pj("&/sample.txt", aliases={"&": "demos/assets"}, abs=True)
```

Use package aliases through the config manager:

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import CM_HVNB

config_db = CM_HVNB.pj("%/config.db", abs=True)
default_yaml = CM_HVNB.pj("&/configs/default.yaml", abs=True)
```

<br />

## 4. Prepare Files and Folders

Use `touch_dir(...)` and `touch_file(...)` before writing artifacts. Both create missing parents and return an absolute path.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import exists_file, touch_dir, touch_file

root = touch_dir("demos/.temp/report")
summary = touch_file(f"{root}/summary.md", content="# Summary")

assert exists_file(summary)
```

Use `clear=True` only when the folder or file should start empty:

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import touch_dir

run_dir = touch_dir("demos/.temp/run", clear=True)
```

<br />

## 5. Read and Write Data

Serialization is the second pillar of this page. Pick a format, call the matching helper, and let `CM_HVNB` supply encoding and indentation defaults. Combine with `pj(...)` and `touch_dir(...)` for a complete artifact workflow.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import append_jsonl, dump_json, load_json, pj, save_txt, touch_dir

root = touch_dir(pj("demos", ".temp", "docs-run"))
meta_path = pj(root, "meta.json")

dump_json({"workspace": "docs-demo", "rows": 3}, meta_path, sort_keys=True)
save_txt("# Run Notes", pj(root, "notes.md"))
append_jsonl({"event": "created", "path": meta_path}, pj(root, "events.jsonl"))

meta = load_json(meta_path)
```

### 5.1. JSON

Use JSON for structured machine-readable artifacts — configs, run metadata, workspace snapshots, and interchange with other tools.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import dump_json, dumps_json, load_json, loads_json

payload = {"workspace": "docs-demo", "items": (1, 2)}  # tuples round-trip
text = dumps_json(payload, sort_keys=True, compact=True)
restored = loads_json(text)

dump_json(payload, "demos/.temp/docs-run/meta.json", sort_keys=True)
assert load_json("demos/.temp/docs-run/meta.json")["items"] == (1, 2)
```

Pass `sort_keys=True` when deterministic key order matters (for example, before hashing). Pass `compact=True` or `indent=None` for one-line output.

<br />

### 5.2. YAML

Use YAML for human-editable structured files such as config templates and small manifests.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import dump_yaml, dumps_yaml, load_yaml

spec = {"model": "demo", "batch_size": 8}
dump_yaml(spec, "demos/.temp/docs-run/spec.yaml")
assert load_yaml("demos/.temp/docs-run/spec.yaml")["batch_size"] == 8
```

YAML shares the same indentation default as JSON through `heavenbase.serialize.indent`.

<br />

### 5.3. Text

Use text helpers for logs, Markdown notes, plain reports, and any human-readable line-oriented output.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import append_txt, load_txt, save_txt

save_txt("# Run Notes\n", "demos/.temp/docs-run/notes.md")
append_txt("event: created", "demos/.temp/docs-run/log.txt")
notes = load_txt("demos/.temp/docs-run/notes.md")
```

`load_txt` returns `""` for a missing file unless `strict=True`. `append_txt` adds a trailing newline after each write.

<br />

### 5.4. JSONL

Use JSONL when a workflow appends one event or record at a time — run logs, streaming exports, and incremental audit trails.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import append_jsonl, dump_jsonl, iter_jsonl, load_jsonl

append_jsonl({"event": "created", "idx": 1}, "demos/.temp/docs-run/events.jsonl")
append_jsonl({"event": "updated", "idx": 2}, "demos/.temp/docs-run/events.jsonl")

rows = load_jsonl("demos/.temp/docs-run/events.jsonl")
for row in iter_jsonl("demos/.temp/docs-run/events.jsonl"):
    print(row["event"])
```

Prefer `append_jsonl` for incremental writes and `dump_jsonl` when you have the full iterable in memory. Use `iter_jsonl` to stream large files without loading every line at once.

<br />

### 5.5. jstr — Python-Facing Text

**jstr** means "JSON when the top-level value is a container, otherwise string." Use `dumps_jstr`, `loads_jstr`, `dump_jstr`, `load_jstr`, and `save_jstr` at text boundaries such as Tool results, memstate rows, and MCP payloads where structured values should stay machine-readable while scalars and other values stay human-readable through `str(...)`.

Rules:

* Top-level `dict` or `list` values (including nested containers) serialize as compact JSON through the same encoder as `dumps_json`.
* All other values, including `int`, `float`, `bool`, `None`, tuples, sets, and custom objects, use `str(obj)`.
* `loads_jstr` parses text that looks like a JSON object or array; invalid JSON in that shape, and all other text, is returned unchanged as a string.

Toolkit and MCP boundaries use this format by default. Each Tool keeps `serializer=None` and resolves to the shared `jstr_serializer` Capsule at execution time, so `toolkit.run_to_str(...)` and FastMCP serving route successful tool outputs through `dumps_jstr`. Local Python callers can still use `toolkit.run(...)` for raw objects.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import dumps_jstr, loads_jstr, save_jstr

assert dumps_jstr(42) == "42"
assert dumps_jstr({"rows": 3, "items": [{"id": 1}]}) == '{"rows": 3, "items": [{"id": 1}]}'
assert loads_jstr('{"rows": 3}') == {"rows": 3}
assert loads_jstr("plain note") == "plain note"

save_jstr({"event": "created"}, "demos/.temp/docs-run/events.jstr")
```

<br />

### 5.6. Binary

Use binary helpers for raw byte payloads — model weights, compressed blobs, image bytes, and other non-text artifacts.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import dump_bin, load_bin, save_bin

payload = b"\x00\x01\x02"
save_bin(payload, "demos/.temp/docs-run/blob.bin")
assert load_bin("demos/.temp/docs-run/blob.bin") == payload
```

`load_bin` returns `b""` for a missing file unless `strict=True`.

<br />

### 5.7. Hex and Base64

Use hex and Base64 helpers when byte payloads need a text-safe representation in files or logs.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import dump_b64, dump_hex, load_b64, load_hex, dumps_b64, loads_b64

raw = b"hello"
assert loads_b64(dumps_b64(raw)) == raw

dump_hex("48656c6c6f", "demos/.temp/docs-run/hello.hex")
assert load_hex("demos/.temp/docs-run/hello.hex") == "48656c6c6f"

dump_b64(dumps_b64(raw), "demos/.temp/docs-run/hello.bin")
assert load_b64("demos/.temp/docs-run/hello.bin") == dumps_b64(raw)
```

`dump_hex` and `dump_b64` decode a hex or Base64 string and write the resulting bytes. `load_hex` and `load_b64` read a binary file and return a hex or Base64 text representation. Use `dumps_b64` / `loads_b64` for in-memory encoding without touching the filesystem.

<br />

### 5.8. Pickle

Use pickle only for trusted local artifacts produced by your own environment. Pickle can execute arbitrary code during load and must never cross a trust boundary.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import dump_pkl, load_pkl

state = {"cache": {"rows": 3}}
dump_pkl(state, "demos/.temp/docs-run/state.pkl")
assert load_pkl("demos/.temp/docs-run/state.pkl") == state
```

<Warning>
  `load_pkl(...)` reads Python pickle data. Use pickle only for trusted local artifacts produced by your own environment.
</Warning>

<br />

## 6. List, Copy, and Delete Artifacts

`list_files(...)` reads direct children. `enum_files(...)` walks recursively. Both return stable, case-insensitive order.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import copy_path, delete_path, enum_files, list_files

for name in list_files("demos/.temp/docs-run", ext="json"):
    print(name)

for path in enum_files("demos/.temp/docs-run", ext=["json", "jsonl"], abs=True):
    print(path)

copy_path("demos/.temp/docs-run", "demos/.temp/docs-run-copy", mode="merge")
delete_path("demos/.temp/docs-run-copy")
```

Copy helpers use explicit conflict modes. Files support `replace`, `skip`, and `strict`; directories also support `merge`.

<br />

## 7. Show a Folder Diagram

Use `folder_diagram(...)` for compact diagnostics in logs, reports, and docs snippets.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from heavenbase.utils import folder_diagram

print(folder_diagram("demos/.temp/docs-run", annotations={"meta.json": "run metadata"}, limit=12))
```

<Warning>
  `delete_dir(...)` and `delete_path(...)` remove local data. Keep demo cleanup targets under known temporary folders such as `demos/.temp/`.
</Warning>

<br />

## Further Exploration

<Tip>
  **Related resources:**

  * [Configuration](/features/configuration) - config-driven defaults used by serialization helpers.
  * [Hash](/features/utilities/hash) - deterministic file-safe identifiers and fingerprints.
  * [Miscs](/features/utilities/miscs) - command execution around local artifacts.
</Tip>

<br />