ecology trace library for the toroidal tesseract transformer
An interpretability trace library for T³, a Clifford-algebra-augmented transformer architecture in the geometric deep learning lineage. The atlas captures live per-head ecology state during inference across architectural lineages and foundation-model substrates. Traces are the citeable artifact; the viewer is one consumer of them.
Three foundation-model substrates — GPT-2, Gemma3, Qwen2.5 — each transferred through T³'s 6-primitive Cl(3,3) ecology and captured at multiple training checkpoints. Twelve architectural lineages span the project's history from v3.3 (single intra-pair coupling) through v3.7+ (full dynamic-Ω stack with σ-MLP width sweep).
Every trace records the live state the architecture computes during a forward pass: per-head primitive EMAs, σ envelope, Cl(3,3) Q invariant, Ω bivector and trivectors, blockade and cosurvival graphs, self-model surprise, ACT halt dynamics, per-stage logit top-K. The schema is stable and documented.
The Toroidal Tesseract Transformer is a transformer architecture that extends standard multi-head attention with a per-head cognitive ecology grounded in six conjugate primitives (entropy, intensity, friction; valence, coherence, chronos) coupled through Hamiltonian rotation in the Cl(3,3) Clifford algebra.
Heads live on a 3-torus, interact through Rydberg-style blockade and gravitational co-survival, and ponder adaptively per stage via output-entropy halt. The ecology is grounded, not auxiliary: it directly modulates attention temperature, key bias, and gradient flow during training.
This atlas is a record of what that ecology actually does during inference, made visible.
T³ sits in the geometric deep learning tradition (Bronstein et al., 2021; Brehmer et al., 2023, Geometric Algebra Transformer), extending standard transformer attention with Clifford-algebra-grounded primitive coupling rather than replacing attention as alternative-architecture work (Mamba, RWKV, xLSTM) does. The atlas itself is interpretability infrastructure in the SAEBench / Neuronpedia tradition: a versioned trace format, capability-flagged lineage records, multi-substrate coverage, designed as a public artifact for downstream analysis. The ecology dynamics — predictive-coding self-models, surprise-driven sigma targets, free-energy-style valence — derive from the active inference literature (Friston, 2010; Buckley et al., 2017).
| lineage | ckpts | substrate | capabilities |
|---|---|---|---|
| loading… | |||
The trace format is documented in SCHEMA.md. Loading is plain JSONL parsing, no T³-specific tooling required.
import json
def load_trace(path):
out = {"meta": None, "geoms": {}, "chain_states": [], "frames": []}
with open(path) as f:
for line in f:
r = json.loads(line)
t = r["type"]
if t == "meta": out["meta"] = r
elif t == "stage_geom": out["geoms"][r["stage_idx"]] = r
elif t == "chain_state": out["chain_states"].append(r)
elif t == "frame": out["frames"].append(r)
return out
Citation block forthcoming. For now, please cite as:
Sutherland, G. (2026). T³ Atlas: an ecology trace library for the Toroidal Tesseract Transformer. https://t3atlas.dev