One Object: Memory, Navigation, and Reportability in an Operator-Only Language Model

Preprint v1.3 — 2026-09-10. DOI: 10.5281/zenodo.22684392. Versioned; dated revisions supersede (the DOI resolves to the latest version). §12 reports an ongoing experimental program and carries its own dates.

Cite as: Sutherland, G. (2026). One Object: Memory, Navigation, and Reportability in an Operator-Only Language Model. Zenodo. doi:10.5281/zenodo.22684392

Garret Sutherland — MirrorEthic LLC

Abstract

We study cl33-opLM, a language model whose only path from computation to output is a stream of explicitly emitted Cl(3,3) bivector operators: a transformer emitter produces per-block operators, a reversible SO(3,3) rotor scan transports state, attention scores are η-metric products of rotors, and the readout sees exclusively operator-derived features. Because generation, memory, steering, and inspection all act on the same algebraic object, questions that are normally interpretive become measurable. We report four results. (1) Two memories: associative recall (MQAR) and relational navigation are mechanistically distinct in this architecture — recall rides a token-similarity induction shortcut that bypasses the operators entirely (operator-dependence ≈ 0), while navigation is operator-native (operator-dependence ≈ 0.85, scale-invariant); we show the recall shortcut and the bypass are the same operation at matched training budget. (2) Parity where it counts: on permutation-composition navigation — a known transformer weak spot — cl33 matches a learning-rate-tuned transformer baseline exactly (0.433 ± 0.009 vs 0.432 ± 0.030) with 3× lower seed variance and a fully inspectable mechanism. (3) The operator stream is a transcript: a probe trained only on emitted operators decodes the current token at 0.86 top-1 over the 50k vocabulary — ~9.6 of ~11.2 bits of token identity per emission, with 80.7% of running text reconstructed verbatim from the operator record alone, and identity confirmed under substitution. The model's only channel to output doubles as a readable record of what passed through it. (4) An architectural workspace: applying the Jacobian-lens verbalizability estimator across the model's depth shows the emitted operators are the most output-transparent representations in a three-model comparison (final-agreement 0.613, above both GPT-2 controls' best layers; replicated on a second model), while the transported state is completely non-verbalizable (0.000) — a split that holds across two held-out domains. The workspace/ocean dichotomy that emerges statistically in standard transformers is, here, a line in the wiring diagram. We frame what the architecture buys (transparency, reversibility, exact memory, causal steering surface), what it costs (a real, measured tax: +29% bits-per-byte and ~5 points average MCQ against a param-and-context-matched from-scratch control at 5B tokens — though continued prose training moves the same model to exact bits-per-byte parity with token-matched Pythia-160m (1.155 at 6.2B tokens; caveats on ruler alignment in §10), showing the tax is data-bound rather than an architecture floor; we claim no perplexity or reasoning superiority), and what it does not claim.

On the axis the architecture is built for, we report capability results with matched baselines that collapse or cannot produce the artifact: exact retrieval at load (1.000 vs a matched transformer's ≈chance at T=1024/KV=64), editable memory with transaction semantics and bit-exact undo, and causally-faithful provenance receipts (attribution@1 1.000 vs a transformer best-head 0.017). These were measured on a reversible-tape intermediate, and its honest failure mode — attention re-carries recall unless the memory is the sole cross-segment channel (§5.4) — is what taught the sole-channel-by-construction design of the current memory organ (§12). That organ's first belief-install battery (install 0.750, algebraic delete 0.000, receipts 24/24) is reported as a pilot. We close with the governance reading: because the operator record is the mandatory carrier of the computation (ablating it multiplies perplexity by orders of magnitude) and the reversible scan makes that record losslessly replayable (exact to 2e-14), the perplexity tax inverts into the price of an audit guarantee that post-hoc interpretability cannot provide at any cost — with the guarantee scoped to the record's mandatoriness, replayability, editability, and decodability, not correctness, and with the scan's own causal share at LM inference measured (independently) as decorative (§3.2).

1. Introduction

Interpretability of standard transformers is archaeology: the artifact is finished, and we dig. Probes, sparse autoencoders, and lens methods recover correlates of computation, with faithfulness always in question because the object recovered is not the object the model computes with.

cl33-opLM inverts the arrangement. The architecture is chosen so that the model's only route to the output runs through a typed, invertible, algebraically-structured object — a per-token, per-block set of Cl(3,3) bivector operators. There is nothing behind the operators to be unfaithful to: the readout is constructed to see operator-derived features only, and we verify the construction empirically (ablating operators multiplies perplexity 8–42× at 32M, 206× at 163M — a ratio that climbs with capability). Memory is an operation on operators (§5). Navigation is composition of operators (§4). Reportability is a property of operators (§9). Steering is addition of operators (App. C). One object.

The purpose of this paper is not to claim superiority over transformers at language modeling. The transparent model pays a real, measured premium against a param+context- matched from-scratch control at 5B tokens (+29% bits-per-byte, §10 Table 1) — and the same machinery-bearing model, continued on prose to 6.2B tokens, reaches exact bits-per-byte parity with token-matched Pythia-160m (§10 Table 1b). Both numbers are reported: the matched cost of the machinery at a fixed budget is real; the ceiling it was once mistaken for is not. The claim is narrower and, we think, more useful: when the generative object is algebraic and exclusive, capabilities and their mechanisms become the same thing, and questions like "does the model use its history as a trajectory or a bag?" or "which internal content is reportable?" stop being interpretive and acquire experiments with pass/fail readouts. Every positive claim in this paper carries an operator-ablation control (the bypass blade): if a behavior survives with the algebra removed, it is a shortcut and it does not count — including our own memory mechanism, which we first built wrong and report as such (§5.1).

The intuitive form of our results: a transformer does not carry a transcript of words — it carries a trajectory of prediction-relevant events from which the words are mostly recoverable. This is true of standard transformers too (our emitter's hidden state decodes tokens at 0.91), but there the events and their accumulation are entangled in one vector. cl33 factors the process into inspectable parts: the operators are the event record (token identity decodes at 0.86, degrades to near-synonyms because the record stores what a token does to prediction, not its orthography — §7); the transported state is the accumulated path (literally s_t = R_t···R_1·s_0: zero token content, non-verbalizable — §3, §9; its causal share is smaller than this factoring once suggested — the §3.1 audit finds flow rides the operator features, with the state a passenger on the probes run so far); and generation reads both through a mandatory interface (bypass 8–42× at 32M, 206× at scale; emission loss 0.48 bits — §2, §7). Every major result in this paper — the two-memory dichotomy, the tape's reader taxonomy, the order-readout null, the workspace/ocean wall — is a facet of this factorization.

A living artifact. The claims in this paper are demonstrated interactively at cl33.t3atlas.dev: a transparent-chat interface over the trained model in which every response is accompanied by its operator transcript, the reverse readout (the §7 inversion run live — the conversation decoded back out of the operators), rewind, and steering. The demo is not an illustration of the paper; it is the same model, same algebra, same receipts, publicly poke-able — and the bottleneck claim is measured on the serving checkpoint itself: ablating the operators multiplies the demo model's perplexity 314× (the 8–206× ratios elsewhere in this paper are the training-lineage measurements at their own scales). Where a reviewer would ask "does this survive contact with arbitrary input," the demo is our answer.

History, reported as science. This architecture's development includes a falsification worth stating plainly rather than politely omitting, because it explains the design and produced one of its cleanest results. The original cl33-opLM carried its recurrent state as a grade-1 6-vector under the grade-preserving rotor action — and its associative memory failed structurally (MQAR ≈ 1/K at every load, §3): a key→value binding is a grade-2 bivector, and a grade-1 carrier has nowhere to hold one. A four-arm causal ladder built in the successor Genesis program isolated the error exactly — the grade-1 arm reproduces the failure, the grade-2 wedge store binds exactly, a param-matched generic control shows the fix is structure rather than dimension (mqar_left_regular.py, reproducible). For a period the language-model line was formally closed as superseded on the strength of that diagnosis. It reopened when two things became clear: continued training closed the model-class fluency gap (Table 1b), and the representation lesson could be incorporated without abandoning the architecture — superposition belongs in an explicit grade-2 store, not the grade-1 carrier, which is exactly how the memory results in this paper are constructed (§5, App. F) and how the ongoing organ program is built (§12). The vector representation failed for one memory formulation; the failure was localized by experiment; the design absorbed the lesson. We consider this sequence — claim, falsification, diagnosis, reincorporation — the paper's strongest argument that the methodology works.

How to read this paper. The systems description that organizes everything that follows: a neural engine (the transformer emitter — opaque, nonlinear, where the learned computation lives), a geometric bus (the operator stream — mandatory, typed, inspectable, the only path to output), and causal peripherals (memory, sensors, steering — machinery attached to the bus rather than baked into the weights, each attaching under a zero-init discipline that makes it bit-exactly removable). The paper is arranged in three parts. Part I — The Object (§2, §7, §8, §9, §10): the bus exists, transcribes both content and state, is the verbalizability peak while the transported state is the floor, and costs a measured, data-bound tax. Part II — Memory on the object (§3, §4, §5, §6, §12): what memory the bus does not give for free, the falsifications that located the sole-channel requirement, and the register organ that now installs, adopts, and deletes beliefs with receipts. Part III — Governance (§13): what the receipts buy, precisely which reversion claim is being made, and the three doors through which weaker guarantees re-enter. Section numbers are stable identifiers carried from earlier drafts; reading order is the Part structure.

Contributions

1. The two-memory dichotomy with a mechanistic identity proof: recall-as-induction and routing-around-the-operators are the same operation (§3).
2. Pre-registered navigation parity with a tuned transformer on permutation composition, with tighter variance and an inspectable mechanism (§4).
3. The reversible tape as the load-bearing intermediate: exact forward-readable memory native to the rotor scan, with the addressing/content separation principle ("token content may SELECT, only algebra may FLOW"). Its capability results (MQAR 1.000, zero bypass, an attention-free challenger at matched probe scale) hold in sole-channel configurations; its measured failure mode — attention re-carries recall whenever it coexists (§5.4) — is what taught the sole-channel-by-construction design of the memory organ (§5, §12).
4. The inversion battery: the input words decode back out of the operator stream — 0.86 top-1 over 50k vocab from one emission, +4.7 bits beyond prefix predictability, identity confirmed under fixed-context token substitution, 80.7% verbatim document reconstruction with semantic (near-synonym) degradation (§7).
5. A claims ladder for self-structure with adjudicated results: transparent recall (closed), structural path-sensitivity (passed), semantic order-readout (null, twice, with a mechanistic explanation), self-modeling (live, carrier localized) (§8).
6. Jacobian-lens evidence that the workspace/ocean split is architectural in cl33: operators are the verbalizability peak of a three-model study (replicated on two cl33 runs, domain-invariant across two held-out domains); transported state is invisible to the lens (§9) and output-causally inert under direct steering — a residual bias out-steers an operator-field bias ~25× with no positional compounding at either site (§3.1, closed 2026-09-08).
7. A capability battery on the axis the architecture is for, each against a matched baseline that collapses or cannot produce the artifact: exact retrieval at long sequence length (1.000 vs a matched transformer's ≈chance at T=1024/KV=64 — with the scoping that addressing is short-range: distance-resolved recall collapses with query–pair gap and is a measured limitation, §5.4 — across a pre-registered fairness sweep), editable memory with transaction semantics and bit-exact undo (provable unlearning-with-receipt), and causally-faithful provenance (attribution@1 1.000 vs a transformer best-head 0.017) (§6).
8. The measured transparency tax and its governance inversion: +29% bits-per-byte and ~5 pp average MCQ against a param-and-context-matched from-scratch control (honestly reported; no superiority claimed, and the tax itself is now shown to be data-bound — continued training reaches class-ruler bpb parity at matched tokens, §10 Table 1b), and the argument that because the reverse trace is the computation, that tax buys an audit guarantee post-hoc interpretability cannot provide at any cost — scoped to invertibility/editability/decodability, not correctness (§10).

Part I — The Object

The bus exists, transcribes content and state, is the verbalizability peak, and costs a measured tax.

2. Architecture (summary)

(Full spec: CL33_OPLM_ARCHITECTURE.md v2.2; tape extension: REVERSIBLE_TAPE_DESIGN.md, CL33_OPLM_V24_TAPE_LM.md.)

• Geometry. Cl(3,3) with η = diag(−1,−1,−1,+1,+1,+1). The 15-dimensional bivector space spans 6 rotations and 9 boosts; rotors R = exp(B·G) form SO(3,3).
• Emitter. A standard transformer (6 layers at probe scale; 8–12 at 5B scale) reads tokens and emits, per block and per token, three bivectors: B_state (the operator), B_q, B_k (query/key rotors). Coefficients are norm-clipped.
• Reversible scan. s_t = R_t · s_{t−1}, fp32 with matrix_exp (bf16 diverges). Inverses are free: R⁻¹ = ηRᵀη. The reverse trace is the explanation — replaying the scan backwards recovers every intermediate state exactly — 2e-14 in fp32 (fp64 reaches machine ε); anchored long-horizon reads reach 6.1e-5 at T = 1024.
• Operator-native attention. Q and K are rotors; the score is their η-metric product. No dot-product attention over hidden states exists anywhere.
• Operator-only readout. Final features are a grade tower over (attention output, state, operators, wedge grades); the readout is linear on these. The emitter's hidden states never reach the output. This is the transparency bottleneck, and it is the load-bearing design decision.
• Bypass blade (used throughout). For any capability, re-evaluate with operators ablated (identity rotors / zeroed operator features). Report the ratio of ablated loss to the unigram/chance floor. Ratio ≈ 1 ⇒ the capability lived in a bypass. On the LM task the base model's ablated-PPL ratio climbs from 8.3× to 42× over training at 32M, and reaches 206× at 163M/5B — transparency strengthens with capability rather than eroding.

Precision discipline: split precision (amp_emit) — fp16 emitter, fp32 algebra. Full fp16 NaNs from η-score/wedge overflow.

7. Reading the words back: the operator stream as transcript

(Full protocol, pre-registration, and adjudication: INVERSION_BATTERY.md. Design: tiered decoders + prefix-only control + fixed-context substitution, document-disjoint splits.)

Sections 3–5 established the forward direction: the operators are causally necessary for the output (ablation multiplies loss 8–206× depending on scale), and by construction nothing else reaches the readout. This section measures the inverse: given only the emitted tuple O_t = concat(B_state, B_q, B_k) (1440-d per position), can the input words be decoded back out? If yes, the operator record is a bidirectional transcript — input decodable backward, next-token decodable forward (§9), trajectory invertible in the middle — and the residual skeptic's position ("the transformer emitter does the real work behind the bottleneck") has nowhere to live: whatever the emitter computes either passes through the operators (measured here) or does not affect output (ablation, §2).

Setup. Frozen scale-C (163M, ~449k steps), 1.23M positions of held-out mix text across 5 domains, GPT-2 BPE (50,257 vocab, unigram floor 0.082 / 11.16 bits), document-disjoint train/eval splits, shuffled-pairing and Gaussian floors.

Results.

• Bits beyond prefix predictability: 4.7 (3.4 against the LM's own head) — the decoding is not a language-redundancy artifact.
• Emission loss ≈ 0.5 bits (operators 1.58 vs emitter hidden 1.10): the bottleneck transmits nearly everything the emitter knows about the current token — and this loss shrank from ~1 bit at 32M. Like the bypass ratio, the bottleneck's fidelity improves with capability.
• The code is distributed and role-structured. No single block exceeds 0.08 alone (32 blocks ≈ 0.01–0.08 each, from the pilot's B_state-only random-split battery; full stream 0.725 linear); families order B_state (0.605) > B_k (0.523) > B_q (0.460) — the state operator carries the most identity, the key rotor advertises what is here, the query rotor encodes what to seek. (The pre-registration predicted all three families ≈ equal; the measured ordering is a pre-reg deviation, reported as such.)
• One object, three tenses. The same emission also carries the previous token (B_state decodes tok_{t−1} at 0.29 — pilot battery) and the forthcoming prediction (O at t−1 decodes tok_t at 0.377 — the forecast channel §9 sees from the output side), while the transported state S decodes ≈ nothing (0.02, pilot): content rides on the operators from the decode direction too. (An earlier draft added "path rides on the state"; that is now closed rather than hedged — S is the accumulated path by construction, but its causal share at the output is measured ≈0: near-zero on flow probes and 25×-dominated by the residual under direct steering, with no positional compounding — §3.1.)

Interventions (identity, not expectation). Holding the prefix fixed and substituting the token at position t, the frozen decoder identifies injected mid-frequency tokens — including semantically strange ones in alien contexts — at 0.82 (plausible 0.94, random-vocab 0.32, rare>20k 0.12, MLP probe). Errors land on the contextually-expected token only 2.9% of the time (linear probe). The channel reads actual token identity, not contextual expectation; identification is frequency-graded at the rare tail — a reading we note is the post-hoc adjudication: the pre-registered prediction that random and rare tokens would identify within 15pp of mid-frequency ones failed (the battery's falsification line — errors collapsing onto expected tokens — did not trigger, which is the part that carries the claim). The same-token-across-contexts test quantifies the blend: one token's operator representation moves substantially with context (within-type variance 65% of global) while remaining decodable (83% mean recovery).

Reconstruction. Decoding entire held-out documents from the operator log alone (±2 window) recovers 80.7% of tokens exactly (per-document 0.52–0.98 — narrowly missing the pre-registered ≥0.85 gate, which we report rather than round), and the misses are near-synonyms — "growth"→"innovation", "petroleum products"→"oil products", "says"→"said". The log reads back as a meaning-preserving paraphrase where it is not verbatim: the identity code degrades semantically, not randomly, consistent with a frequency-graded code on a semantic operator geometry (App. C).

What we claim: the operator stream is a bidirectional, frequency-graded transcript — ~9.6 of ~11.2 bits of current-token identity per emission, 4.7 bits beyond predictability, with identity confirmed under substitution. What we do not claim: verbatim recovery of the deep-rare vocabulary; the code is distributed, context-modulated, and frequency-weighted.

8. The claims ladder: what the model can read of itself

We evaluate self-structure with an explicit ladder, most-skeptical-first framing, and the bypass blade at every rung. (Full protocol and adjudications: SELF_STRUCTURE_CLAIMS.md.)

Claim 1 — transparent recall. The model retrieves via a mechanism we can read. CLOSED by §5: recall is a named read (increment mode) of a lossless record, with operator-ablation destroying it.

Claim 2a — structural path-sensitivity. The model's behavior depends on its trajectory as a path, not as a bag of visited states. Test: counterfactual history — block-swap surgery on the tape record preserving the operator-entry multiset while recomputing the displacements it implies (the P1 intervention; the distinct P2 same-visited-state variant probes the state-mode reader and is not conflated here), injected via tape-override hooks. PASSED for displacement mode: outputs shift when path structure shifts, and the shift vanishes under operator ablation (so it is not a token-order artifact). This is exactly what the §5.3 taxonomy predicts: displacement is the global-path reader; only the path reader should be path-sensitive — and increment/state modes serve as built-in negative controls.

Claim 2b — semantic order-readout. The model can report trajectory order in language (narrative-order probes). NULL, twice (full model, then scan-only retest — which came back null/inverted: records get discounted). Mechanistic explanation: order-content is attention-dominated at readout; the trajectory channel influences prediction (2a) without being linearly decodable into tokens. We initially treated this as a failed experiment; §9 reframes it — the trajectory lives in the non-verbalizable part of the model, and the null is the ocean behaving like the ocean. A CE-pure rule (no auxiliary order-pressure losses just to pass our own probe) is in force; the aux variant is deferred as a side experiment, not a rescue.

Claim 3 — self-modeling (the prize; held skeptically). The model distinguishes its own generation history from a compatible foreign one. LIVE, not passed. Current evidence: a generator-identity signal 4× the content signal, carried on the operator record (blade decomposition: operators-only 0.092 vs tokens-only 0.0012 — the carrier is blade-2, the algebra, not the tokens). The phase-0 probe design confounded live-vs-dead records with own-vs-foreign and was redesigned; the honest current statement is that we can name the boundary — ownership vs familiarity — and have localized the carrier, but the discriminating probe (twin-with-same-seed training; ownership under matched familiarity) has not been run. We commit to reporting it either way.

Blade rule (all rungs). Any behavior surviving operator ablation is bypass and does not count — the same standard the tape was held to.

9. The workspace is a wire: Jacobian-lens results

Instrument. The Jacobian lens (Anthropic) asks, mechanically, what an internal activation is disposed to make the model say: lens_l(h) = unembed(J_l h), with J_l the input-averaged Jacobian from layer l to the final representation. No trained probes. On large models it reveals a workspace: a privileged subspace whose contents are verbalizable, surrounded by an ocean of computation that shapes behavior but is never linearly reportable — a structural echo of Global Workspace Theory's broadcast bottleneck.

Method. Faithful estimator port to cl33 (one-hot cotangents at all valid target positions, source-averaged, SKIP_FIRST=16; batch-replication for tractability). Taps: the six emitter layers h1–h6, the emitted operators B (post-clip), the transported state S (differentiable scan copy). Controls: stock GPT-2 124M and the warm-start +5B-mix GPT-2 control. Two cl33 models are reported — scale-C (163M) and scale-B (236M), both at 5B-token final — so the split is tested across two independent runs. Fitting used domain-balanced prompts (300 for the GPT-2 controls, 60 for cl33), 40 disjoint eval prompts. Metrics: future-token MRR and final-agreement per tap.

Results (final-agreement by depth, in-domain).

Five readings:

1. No workspace at 124M by default. Stock GPT-2's first seven layers are pure ocean — the pre-registered prediction, confirmed. Everything interesting it computes early is linearly invisible to its own output basis until the end.
2. The operators are the verbalizability peak of their own stack on every domain tested, and above every control layer in-domain — 0.613 (scale-C), above the warm control's final layer, replicated at 0.538 on the independently-trained scale-B. (One out-of-domain exception, reported: on PubMed the warm control's final layer, 0.563, edges scale-C's operator tap, 0.552.) Not mysterious, and that is the point: in a transformer the lens asks whether a layer happens to contain the output; in cl33 the operators are the only path to the output, so training must put the output there. The workspace is not emergent — it is the bottleneck, made.
3. The state is completely non-verbalizable. S scores 0.000 on both models — floor. This is the paper's ocean, realized as a named architectural component: the workspace/ocean boundary, statistical in standard models, is here the line between "emitted" and "transported" in the wiring diagram. An earlier draft paired this with "causally load-bearing"; the §3.1 audit weakened that pairing, and the 2026-09-08 constant-bias probe now closes it (§3.1, point 3): at the output, a residual bias out-steers an operator-field bias ~25×, and neither effect compounds with position — the state's accumulated history does not reach the readout. S is the non-verbalizable and output-causally-inert residue of the architecture: non-reportable and, for output steering, ≈0 causal share. The workspace/ocean split is thus doubly clean — what is transported is neither readable nor (at the output) load-bearing. The retro- explanation of the Claim-2b null ("order-content lives on the S side") remains demoted to hypothesis.
4. Upstream emitter organization is model-dependent; the bottleneck is not. scale-C's emitter pre-organizes — a smooth ramp .088 → .517 into B — while scale-B's emitter stays near-flat (.049 → .097) and jumps at B, closer to the original "dead emitter, workspace at B" pre-registration. Whether the bottleneck's gradient reaches back through the emitter varies with training (scale-B saw ~6× fewer optimizer updates); that B is the peak and S is the floor does not.
5. The split is domain-invariant (confound closed). The fit/eval prompts above share the training-mix domain, which could inflate absolute levels. To rule this out we re-fit and re-evaluated all four models on two held-out domains none trained on — PubMed abstracts (biomedical) and congressional bills (legal). The two load-bearing signatures survive both: cl33's B remains the verbalizability peak (scale-C B .552 / .751, scale-B B .490 / .721 on PubMed / legal) and S remains at floor (≤ .018 everywhere); stock GPT-2 keeps its dead-then-cliff shape (no mid-workspace) and warm-start keeps its bottleneck-free ramp, on both domains. Levels move with domain; shape does not. One honest scoping point falls out: on legal text — so formulaic that even shallow features predict the next token — the emitter lights up too (scale-C .54–.62), so reading #4's "dead emitter" contrast is itself a property of harder-to-predict text. What is architectural, and domain-robust, is B-is-peak, S-is-floor; how far B towers over the emitter is not.

Caveats. 124M–236M-class models only; first-pass estimator with 60 fitting prompts for cl33 (one repetitive-text lens-fit artifact noted: a single mid-emitter spike on scale-B/legal, with B still the clear peak); the emitter-organization effect (reading #4) deserves a controlled comparison — the same emitter trained without the bottleneck — before strong claims. The 5B-final rerun and the OOD domain-invariance control, both pre-registered gates in earlier drafts, are now discharged.

9.6 A regime sensor on the bottleneck (2026-09-08 — first measurement, one control open)

If the operators transcribe the emitter's state as well as its content, the bottleneck is an instrument, not only a record. First measurement: a plain difference-of-means direction in operator space separates text regimes (fiction vs code) at 0.962 held-out; four regime centroids (fiction / code / math / encyclopedic) classify a single token's emission four-ways at 0.802 (chance 0.25), spread across a multi-axis manifold (PC variance 0.52/0.28/0.19) whose axes each use the full compact+noncompact blade geometry (~uniform sector mass — no sector starvation). The load-bearing control: hold the token fixed and vary only the surrounding domain. Classification survives essentially unchanged (pooled 0.811: " the" 0.905, "." 0.877, "," 0.700) — the signal is contextual computational state, not lexical identity: the same symbol is emitted as a measurably different operator depending on the regime the emitter is in. Because the readout consumes only operators, the material the sensor reads is on the causal path by construction; whether the readout uses the particular direction the sensor reads is a separate question — one the closed-loop steering test below partially answers, and the open control decides. A first closed-loop test (steer the residual toward the math centroid, read the sensor) shows a monotone dose-response with the fixed-token invariance intact, and at sufficient gain the generated text itself turns numeric. One control is open and we hold the stronger claim until it closes: at low gain the sensor crosses the regime boundary before sampled generation does, and sensor and actuator directions derive from the same contrastive means — so the sensor-leads-behavior gap must be adjudicated (faithful early warning vs partial injection echo) by classifying sampled generations across the gain range before "state sensor" hardens beyond "readable projection of upstream computation."

10. Cost accounting and limits

The transparency tax, measured (locked, matched, fp32). With the from-scratch matched control complete, the tax is no longer provisional. On a single-session, full-set, right-padded, fp32 lm-eval-harness (v0.4.11), on a common wikitext ruler at 512:

The matched vanilla transformer wins bits-per-byte by +29% over scale-C (and +20% over the larger scale-B) and 6–7 of 8 downstream tasks. This is the honest cost of forcing every prediction through a reversible, per-token operator bottleneck. We report it plainly and make no claim of perplexity or reasoning competitiveness. An earlier draft guessed a ≈1.2× premium; the measured figure is larger (~1.29× on the loss ratio) and we correct it. Bits-per-byte is the confound-free ruler; word-perplexity dramatizes the same underlying gap through tokenization and chunking differences, and we do not report it as a cross-comparison.

A scaling dissociation, stated plainly. From 163M→236M, cl33's bpb improves (1.309→1.218) while avg-MCQ is flat (0.435→0.440): 73M extra parameters bought zero downstream accuracy. Downstream MCQ is param-saturated / data-bound at 5B and does not track cl33's own capability scaling — at fixed data it is the wrong yardstick for this architecture (the axis it is built for is §6; Table 1b below shows MCQ does move when the data moves). The model is genuinely fluent (coherent multi-sentence English) and its operator index sharpens with scale (nearest-neighbor same-category rate 0.79 at 5B vs 0.68 at 32M).

Table 1b — the tax is data-bound, not an architecture floor (2026-08-03). The scale-B model, continued on a prose-weighted mix to 6.2B tokens at 1024 context, was benchmarked on the same full-set fp32 right-padded harness (new session; the locked Table 1 above is untouched):

The bits-per-byte reaches parity with token-matched Pythia-160m to three decimals (1.15501 vs 1.15452) — the model-class fluency gap closes at matched tokens. Against the (now un-matched, still-5B) A1 control the gap narrows from +20% to +14%; the true matched tax at 6.2B is unmeasured — A1's own convergence (1.017 → 1.012 over its final 0.5B) suggests it sits near +14–15% rather than the +29% headline, but that is an inference, not a measurement. Three honest caveats: the Pythia row is carried over from the model-class context session (fp16; its source table marks it "scale intuition," not precision-matched — the parity claim is therefore cross-session), the prose-weighted mix partially aligns with the wikitext ruler (Pythia's Pile is also prose-heavy, so the class comparison remains roughly fair), and A1 was not extended, so Table 1's matched tax remains the only fully controlled number. MCQ also moved for the first time (+1.1pp avg; BoolQ +5.0pp — notable because a bias-immune re-analysis of BoolQ found cl33 the only model in the comparison with significant passage signal: answer-bias- corrected AUC 0.522, p = 0.037, versus Pythia's raw 0.61 collapsing to chance AUC 0.494 once its yes-prior is removed), qualifying the "param-saturated" reading above: MCQ was data-bound, and prose tokens were what it wanted. The claim discipline is unchanged — no superiority is asserted — but "the bottleneck costs a fixed 29%" is now falsified by the architecture's own training curve: the tax is a property of a checkpoint, not of the algebra.

The governance consequences of this cost table — where the tax inverts, and what exactly is guaranteed — are collected in Part III (§13).

Methodological note that gates the numbers. The lm-eval adapter inherited a left-padding step that corrupts the reversible scan (leading pads inject spurious rotors); it drove one binary task below chance until we replaced it with right-padding (batched output then matched batch-1 exactly). Every operator-model eval must right-pad. All cl33 harness numbers are full fp32 (emitter and algebra); the fp16-emitter split-precision path is training-only and never touched the eval numbers (verified: the eval adapter runs amp_emit=False).

† A1 stopped at 4.915B tokens (its GPU was reallocated ~1.7% short of the 5.0B budget). Its bits-per-byte had converged — 1.017 → 1.012 over the final 0.5B — so the token gap does not move the tax; we report it here rather than paper over it.

• Perplexity ruler. bits-per-byte, matched params/context/tokenizer/corpus, single session. word-perplexity against differently-chunked or longer-context baselines is not comparable and we do not report it as such.
• Compute. fp32 matrix_exp scan is the tax; split precision recovers most emitter throughput. Tape adds O(T²) addressing over 6-d keys (cheap constants, honest quadratic).
• Reversibility scope. Per-step exact inversion with anchoring; boosts create an expressivity/reversibility tension (expansion-as-memory) that we surface rather than hide — the compact/non-compact decomposition of learned directions is a pre-committed analysis (App. C).
• Not claimed: LM superiority; sub-quadratic memory; Claim 3; any consciousness gloss on the GWT correspondence — the correspondence is structural (reportable bottleneck vs non-reportable processing), and we use it only as a naming convention.
• Negative results reported: wedge associative memory (bypasses or fails); gauge-throttle optimizer (all levers hurt); Claim-2b (twice); sequential falsified configs in the lineage (v6-nuclear, Dec-steering, lattice-correlation-only, widen64) — each killed one alternative and is part of why the surviving architecture is this simple.

Part II — Memory on the Object

What the bus does not give for free, the falsifications that located the sole-channel requirement, and the organ that installs, adopts, and deletes with receipts.

3. Two memories, one identity

Question. When the model retrieves, does it use the operator machinery or route around it?

Setup. MQAR (multi-query associative recall; Beck et al. harness) for associative memory; a KGE-style relational task (compose R·head ≈ tail along paths) for relational memory. Both scored with the bypass blade.

Result (the dichotomy).

Result (the identity). The induction shortcut and the bypass are not two problems — they are the same operation. Recall-by-induction is token-similarity matching that never consults the algebra; the wedge-memory design matrix (§5.1) shows every configuration that recalls well before the tape does so through a token-content value channel, and every configuration that keeps values algebraic fails to recall (≈ 1/KV) at matched training budget — Appendix F shows the algebra-value store eventually groks to full recall at roughly 10× the training cost, so the identity is about learning efficiency and shortcut formation, not impossibility. The rotor scan cannot perform outer-product writes: a rotor is orthogonal under η, so "store this key–value pair" has no native expression in the transported state. This is the precise gap the tape (§5) is designed to close, and the reason its design rule is addressing/content separation rather than "more algebra."

Interpretation. Prior to the tape, cl33's honest self-description was: a navigator that borrows a transformer's memory. Navigation — composing a trajectory of operators — is native, transparent, and load-bearing. Recall was borrowed, opaque, and bypassing. The rest of the paper is about making memory native (§5) and then asking what the model can read of its own trajectory (§7–§9).

3.1 Channel-attribution audit (2026-07-15): which operator path is causal?

Operator-dependence throughout this paper is measured by full operator ablation (zeroing B everywhere). That blade is sound for the bypass question — does the capability live outside the algebra? — but it cannot localize within the algebra, because the emitted operators reach the readout by two distinct paths: via the scan (they transform the transported state) and directly as readout features (B plus its grade tower). An audit on a branching-track probe (infer a flow k from a demo, carry it across a filler gap, apply it to a novel start) decomposed the two, with three results.

1. The flow rides the feature path, not the state. Swapping B-conditioned operators into the scan alone leaves the branch on A (flip 0.000); swapping them into the readout features alone flips it completely (1.000). Replacing the scan outright with identity rotors costs ≈0 accuracy at every distance (0.96–1.00, including gap-1024 extrapolation at 8× the training gap); the state-only channel is weak and distance-limited (0.49→0.19 with distance).
2. Distance-flatness comes from attention, not transport. The emitter is itself a transformer attending over the whole visible context; the flow-conditioned operator it emits at the query is how the rule crosses the gap. The η-attention can be removed (scan_only) without loss; the emitter's internal attention cannot be — and masking the demo at generation time collapses continuation to chance even when the rule is inferable from tokens in view (with the recorded caveat that PAD-masked demos are out-of-distribution, so part of the collapse may be brittleness rather than dependence). Continuation is demo-conditioned emission, re-read each step, not a rule stored in the transported state. Relatedly, on-manifold state perturbations show no restoring dynamic: the output projection is insensitive to ~1× a class-separation of displacement at emission time, and trajectories knocked off their flow stay off — exactly what reversibility (no contraction) predicts.
3. The causal share of S at the output — measured (2026-09-08). A constant-bias steering probe on the frozen prose base closes what earlier drafts left open. At matched perturbation scale, a residual-stream bias produces ~25× the output effect (KL to the unperturbed distribution) of an operator-field bias on B_s; and at neither site does the effect grow with position (late/early ratio 1.0–1.1× at every magnitude). The second number is the decisive one: a persistent operator bias provably accumulates in the transported state through the rotor product, so if S's accumulated displacement reached the readout, the effect would compound along the sequence. It does not. Scan-compounding is falsified at the output: the state remains the substrate on which operators act, not a carrier whose history the readout consumes. The KGE navigation task of this section has still not been decomposed path-by-path ("operator-carried," op-dep 0.85, remains the honest label there), but the general hedge — "S's causal share is unresolved" — is now resolved for output steering: it is ≈0.

Design consequence (feeds §5). The tape's recall/receipt properties were measured directly — in configurations where the tape is the sole cross-position channel — and the same audit week recorded the matching caution: in any configuration where attention coexists with the tape, tape-causality must be re-verified, because if attention re-carries recall the tape becomes a passenger and editing it will not move the output. The subsequent v2.4 run measured exactly that failure mode (§5.4), which is why the successor designs make the memory the sole cross-segment channel by construction. The audit also sharpens the division of labor: the scan state alone neither recalls (§3) nor transports rules across distance; every distance-free capability we have measured is carried by an attention-addressed read of the operator record.

3.2 Independent replication and extension (Watson, 2026-09-10): the scan is

decorative at LM inference

An independent, pre-registered, eval-only audit by Nell Watson on the released checkpoints (both models, the shipped frozen slice and WikiText-103, fp32, hash-verified) first reproduced the bottleneck table to one decimal, then extended the §3.1 decomposition to the language model itself. Five conditions; PPL ratio vs native: operators zeroed 270×/115× (the bottleneck, replicated); S pinned to s₀ with attention on 1.19–1.22×; S time-shuffled 1.00×; η-attention zeroed 1.00×; S pinned and η-attention zeroed (readout sees only the local operator window through the grade tower) 1.00×. The time-shuffle control is the decisive one — we replicated it on our own harness (0.996) — and it shows the pinning cost is an artifact of the intervention rather than lost ordered history (Watson's shared-LayerNorm explanation is the candidate mechanism; it has not yet been separately isolated). The conclusion, stronger than §3.1 and measured on the LM proper: at inference on these checkpoints, the reversible scan and the operator-native attention over it are causally decorative; every bit of context reaches the output through the emitter's own attention, expressed in the emitted operators. Two scopings keep this honest in both directions: (a) in sole-channel/tape configurations (§5, §6) the scan's prefix products are the memory and ablating them destroys recall — the decorativeness is a property of the full LM configuration, not of the substrate; (b) the current evidence is consistent with training preferring the grade tower's direct route to B_t, B_{t−1}, B_{t−2} once it became available, but does not yet distinguish "the tower out-competed the scan" from "the scan mattered only as a training scaffold" from "the full-scale model never needed it" — and, as Watson notes, "the reversible path cannot do work" is not shown either. The v0 pre-tower model did use the transported state (the model_v2 docstring's "collapses to unigram" prediction was written for, and was true of, that configuration; it is stale for the shipped one). Half the mechanism is closed by algebra rather than ablation: §5.1's binding impossibility (a grade-1 carrier under the grade-preserving action cannot hold grade-2 content) already establishes the substrate-inadequacy branch for the memory function, with its constructive converse measured in the left-regular successor line. The two controls that would close the residual — whether the state could have carried generic context and the tower out-competed it (a no-tower model), or contributed nothing even during training (a no-scan-from-init model) — are queued as a v1.3 table at probe scale, scoped to that narrower question.

One reading of this result strengthens the transcript claims rather than qualifying them. If the readout consumes only a local operator window and the transported history carries nothing, then each emission must be a contextually complete summary: the emitter's attention integrates the visible history into every B_t before emission. That is the mechanism behind the density of the record — why a single emission decodes its token at 0.86 with +4.7 bits beyond prefix predictability (§7), and why regime state reads out of one token's operators under lexical control (§9.6). The log is information-rich because the decision path is local: everything the context contributes has to be compressed into the emission itself. And the completeness guarantee is unchanged in the only direction governance needs: nothing behaviorally relevant can bypass the record — what the model does not transcribe, it provably cannot act on.

4. Navigation parity (pre-registered)

Task. Permutation composition — apply a composition of permutations read from context; a TC⁰-flavored weak spot for fixed-depth transformers and the capability class closest to cl33's native operation (rotors compose; permutations are rotors at the integer points).

Protocol (pre-registered; deviations logged in the pre-reg memory). Learning-rate sweep for both models; winner LR re-run with 3 seeds; report mean ± sd of the mean-of-3 at the winning LR. (An earlier "cl33 matches-or-beats" framing from a max-of-5 readout was retracted when the mean-of-3 protocol shed the max-inflation — we report the corrected protocol only.)

Result.

Statistically indistinguishable. Two secondary observations: cl33's seed variance is ≈3× tighter, and one cl33 seed (43) reached its score with zero measured bypass — an existence proof that the parity-level solution can live entirely in the algebra. Mechanism note: grokking on this task coincides with rotor spectra collapsing toward permutation scale.

What we claim: parity + consistency + inspectability on the task family nearest the architecture's native operation. What we do not claim: superiority. A gauge-throttle optimizer intervention intended to force more-algebraic solutions was falsified (every lever hurt; weight decay worst) and is reported in App. D as a negative result.

5. The reversible tape: exact memory native to the scan

5.1 The failure that shaped the design

We first attempted associative memory as a wedge-product matrix store (C += k∧v read by q⌋C — mLSTM's matrix memory expressed in Cl(3,3)). The design matrix:

Context-mixed keys cannot re-form at query time (falsified twice); token values recall but bypass. The untested cell — token addressing with algebra-only content — is the tape.

5.2 Design principle

Token content may SELECT; only algebra may FLOW.

The scan already writes a lossless record: prefix products P_t = R_t·…·R_1 give the exact relative transport R_{t←i} = P_t · ηP_iᵀ η between any two positions, for free, by reversibility. The tape adds only a read head: a context-free grade-1 projection of the token embedding produces match scores α over strictly-past positions (with an induction shift: match position i, read i+1); the returned content is Σ α_i · f(tape_i) where f yields exclusively algebra objects. Token information can influence which positions are read (a low-bandwidth softmax channel) but no token content vector ever reaches the readout. Numerical discipline: raw prefix products diverge by T ≈ 1024 (error 1.3e6); anchored re-orthogonalization every K = 64 steps is mandatory and cheap.

5.3 Value modes and what each one is

Three findings:

1. Recall without bypass, both blades. Increment mode solves MQAR perfectly at every key–value load, and ablating the operators destroys recall (ratio 4.88×, far above the ≈1 a bypass would show). The two-memory gap of §3 is closed natively in this configuration — scan-only at probe scale, with the tape the sole cross-position channel; §5.4 records what happened when that condition was relaxed.
2. The mode taxonomy is a reader taxonomy. The same tape read three ways yields three different capabilities: increments give adjacency (recall), displacements give path integration (helps general prediction: best PPL), states give a cheap high-transparency feature channel. This taxonomy does independent work in §8.
3. The attention-free challenger. Scan + tape with no attention at all reaches VAL 19.02 at matched 32M/8k — beating the full model with tape (20.10) and the no-tape baseline (21.30). Honesty: this is not sub-quadratic — tape addressing is an O(T²) softmax over 6-d token keys. The win is simplification + quality, not complexity class. (A pre-registered 5B challenger run of this config was planned; it was superseded by the v2.6 redesign before running — the 5B evidence on η-attention's inertness, §3.1, arrived first.)

5.4 Pre-registered v2.4 gates

The 5B retrain (multi-mode tape, anchored displacement) carried pre-registered gates — PPL vs the from-scratch matched control, MQAR at deploy, bypass ratios per mode, J-lens rerun (§9) — recorded in CL33_OPLM_V24_TAPE_LM.md before the runs.

A co-adaptation regression, diagnosed and repaired (reported as a result). The first co-trained tape run (warm-started from a no-tape 5B checkpoint, with an auxiliary MQAR loss to teach addressing) reached the tape gate — recall 0.77 within the first fifth of the token budget — but regressed language modeling: a matched-stream fp32 comparison showed the shared base pulled off-optimum (base-alone val 31→88 across training). The mechanism is not a gradient conflict — the LM and auxiliary gradients are near-orthogonal on the shared emitter (cosine ≈ 0) — but a gradient magnitude imbalance: the auxiliary signal is ~4.6× larger there and additionally reshapes the shared readout toward memory tokens. The fix is a principled coupling, not a loss weight: cap the auxiliary gradient magnitude to a fraction (κ = 0.5) of the LM gradient's on the shared emitter, and route it off the shared readout so memory output is carried by a dedicated head. Under the fix the result exceeds the naive run on both axes: the deployed model returns to the LM baseline (1024-context fp32 val 32.5→31.4 over steps 2k–6k, versus scale-B's 30.97) while recall climbs past the naive run's plateau (0.77→0.80→0.89) and the hollowed base heals monotonically (base-alone 87→62→52). The naive coupling had bought recall 0.77 at a standing +11% LM penalty; the principled coupling buys recall 0.89 at ≈baseline LM (+1.2% vs scale-B, with the caveat that the deployed 1024-context val and the 512-context baseline are not exactly the same ruler). This is the kind of pathology the operator factoring makes visible and addressable: because the memory path and the base path are typed and separable, the imbalance is measurable per-parameter-group and correctable as a gradient operator rather than a blind hyperparameter search.

What the calibrated instruments then showed (the honest coda). Three follow-up measurements, run within days of the repair, scope the 0.89 sharply and are the reason the successor designs exist. First, the 0.89 is a short-range number: the benchmark's query–pair gap is ~40 tokens, and a calibrated distance sweep found recall collapsing 0.94 → 0.05 as the gap grows toward 448 (worse with natural-text filler) — the work log records this as falsifying the tape's distance-independent framing, and we adopt that verdict here. Second, the learned address projection — context-free by design — was co-opted by language-modeling context on natural text (exact-match addressing 0.006 against 0.26 on synthetic), the design property failing to survive co-training. Third, decode-level attribution in the full configuration showed the tape's own vote on recalled tokens at 0.000, with --no_tape recall ≥ tape-on: in the presence of attention, attention re-carried recall and the tape rode as a passenger — precisely the §3.1 caution realized. These three results — distance-limited addressing, address co-option, and passenger collapse — are not appendix caveats; they are the measured boundary of the v2.4 design and the direct motivation for the dual-address revision and the segment-recurrent redesign in which the memory is the sole cross-segment channel by construction (§12).

6. Capability doors: editable, provenance-bearing, load-invariant memory

(Full pre-registration + adjudication: CAPABILITY_DOORS.md. All results here are on the synthetic MQAR task at probe scale, in scan-only configurations where the tape is the sole cross-position channel. The §5.4 coda applies: in configurations where attention coexists with the tape, edit/receipt causality must be re-verified — the v2.4 full config measured the tape as a recall passenger, and editing a passenger does not move the output.)

Once memory is a typed, exact, position-addressed record (§5), three capabilities follow that no opaque KV cache offers. Each carries the bypass blade.

Door 2 — editable memory (transaction semantics). The counterfactual-override path is used as a write API on a trained increment-tape model: swap two records (queries return the swapped value, efficacy 1.00), delete a record (target recall → 0.008 ≈ chance, other pairs unchanged — locality 1.00), transplant a record from a different sequence (foreign value retrieved, 1.00 — records are portable at this scale), and undo (bit-exact restoration, max |Δlogits| = 0). Identity override is also exact (Δ = 0). This is surgical context editing, auditable RAG, and — since delete is targeted forgetting with a receipt and an undo — machine unlearning with provenance.

Door 3 — provenance receipts (faithful, causal attribution). The tape's address weights are a faithful attribution: argmax-α identifies the ground-truth write position with attribution@1 = 1.000, and the receipt is causal — deleting the argmax- addressed record drops that query's recall to 0.000 while deleting a non-source record leaves it at 1.000. A matched transformer's best attention head reaches 0.977 only under oracle head-selection, and that oracle head is not stable across load (it drifts L1H2→L1H3→L0H1 across vocab/length; where the transformer's attribution collapses under load, the diagnosis is that no attribution circuit formed — not that attention "smears"). The tape's address is a single canonical, causally editable interface; the transformer's is an emergent, condition-dependent one.

Door 1 — load-invariance at T = 1024, and a pre-registered fairness sweep. On MQAR the scan+tape holds ceiling as load grows (KV 64/128/256 at T=1024 → 0.997/0.999/0.999, zero bypass), while the same fixed-96-d-state model without the tape sits at chance — the tape rescues exact recall the recurrence structurally cannot do. Validation stops at T=1024: a T=2048/KV=128 run did not grok within the grid's step budget (an honest gap; T≥2048 is unverified). The matched d128 transformer, by contrast, failed to learn T=1024/KV=64 across a pre-registered six-config sweep (three learning rates at depth 2, plus depth-3, depth-4, and 2×-width runs at the winning rate; all 12k steps, all ≈ chance), where the tape reaches 0.99 in under 5k steps. We scope this honestly: MQAR is provably solvable by attention at sufficient scale (Zoology built it to showcase attention), so the claim is not that transformers cannot — it is capacity + learning efficiency: the tape forms exact, zero-bypass, editable, receipt-bearing retrieval at lower capacity and fewer steps than the transformer needs to form any retrieval at all.

12. The memory organ: from deadlock to a certified peripheral (2026-08 → )

The results above treat memory as a channel inside the operator LM. A successor program — reported here as measured intermediates with pre-registrations, not finished claims — grafts the Genesis two-channel memory (validated separately on frozen GPT-2/Gemma/R1 backbones) onto the frozen prose base of Table 1b as an explicit register-bank organ: 0.1–2.5M trainable parameters (per arm) against 236M frozen, wedge-plane records with trained-encoder addressing, zero-init injection, and a removability gate that holds at literally 0.00e+00 ΔCE in every run (organ-off ≡ base).

The program's value so far is a diagnostic ladder — each rung a small set of pre-stated changes with separated instruments, each verdict measured:

1. Interface deadlock. Next-token CE alone never differentiates addressing — the register softmax stays exactly uniform (9,000 steps with raw-embedding features; 3,000 with emitter-tap features before that arm was cut). Cause: a chicken-and-egg between addressing and decoding, plus retrieval-demand density orders below every precedent that formed recall.
2. Labels break it. Planted-episode data makes the correct register a free label; a supervised address loss moves address accuracy 0.000 → 0.5 in 100 steps — what CE never did in 9,000.
3. Context is mandatory for addressing. Raw-embedding queries plateau structurally (the query token is a generic "is"); features tapped from three emitter blocks address unique-vocabulary episodes at 1.000.
4. Retrieval forms. With episode demand, seam-weighted loss, and per-8-token registers: sustained recall 0.458 with recall-off = 0.000 (zero bypass) — and a clean granularity dose-response (P=8 beats P=16 on both axes). Measured at full episode density: recall never crossed the pre-set 0.5 threshold that would have opened the density anneal, so retrieval-under-realistic-sparsity — the program's pre-registered decision point — remains unmeasured, and clean-text ΔCE stayed negative at this density.
5. The decode ceiling is the read mixture, not the decoder. At matched read width (H=8), two structurally different decoders — a free logit matrix and a KGE-style codebook whose write and read sides are the same tensor — land on the same conditional recall (0.565 / 0.567 given a correct register), localizing the ceiling to the unsupervised within-register mixture both consume. A 2×-width read measures conditional recall 0.716 (n=102) — width partially dilutes mixture interference — but pays for it in slower addressing, leaving total recall unchanged; the mixture, not the decoder, remains the binding constraint. The pre-registered test that removes the mixture entirely (per-token registers: the register is the binding; predicted conditional decode ≥ 0.85) has now run — rung 6.
6. Mixture removed: the memory path is exact, and the losses decompose into three named numbers. With per-token registers and the tied codebook, the memory path alone decodes at 1.000 (53/53): given the right register and a pure read, the record is not lossy at all. The remaining gap to headline recall then separates cleanly — soft-α selection conditional 0.463 vs hard top-1 read 0.756 (the α-softness cost, ~0.29), hard read 0.756 vs pure decode 1.000 (the integration cost through the frozen base, ~0.24), and addressing itself at 0.57–0.69. Nothing is mysterious in the residual: an exact record, read through a soft mixer, injected into a base that never trained to consume it. Each named loss is a plumbing target, not a representation problem — the record layer is exact on that evidence; which value substrate feeds it best is adjudicated by the control arm below.

The endpoint has a first measurement. The battery the governance section implies — install → adoption → certificate-bearing forget → revert — was run as a pilot on the rung-6 checkpoint: 24 facts never present in any context, written through the organ's own write path onto virtual segments, queried cold. Install top-1 0.750 (median rank 1, mean Δlogprob +8.80); in generation, 4 of 4 sampled queries adopt the installed fact; algebraic delete → 0.000; organ-off → 0.000 (zero bypass); address receipts 24/24 (the α argmax lands on the installed register in every case). Install efficacy landing on the same number as the hard-read integration ceiling (0.750 ≈ 0.756) says the pilot is at the measured plumbing limit — installs are as good as integration currently allows, no worse. Pilot-scale, single-seed, one config; reported as such.

The installed belief passes through the transcript. A four-cell factorial on the same checkpoint separates the two paths by which a read can reach the output: internal injection only 0.407, direct logit path only 0.017, both 0.712, neither 0.000 — interaction +0.288, internal-carrier fraction 0.571. The dominant carrier is the pre-bottleneck injection: the retrieved memory alters which operators get emitted, and the behavior change rides those operators to the readout. The receipt is direct: on install queries the emitted B_s displaces by 0.777 on hits vs 0.591 on misses. Because the readout consumes only operators (§2), this closes the chain memory → exposed cognition → behavior with a measurement at each arrow — the property an opaque-base memory graft cannot exhibit even in principle.

Two methods results the sweep paid for. First: injection-layer arms L6 and L11 both matched the incumbent layer (L9) on every training-time validation metric, yet measured sharply different install efficacy (L11: 0.083 with a validation profile identical to L9's at the same step). Selection by validation recall would have chosen wrongly; only the install battery measures leverage, and the pre-registered rule — primary metric: install efficacy; validation curves never select — is what caught it. Second, and larger: the incumbent's efficacy number (0.750) came from a checkpoint trained 3× longer than the sweep arms (no intermediate checkpoint had been kept) — precisely the step-mismatched comparison the pre-registration forbade. Re-running L9 at the arms' budget eliminated its apparent advantage entirely: at matched 5k steps, L9 measures 0.208 — identical to L6 (secondaries lean L9: median rank 7 vs 20; secondaries do not select). The "3.6× layer effect" the mismatched comparison suggested was in fact a training-duration effect: install leverage at the same layer grows 0.208 → 0.750 from 5k → 15k steps of interface training. The corrected reading: among tolerant layers, where you inject matters far less than how long the interface trains — and a plausible, val-supported, wrong conclusion was two pre-registered rules away from the paper.

Speed vs ceiling, adjudicated under pre-registration. A control arm with learned operator-derived values (no codebook) was predicted to hit a representational ceiling well below the codebook's exact decode. The prediction landed in its pre-registered middle branch: the control escaped the predicted ceiling family (hard-read conditional 0.871; memory-only 0.914) yet stayed short of exact (codebook 1.000), and it out-integrated the codebook at every in-pipeline mode while acquiring faster — opening the program's density anneal first (dense-plant training relaxed to realistic sparsity with recall retained at 0.44 and clean-text ΔCE healed to ≈0: the dense curriculum is a learning scaffold, not an operating requirement). The codebook's exactness holds at the record layer; its end-to-end advantage is not established, and the two arms differ by a curriculum confound (only the control reached the sparse regime) that the next arm is designed to remove. We report the branch structure rather than a winner because the pre-registration was written before the numbers existed.

The remaining open items are plumbing — addressing sharpness, per-position integration gating, read abstention — and the full battery at scale. Every rung above, including the four nulls and the wrong predictions, is logged with dates and checkpoints in the project work log.

Part III — Governance

What the receipts buy, which reversion claim is being made, and the doors through which weaker guarantees re-enter.

13. Governance: the audit object

Where the tax inverts. Table 1 compares cl33 to an unconstrained transformer on perplexity. In a regulated or high-stakes deployment the relevant comparison is different — cl33 versus transformer-plus-its-audit-stack on guarantee-per-dollar — and it flips. Post-hoc interpretability (SAEs, probes, attention attribution) is itself a compute-and-process tax that buys only an approximation: §6 Door 3 makes this concrete, with the transformer's best attention head attributing the true source at 0.017 (rank 40) and drifting across load, versus the tape's exact, causal 1.000 receipt. cl33's audit trail is the computation's mandatory carrier — the operator record through which all output flows (270× on ablation, independently replicated) — and the reverse trace (R⁻¹ = ηRᵀη, exact to 2e-14) makes that record losslessly replayable, not a learned proxy of it. The refinement §3.2 requires, stated plainly: the record is causal; the scan's invertibility is a property of the log, not of the decision path, whose transported state is measured decorative at LM inference. For deployments that must prove what a model did — machine unlearning with a receipt (Door 2), decision provenance, tamper-evident memory — the ~1.29× perplexity tax becomes the price of a guarantee that post-hoc tooling cannot provide at any cost. We scope this precisely: what is guaranteed is that the computation is invertible, the memory editable with receipts, the emission decodable — not correctness, calibration, or non-hallucination. Transparency is not correctness; a cl33 model can be confidently wrong, but you can prove exactly how it got there and edit it.

What "revert" means — a three-tier hierarchy. "Install a belief, remove it, and the model reverts" is three claims of different strengths, and the architecture supports stating which one is being made:

1. Computational reversion (strongest, measured). With the base frozen and the organ zero-initialized-and-gated, deleting a register restores the bit-exact forward computation on the same prompt: organ-off ≡ base to 0.00e+00 ΔCE in every run, and the §12 pilot measures delete → 0.000 with the installed fact unreachable. This tier is a property of the frozen-substrate design, not of training success.
2. Behavioral reversion (fresh prompts). After deletion, novel prompts probing the belief return to baseline behavior. Measured in the pilot at the same 0.000; at scale this tier requires a battery, not an identity argument, and we scope it as such.
3. Conversational reversion (weakest — inexact by construction, auditable). If the installed belief influenced generated text that re-entered context, deletion does not un-say it: the influence has sedimented into the conversation history, which is ordinary context the model rightly conditions on. This is the operator-model form of the sediment lesson (not-recoverable ≠ not-behaviorally-causal). What the architecture offers here is not exact reversion but an audit trail: the receipts date-stamp exactly which generations occurred under the installed belief, so the contaminated span is identifiable even though it is not erasable.

Where the sediment caveat re-enters (three doors, named so they can be watched). The tier-1 guarantee is conditional, and each condition is a door through which weaker-than-computational reversion re-enters: (a) co-training — any arm that unfreezes the base (§12's v3.1 option) trades bit-exact organ-off for "characterized degradation," a trade that must be made knowingly and labeled; (b) persistent banks — when memories are written by processes that themselves read memory, deleting a record does not delete its causal descendants; a write-provenance cascade (each record carrying the receipts of the reads that produced it) is the pre-registered requirement before any long-lived bank is deployed; (c) context sedimentation — tier 3 above, present in any deployment where outputs re-enter inputs. None of these voids the architecture's guarantees; each converts one guarantee from an identity into a measurement, and the receipts are what make that measurement possible.

14. Related work

Linear-recurrent and matrix memories. mLSTM/xLSTM (Beck et al., 2024), DeltaNet-style fast weights (Schlag et al., 2021; Yang et al., 2024), and state-space models of the Mamba lineage (Gu & Dao, 2023; Dao & Gu, 2024) form the honest competitive set for Appendix F (the linear-transparent store): linearity without receipts. The lineage runs from the original fast-weight programmer (Schmidhuber, 1992) through fast-weight associative memory (Ba et al., 2016) and the linear-attention recurrence (Katharopoulos et al., 2020) to their identification (Schlag et al., 2021). Our wedge store is the Cl(3,3) expression of the mLSTM outer-product memory, and its measured failure modes (91% bypass with token keys; slow grokking with algebra values) delimit ours rather than flatter them. The recall–throughput frontier this family trades on is mapped by Arora et al. (2024); our tape sits off that frontier deliberately — an honest O(T²) for exact, receipted recall. Memory-based test-time adaptation (Sun et al., 2024) is the nearest frame for the tape-as-TTT reading (§6): where TTT layers update a hidden model by gradient, the tape substitutes an exact, undoable memory write. What none of this family provides — and what the tape and register formulations are for — is causal attribution receipts and exact, certificate-bearing deletion.

Induction heads and in-context retrieval. The two-memory identity of §3 (recall-as-induction = routing-around-the-operators) is an architectural corollary of the induction-head literature (Olsson et al., 2022), and our recall instrument is the MQAR task of Arora et al. (2023): we do not dispute that transformers ride token-similarity shortcuts; we make the shortcut visible and ablatable and show what remains without it.

Model editing and unlearning. ROME/MEMIT-style weight editing (Meng et al., 2022; Meng et al., 2023) modifies parameters with no certificate and contested locality (Hase et al., 2023); machine-unlearning methods (Bourtoule et al., 2021) approximate removal with no proof object. Our editable-memory results (§6, Door 2) claim something narrower and stronger where it applies: exact algebraic erasure from the explicit memory state, with a receipt, accompanied by behavioral reversion — scoped explicitly NOT as unlearning of the backbone.

Steering and representation control. Steering by added directions is established for standard transformers — representation engineering (Zou et al., 2023), activation addition (Turner et al., 2023), and persona vectors (Chen et al., 2025). §9.6 and App. C place cl33 in that family with one structural difference: the steered object's downstream effect is readable from a mandatory typed interface (the operator stream), so dose–response, saturation, and regime adoption are measured on the causal path rather than probed beside it.

Reversible architectures. RevNets (Gomez et al., 2017) and reversible transformers (Kitaev et al., 2020) invert computation for activation-memory efficiency — the inversion exists to avoid storing. Here reversibility is the explanation interface: R⁻¹ = ηRᵀη is run backward to produce the audit trail, and the 2e-14 exactness is a property the paper's governance claims rest on.

Geometric and Clifford networks. Clifford/geometric-algebra layers (Brandstetter et al., 2023; Ruhe et al., 2023; Brehmer et al., 2023) are typically used as equivariance-preserving feature maps inside otherwise-standard readouts. cl33's distinction is exclusivity: the algebra is not a feature map but the only path to output, which is what converts interpretability questions into ablation experiments.

Lens methods and verbalizability. Logit-lens (nostalgebraist, 2020) and tuned-lens (Belrose et al., 2023) read representations through trained probes, inheriting the faithfulness question shared by probing classifiers generally (Belinkov, 2022) and attention attribution in particular (Jain & Wallace, 2019); sparse-autoencoder dictionaries (Bricken et al., 2023; Cunningham et al., 2024) buy interpretable features at the same post-hoc remove. The Jacobian-lens instrument of §9 estimates verbalizability without a trained readout, and the three-model comparison places the operator stream above both GPT-2 controls' best layers.

Global Workspace Theory (Baars, 1988; Dehaene et al., 1998) supplies naming only (reportable bottleneck vs non-reportable processing); no cognitive claim is made or needed. TC⁰ expressivity limits (Merrill et al., 2022; Merrill & Sabharwal, 2023) supply the provenance of the permutation-composition task used for the §4 parity study, with the task's transformer-hardness established by Liu et al. (2023) and its state-tracking framing by Merrill et al. (2024). Baselines throughout are GPT-2 (Radford et al., 2019) and the token-matched Pythia suite (Biderman et al., 2023).

References

• Arora, S., Eyuboglu, S., Timalsina, A., Johnson, I., Poli, M., Zou, J., Rudra, A., & Ré, C. (2023). Zoology: Measuring and improving recall in efficient language models. arXiv:2312.04927.
• Arora, S., Eyuboglu, S., Zhang, M., Timalsina, A., Alberti, S., Zinsley, D., Zou, J., Rudra, A., & Ré, C. (2024). Simple linear attention language models balance the recall-throughput tradeoff. arXiv:2402.18668.
• Ba, J., Hinton, G., Mnih, V., Leibo, J. Z., & Ionescu, C. (2016). Using fast weights to attend to the recent past. NeurIPS 2016. arXiv:1610.06258.
• Baars, B. J. (1988). A Cognitive Theory of Consciousness. Cambridge University Press.
• Beck, M., Pöppel, K., et al. (2024). xLSTM: Extended Long Short-Term Memory. NeurIPS 2024. arXiv:2405.04517.
• Belinkov, Y. (2022). Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1), 207–219.
• Belrose, N., Ostrovsky, I., McKinney, L., Furman, Z., Smith, L., Halawi, D., Biderman, S., & Steinhardt, J. (2023). Eliciting latent predictions from transformers with the tuned lens. arXiv:2303.08112.
• Biderman, S., Schoelkopf, H., Anthony, Q., et al. (2023). Pythia: A suite for analyzing large language models across training and scaling. ICML 2023. arXiv:2304.01373.
• Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., et al. (2021). Machine unlearning. IEEE S&P 2021, 141–159. arXiv:1912.03817.
• Brandstetter, J., van den Berg, R., Welling, M., & Gupta, J. K. (2023). Clifford neural layers for PDE modeling. ICLR 2023. arXiv:2209.04934.
• Brehmer, J., de Haan, P., Behrends, S., & Cohen, T. (2023). Geometric algebra transformer. NeurIPS 2023. arXiv:2305.18415.
• Bricken, T., Templeton, A., Batson, J., et al. (2023). Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread. transformer-circuits.pub/2023/monosemantic-features.
• Chen, R., Arditi, A., Sleight, H., Evans, O., & Lindsey, J. (2025). Persona vectors: Monitoring and controlling character traits in language models. arXiv:2507.21509.
• Cunningham, H., Ewart, A., Riggs, L., Huben, R., & Sharkey, L. (2024). Sparse autoencoders find highly interpretable features in language models. ICLR 2024. arXiv:2309.08600.
• Dao, T., & Gu, A. (2024). Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality. arXiv:2405.21060.
• Dehaene, S., Kerszberg, M., & Changeux, J.-P. (1998). A neuronal model of a global workspace in effortful cognitive tasks. PNAS, 95(24), 14529–14534.
• Gomez, A. N., Ren, M., Urtasun, R., & Grosse, R. B. (2017). The reversible residual network: Backpropagation without storing activations. NeurIPS 2017. arXiv:1707.04585.
• Gu, A., & Dao, T. (2023). Mamba: Linear-time sequence modeling with selective state spaces. arXiv:2312.00752.
• Hase, P., Bansal, M., Kim, B., & Ghandeharioun, A. (2023). Does localization inform editing? Surprising differences in causality-based localization vs. knowledge editing in language models. NeurIPS 2023. arXiv:2301.04213.
• Jain, S., & Wallace, B. C. (2019). Attention is not explanation. NAACL-HLT 2019. arXiv:1902.10186.
• Katharopoulos, A., Vyas, A., Pappas, N., & Fleuret, F. (2020). Transformers are RNNs: Fast autoregressive transformers with linear attention. ICML 2020. arXiv:2006.16236.
• Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020). Reformer: The efficient transformer. ICLR 2020. arXiv:2001.04451.
• Liu, B., Ash, J. T., Goel, S., Krishnamurthy, A., & Zhang, C. (2023). Transformers learn shortcuts to automata. ICLR 2023 (oral). arXiv:2210.10749.
• Meng, K., Bau, D., Andonian, A., & Belinkov, Y. (2022). Locating and editing factual associations in GPT. NeurIPS 2022. arXiv:2202.05262.
• Meng, K., Sen Sharma, A., Andonian, A., Belinkov, Y., & Bau, D. (2023). Mass-editing memory in a transformer. ICLR 2023. arXiv:2210.07229.
• Merrill, W., Sabharwal, A., & Smith, N. A. (2022). Saturated transformers are constant-depth threshold circuits. TACL, 10, 843–856. arXiv:2106.16213.
• Merrill, W., & Sabharwal, A. (2023). The parallelism tradeoff: Limitations of log-precision transformers. TACL, 11. arXiv:2207.00729.
• Merrill, W., Petty, J., & Sabharwal, A. (2024). The illusion of state in state-space models. ICML 2024. arXiv:2404.08819.
• nostalgebraist (2020). interpreting GPT: the logit lens. LessWrong. lesswrong.com/posts/AcKRB8wDpdaN6v6ru.
• Olsson, C., Elhage, N., Nanda, N., et al. (2022). In-context learning and induction heads. Transformer Circuits Thread. arXiv:2209.11895.
• Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI technical report.
• Ruhe, D., Gupta, J. K., de Keninck, S., Welling, M., & Brandstetter, J. (2023). Geometric Clifford algebra networks. ICML 2023. arXiv:2302.06594.
• Schlag, I., Irie, K., & Schmidhuber, J. (2021). Linear transformers are secretly fast weight programmers. ICML 2021. arXiv:2102.11174.
• Schmidhuber, J. (1992). Learning to control fast-weight memories: An alternative to dynamic recurrent networks. Neural Computation, 4(1), 131–139.
• Sun, Y., Li, X., Dalal, K., et al. (2024). Learning to (learn at test time): RNNs with expressive hidden states. arXiv:2407.04620.
• Turner, A. M., Thiergart, L., Udell, D., Leech, G., Mini, U., & MacDiarmid, M. (2023). Activation addition: Steering language models without optimization. arXiv:2308.10248.
• Yang, S., Wang, B., Zhang, Y., Shen, Y., & Kim, Y. (2024). Parallelizing linear transformers with the delta rule over sequence length. NeurIPS 2024. arXiv:2406.06484.
• Zou, A., Phan, L., Chen, S., et al. (2023). Representation engineering: A top-down approach to AI transparency. arXiv:2310.01405.

Appendices (planned)

• A. Full MQAR battery — all KV loads × modes × η sweep; TTT-free.
• B. Numerics — anchored prefix products, T=1024 divergence study, fp32/bf16.
• C. Steering & the four operations — per the publication gates: full analogy battery with rank distributions (not exemplars), read-vs-write baselines on emitter hiddens and matched transformer embeddings, steering statistics (N×M success rate, self-PPL coherence, off-target drift), compact/non-compact decomposition.
• D. Negative results in full — wedge matrix, gauge throttle, 2b probes, phase-0 Claim-3 confound and redesign.
• E. 5B pre-registration — gates verbatim, with hashes/dates, before unblinding.
• F. Linear-transparent memory — the trilemma and its grok-driven break (moved from §11; drafted below).

Appendix R (drafted): reproducibility manifest

Every table maps to an in-tree script, a checkpoint, and an exact command, and every result cites its dated work-log entry (all paths relative to the cl33_oplm project root; run logs preserved verbatim).

Released artifacts. The two central frozen checkpoints are public at huggingface.co/mirrorethic/cl33-oplm — the prose base of Table 1b (236.5M, step 189307) and the serving/demo model of cl33.t3atlas.dev (236.5M, step 13996) — with the load-only model definition, the reverse-readout probe (its held-out evaluation card stored inside the artifact), SHA-256 hashes, and two verified one-command reproductions: repro_bottleneck.py (zeroing the emitted operators multiplies perplexity exactly 270× on a shipped frozen slice of the validation mix — a different draw of the same distribution measured the paper's 314× — and ≈106×/≈112× on public WikiText-103, the off-domain form of §1's bottleneck claim) and repro_reverse_readout.py (text decoded from the operator stream alone, probe top-1 0.860 held-out). The bundle's REPRODUCE.md states its scope plainly: it contains what is necessary to independently test the published claims on frozen artifacts, not the training stack or the §12 program (which the paper itself labels ongoing). Reproducibility surface ≠ complete source disclosure; what a reader needs is the artifact and the measurement, and both are now unconditional.

Baselines: Pythia-160m rows use public HuggingFace revisions (step-matched); GPT-2 rows use the public 124M release. The eval adapter's right-padding requirement (§10 methodological note) applies to every cl33 row.

Sources of record: WORK_LOG.md (this dir), REVERSIBLE_TAPE_DESIGN.md, SELF_STRUCTURE_CLAIMS.md, CL33_OPLM_V24_TAPE_LM.md, CL33_OPLM_ARCHITECTURE.md, CL33_OPLM_V23_RM.md, nav parity pre-reg memory, J-lens artifacts at /mnt/consciousness-storage/jlens/.

Appendix F (moved from §11): Linear-transparent memory — the trilemma and its break

(Design + pre-registration: LINEAR_TRANSPARENT_MEMORY.md.)

The tape's exactness and receipts come at an O(T²) softmax address (it materializes a distribution over all past records). The field's linear-time memories (mLSTM, DeltaNet, Mamba) compress records into a fixed-size state — O(T) — but have neither receipts nor surgical editability. Can one architecture have all three of {linear, transparent, recall}? We hold both endpoints: the wedge (mLSTM expressed in Cl(3,3): O(T), but its token-content value channel bypasses the operators 91%) and the tape (O(T²), zero-bypass, attribution@1 = 1.0).

A three-corner trilemma, then broken. At a probe configuration (KV=8), three variants: token-value wedge — recall 0.87 but 95% bypass, O(T); tape — recall 1.0, zero bypass, O(T²); and the untested cell, an algebra-value wedge (token key + algebra-only value) — O(T) and zero-bypass, but recall only 0.26 at a short training budget. The apparent trilemma (any two of the three) dissolved on inspection: the algebra-value wedge was still climbing. Given the budget to converge, it grokked to 0.99 recall — linear, zero-bypass, and accurate, all three — via a phase-transition jump (0.39→0.94 between 3k and 5k steps, cross-entropy collapsing), at ~10× the tape's learning time.

Why it groks (a mechanistic account, contributed by the collaboration). Quadratic memory externalizes address formation into an explicit distribution over separately-kept records — nothing to learn. Linear memory must learn an internal geometry that simultaneously prevents write-interference and supports decoding. The token-value wedge is fast only because token embeddings hand it a ready-made near-orthogonal basis (that is the bypass); the algebra-value wedge must build that geometry from the operators, which is a genuine — and slow — representation-learning problem. The grok jump is the signature of finding that geometry.

The honest boundary. This is an existence proof at toy scale: at a realistic 8k vocab the algebra-value wedge did not grok within the same budget — the fixed-size compression's decoding capacity is the binding constraint, and it scales with vocabulary. So linear + transparent + recall is achievable but not yet tractable at scale — a mechanistically understood open problem, not a wall. The design compass it hands us (delta-rule writes to reduce interference by construction, orthogonal initialization, decorrelation penalties) is the next work.