≈8.3% less train time to val ≤ 3.3821 — 3/3 seeds
NoCap-Test trains GPT-2 ≈124M on FineWeb on one GPU until validation loss reaches ≤ 3.3821, faster than a fair stock baseline. The score is first-passage train time: clock time until the run first crosses that threshold.
The challenge is about algorithmic ideas that might generalize, not hyperparameter tuning or copying Modded-NanoGPT wholesale.
We adopt
Keller Jordan’s Muon
(explainer; published Ortho/NS, not our invention).
Plain Muon alone failed or lost under our seal
(failure ledger).
What we sealed is Muon on a
B32 systems path
plus
timing-cheap split-QKV
after a costlier split was killed
(what QKV is;
kill → bank → 3/3).
Joint-QKV Muon on the same stack missed the quality bar where measured.
Speed is vs our sealed AdamW F1_seal (3.866 h,
explained here):
3/3 seeds, mean ≈8.3%.
Not an official NoCap win, and not a matched % vs joint Muon stock.
Sealed trainer + SHA freeze:
TRAINER_DELTA ·
kit/ (this site) ·
view source ↗.
What the number means
Three seeds first-cross val_disjoint ≤ 3.3821 in
3.51–3.57 h (mean 3.544 h, mean ΔT% ≈ +8.3%
vs F1_seal 3.866 h).
Terminal candidate CE ≈ 3.37.
Per-seed T and ΔT% are in the scoreboard and
seeds explainer.
How it is measured
Pin: 1× A100-SXM4-40GB · torch 2.13.0+cu130 · GPT-2 ≈124M · FineWeb · official disjoint val. Workspace codes: Clock-W, Claim-W1, Claim-W2.
Read more
How to read the stack. Process first, then three banked pieces — equal weight under the sealed recipe.
- Kill → bank → seal — close dead ends, bank components, seal vs F1_seal.
- Muon — Keller Jordan optimizer (adopted, not invented here).
- B32 systems path — banked step-time path (B32×accum16).
- QKV primer — what Query / Key / Value are.
- Timing-cheap split-QKV — banked layout after the dual-polar tax kill.
- F1_seal — AdamW public denom 3.866 h.
Sealed speed is vs F1_seal only. Matched cand-vs-stock ΔT% = N/A. Not an official win.
- Trainer delta · public kit Sealed cheap-split trainer + muon opt · PREREG SHA freeze · flags
- QKV · Query / Key / Value Stranger primer before joint vs timing-cheap split
- Not claimed Why the hedge: official win / F1_seal only / matched ΔT% N/A
- Seeds 3/3 Multi-seed seal bar · each seed’s T / ΔT% vs F1_seal
- Workspace codes Clock-W · Claim-W1 · Claim-W2 — ops roles
- AdamW F1_seal Public denom · first-passage 3.866 h · not Muon
- Official disjoint val val_disjoint · bar ≤ 3.3821 · first-passage · not train CE / EMA
- Kill → bank → 3/3 Process: close dead ends, bank components, only then seal — report insight
- B32 systems path B32×accum16 banked systems path under the claim recipe
- Failure ledger claim_muon_par / claim_muon_wsd, dual-polar tax, Tier A NO-GOs
- Timing-cheap split-QKV Second mechanism vs dual-polar tax; split ≠ Ortho
- Split ≠ Ortho Ortho/NS×5 vs timing-cheap split; same NS; seal needs both
- Muon primer · #split-vs-ortho Keller NS×5 attributed; campaign note on the Muon page
- Evidence — sealed scoreboard Fastest numeric check
- Report — full campaign writeup Compute tiers, verification, non-claims
- Numbers — sealed T / ΔT% resolution order
- Docs — challenge terms, glossary, how to read evidence
- TensorTonic GPT-2 track External — 14-module decoder stack viz (Plus); pattern reference for d12 architecture primer
- TensorTonic training-loop scrubber External — epoch slider + loss-curve readouts (free); metaphor for WSD / update-index pedagogy