NoCap-Test evidence · think2earn campaign mechanism · no sealed ΔT%
Source code ↗

Learn the mechanism · code-adjacent

Timing-cheap split-QKV

Separate Q/K/V vs fused joint · campaign bank · not Ortho/NS

New to the letters? Start with what Query, Key, and Value are — then return here for packing and campaign cost.

Under Muon, attention can use a fused joint QKV projection or separate Q, K, and V matrices. Keller Jordan notes that separate often helps under Muon — we do not claim inventing that layout preference.

This page is about the campaign’s timing-cheap split path: a banked implementation with low step-time tax, flags muon_split_qkv=1 · match_joint=0. Same Newton–Schulz / Ortho kernel as joint Muon. Split ≠ inventing Muon.

From Keller Jordan’s blog (reference) Separate Q/K/V tends to beat fused QKV under Muon — architecture/layout, not the Ortho kernel.

Our implementation Timing-cheap wrappers in kit/muon_opt_split_qkv_*.py; still call zeropower_via_newtonschulz5.

Our commentary Diagrams and beats on this page are campaign educational material. No sealed ΔT% invented here — sealed wall-clock is full stack vs F1_seal on the report.

muon_split_qkv=1 match_joint=0 NS ×5 shared dual-polar ~1.026× tax

QKV primer Split ≠ Ortho Muon · #split-vs-ortho F1_seal denom GLOSSARY FAILURE_LEDGER RECIPE_CARD TRAINER_DELTA

Definition

Joint QKV = one fused projection for Q, K, and V (claim-stack stock control: muon_split_qkv=0). Split-QKV = separate matrices. Timing-cheap = this campaign’s split implementation tuned so step-time tax stays low enough to bank — unlike dual-polar, which got CE but lost on wall-clock.

If “Q / K / V” is unfamiliar, read the stranger primer first: qkv-explained.html (Query asks · Key matches · Value mixes). This page assumes those roles and focuses on packing cost under the fair clock.

Our implementation Candidate arm (claim-stack)

muon_split_qkv = 1
match_joint = 0
Ortho kernel = same zeropower_via_newtonschulz5 (NS ×5)

Problem 01 · layout joint vs separate

What changes when you split QKV?

The optimizer is still Muon on 2D weights. The change is how attention projects queries, keys, and values: one fused tensor vs three. Under Muon, that layout choice matters for training dynamics — Keller notes separate often helps.

Interactive · joint ↔ separate layout packing only · Muon route unchanged
PROJECTION PACKING CAMPAIGN NOTE W_QKV · fused One tensor → slice to Q, K, V Q K V Fused baseline Stock control · step-tax baseline matched ΔT% N/A Separate · timing-cheap Banked candidate packing low step-time tax OPTIMIZER ROUTING · unchanged Muon on 2D weights · NS ×5 Same zeropower_via_newtonschulz5 · embed/head stay AdamW Split-QKV = campaign packing only · not Ortho/NS
Layout joint fused
muon_split_qkv 0
match_joint 1
Projection W_QKV
Muon route Muon · NS ×5

Now: stock control packs Q, K, and V into one fused matrix. Optimizer routing is unchanged — still Muon on 2D with the same NS kernel.

Toggle or press J / S · projection packing only · no sealed ΔT%

Layout preference ≠ Ortho kernel. Toggle switches packing labels only — not sealed ΔT%. Attribution for Muon stays with Keller Jordan.

Problem 02 · claim boundary split ≠ Ortho

Not Muon Ortho — and not inventing Muon

Newton–Schulz Ortho lives in zeropower_via_newtonschulz5 (blog newtonschulz5). Split-QKV wrappers call that same function. Separating Q/K/V does not invent Muon and does not redefine Ortho.

Split-QKV ≠ Ortho/NS. Dedicated compare: split-vs-ortho-explained.html. See also muon-explained.html#split-vs-ortho and #claim-boundary. Sealed wall-clock claims are for the full candidate stack (Muon B32 + timing-cheap split-QKV) vs F1_seal — not a Muon-alone % on this page.
Muon Ortho/NS  →  kit/muon_opt.py · zeropower_via_newtonschulz5
Split wrappers  →  kit/muon_opt_split_qkv_*.py · call same NS
Timing-cheap  →  campaign banked mechanism (this page)
Problem 03 · cost / quality three beats

Joint · timing-cheap · dual-polar

Dual-polar split showed a CE edge (~+0.015) but a step-time ratio ≈ 1.026 — the quality gain was eaten by tax → NO-GO (FAILURE_LEDGER). Timing-cheap was the banked alternative: low step-time tax, used in sealed candidates. Joint control quality-missed where measured; matched ΔT% is N/A.

Cost rail · fused → bank → dual-polar kill Auto / Step / Reset
BEAT A · CONTROL
Joint QKV

Fused projection. Stock arm on the shared Muon B32 recipe.

muon_split_qkv=0 · match_joint=1

STEP TAX · baseline
QUALITY @ BAR · miss where measured

Seeds 1337 / 2029: no first-cross → matched ΔT% N/A

BEAT B · BANKED
Timing-cheap split

Separate Q/K/V with low step-time tax. Banked @800/@2400; candidate arm.

muon_split_qkv=1 · match_joint=0

STEP TAX · low (banked)
QUALITY PATH · sealed candidate stack

Sealed % lives on report vs F1_seal — not invented here

BEAT C · KILLED NO-GO · 1.026×
Dual-polar split

Earlier split variant. CE moved; wall did not.

~+0.015 CE · time ratio ≈ 1.026

STEP TAX · too high
CE EDGE · eaten by tax → NO-GO

Not in the sealed candidate recipe · FAILURE_LEDGER

Variant Joint QKV
Step tax baseline
Campaign fate stock control
Flags split=0 · match=1
Muon route Muon · NS ×5

Reading beat A: fused projection is the stock control arm. Step tax is the baseline; quality missed where measured — not a sealed speed claim on this page.

Beat A · Joint QKV — stock control. Step tax baseline; quality missed where measured.

Meters are schematic (relative teaching), not sealed percentages. Auto or Step: fused baseline → timing-cheap bank → dual-polar 1.026× NO-GO.

Variant Cost story Quality story Campaign fate
Joint QKV Baseline step path Missed first-cross where measured Stock control · matched ΔT% N/A
Timing-cheap split Low step-time tax (banked) Candidate arm on claim-stack In sealed full stack vs F1_seal
Dual-polar split Time ratio ≈ 1.026 ~+0.015 CE NO-GO — CE eaten by tax
Problem 04 · recipe language flags → claim

Flags, bank, sealed claim shape

Component bank (systems + timing-cheap @800/@2400) is real evidence — bank ≠ claim until full-horizon seals. Public sealed language compares the full candidate stack to AdamW F1_seal, not joint stock and not Muon alone.

Stock controlCandidate
QKV joint timing-cheap split
muon_split_qkv 0 1
match_joint 1 0
NS / Ortho Same kernel (NS ×5)

Claim boundary. Keller’s blog preference for separate Q/K/V under Muon is a layout note — not our timing-cheap implementation and not a sealed wall-clock %. This page’s banked mechanism is campaign timing-cheap split-QKV (muon_split_qkv=1); Ortho/NS stays Keller’s kernel. Ecosystem lineage: Modded-NanoGPT — linked, not claimed as our invention. No matched candidate-vs-stock first-cross ΔT% (joint quality-missed; seed 4242 stock not run). Do not read dual-polar’s ~1.026 ratio, bank screens, or this page’s diagrams as the sealed under-baseline %. Sealed ΔT% is full stack (Muon B32 + timing-cheap split) vs F1_seal on the report / NUMBERS_CANON / RECIPE_CARD. Hedge defense: not-claimed-explained.html.