NoCap-Test evidence · think2earn attention primer · before split
Source code ↗

Learn attention letters · before packing

QKV

Query · Key · Value — what the three attention projections are · before joint vs split

Transformer attention needs three projections of each token: Query (Q), Key (K), and Value (V). QKV names the three projections: Query, Key, Value. This page teaches what they do. It does not invent Muon, Ortho, or split layouts — and it invents no sealed ΔT%.

Our commentary Diagrams decode the letters for reading the claim stack — not a speed claim.

Standard architecture Q/K/V come from the Transformer attention literature — not a BottleCap novelty.

Next on this site How we pack those projections (fused joint vs separate) under Muon: timing-cheap split-QKV.

Query · Key · Value attention projections not Ortho/NS no sealed %

Timing-cheap split-QKV Split ≠ Ortho GLOSSARY · Joint / split QKV Muon explained intro

Definition

In self-attention, every position builds a query (what am I looking for?), compares it to other positions’ keys (what do I advertise?), and mixes their values (what content do I pass along?). Those three vectors are linear projections of the same hidden state.

Our commentary Conceptual

x → WQ → Q  (ask)
x → WK → K  (match)
x → WV → V  (content)
attention(Q, K, V) → weighted mix of V by Q·K scores

“QKV” on this site means those three projections as a group — whether stored as one fused matrix or three separate ones.

Problem 01 · roles Q · K · V

Three roles in one sentence each

Math: dot-products and a weighted sum. Names label the three jobs.

Letter Q

Query

What this token is searching for in other positions.

Letter K

Key

What each position offers as a match target for queries.

Letter V

Value

The payload mixed in when a query matches a key.

Figure · Teaching labels only. Not a FLOP or timing claim.

Visual · One token → three projections Our commentary
x Q · Query K · Key V · Value Attention mixes V using scores from Q · Kᵀ

Figure · Schematic self-attention path. Official speed claims stay on the report vs F1_seal.

Problem 02 · attention step score → mix

How attention uses them

For each query, compare it to keys (usually a scaled dot product), turn those scores into weights with softmax, then take a weighted sum of values. Step through the toy example below — pattern inspired by TensorTonic · softmax (slider readouts) and the attention stack in positional-encoding.

Teach beats · Q → K → softmax → V Step / Auto · TT softmax-style
The cat sat TOY SEQUENCE · query from “cat” Q ask vector Q · Kᵀ scores 0.8 2.2 0.1 softmax weights The cat sat Σ wᵢ · Vᵢ —
beat Q
raw score —
weight —
mix —
1.0 Sharpen ↑ · flatten ↓ (TT softmax knob)

Project hidden state → Q. One token (“cat”) asks what to attend to.

Manual · step 1/4

Figure · Toy 3-token self-attention. Numbers are pedagogical — not FLOPs, not campaign timing. CUDA softmax study plan exists at tensortonic.com/study-plans/cuda-basics/cuda/softmax.

Problem 03 · packing joint vs separate

Joint vs separate packing

The roles stay Q, K, V either way. What changes on the claim stack is how the weight matrices are stored and updated:

Joint QKV

One fused WQKV

Slice into Q, K, V after the matmul. Claim-stack stock control (muon_split_qkv=0).

Separate / split

Three matrices WQ, WK, WV

Candidate path under Muon. Campaign bank: timing-cheap split.

Keller Jordan notes separate Q/K/V often helps under Muon — that is a layout note, not inventing Ortho/NS and not our sealed %. Dedicated pages: split-qkv-explained.html · split-vs-ortho-explained.html · muon #split-vs-ortho.

Problem 04 · claim boundary primer only

What this page does not claim

  • Not inventing attention, QKV, or Muon.
  • Not Ortho / Newton–Schulz — that is the Muon kernel (muon-explained).
  • Not the sealed wall-clock result — that is Muon + B32 + timing-cheap split vs F1_seal on the report.
  • Not a matched % vs joint Muon stock (that endpoint is N/A — not claimed).
Claim boundary. Read this page to decode the letters. Read timing-cheap split-QKV for the banked campaign treatment, and F1_seal for the public ΔT% denominator (3.866 h).