# NoCap-Test Benchmark — Complete Subpages Technical Reference

This document compiles the **complete, unabridged technical documentation** for all subpages of the **NoCap-Test LLM Benchmark**. Every section has been rewritten in **simple technical English** while preserving all underlying mathematical formulas, PyTorch code implementations, matrix operations, hardware specifications, and scientific boundary rules.

> **Project Repository**: [`/home/q/Downloads/software_i_built/nocap-benchmark`](file:///home/q/Downloads/software_i_built/nocap-benchmark)  
> **Original Source**: Carol Calin / BottleCapAI (`https://nocap.think2earn.com/`)

---

## 📑 Table of Contents

1. [Page 01: Main Evidence & Executive Summary (`index.html`)](#page-01-main-evidence--executive-summary)
2. [Page 02: Muon Optimizer Technical Deep-Dive (`muon-explained.html`)](#page-02-muon-optimizer-technical-deep-dive)
3. [Page 03: B32 Systems Path & Precision Architecture (`b32-systems-explained.html`)](#page-03-b32-systems-path--precision-architecture)
4. [Page 04: Timing-Cheap Split-QKV Attention Mechanics (`split-qkv-explained.html`)](#page-04-timing-cheap-split-qkv-attention-mechanics)
5. [Page 05: Fair-Clock Measurement Methodology (`FAIR_CLOCK.html`)](#page-05-fair-clock-measurement-methodology)
6. [Page 06: Claim Boundaries & What Is Not Claimed (`not-claimed-explained.html`)](#page-06-claim-boundaries--what-is-not-claimed)
7. [Page 07: Scoreboard, Recipe & Numbers Canon (`NUMBERS_CANON.html` & `RECIPE_CARD.html`)](#page-07-scoreboard-recipe--numbers-canon)
8. [Page 08: Failure & Retracted Claims Ledger (`FAILURE_LEDGER.html`)](#page-08-failure--retracted-claims-ledger)
9. [Page 09: Campaign Chronology & Timeline (`CAMPAIGN_TIMELINE.html`)](#page-09-campaign-chronology--timeline)

---

## Page 01: Main Evidence & Executive Summary

### 1.1 Plain Terms Core Finding
A modified training stack combining **Muon Optimizer**, **B32 Systems Precision**, and **Timing-Cheap Split-QKV Attention** trains a ~124 Million parameter GPT-2 language model on the FineWeb dataset to reach target validation quality (`val_disjoint ≤ 3.3821`) **≈8.3% faster** than the sealed stock AdamW baseline on a single NVIDIA A100 GPU.

### 1.2 Benchmark KPI Summary
- **Target Quality Threshold**: Disjoint Validation Cross-Entropy Loss $\le 3.3821$.
- **Sealed Baseline Clock (`F1_seal`)**: **3.866 Hours** (13,918,669 ms) using standard AdamW.
- **Candidate Seed 1337 (`Claim-W1`)**: **3.553 Hours** (12,789,522 ms) $\to$ **+8.1% Faster**.
- **Candidate Seed 2029 (`Claim-W2`)**: **3.567 Hours** (12,840,828 ms) $\to$ **+7.7% Faster**.
- **Candidate Seed 4242 (`Claim-W1`)**: **3.511 Hours** (12,640,471 ms) $\to$ **+9.2% Faster**.
- **Mean Candidate Clock**: **3.544 Hours** $\to$ **≈ 8.3% Time Saved**.
- **Hardware Environment Pin**: 1x NVIDIA A100-SXM4-40GB GPU · PyTorch 2.13.0+cu130.

---

## Page 02: Muon Optimizer Technical Deep-Dive

> **Source Page**: [`muon-explained.html`](file:///home/q/Downloads/software_i_built/nocap-benchmark/index.html#innovations)  
> **Original Concept**: Keller Jordan (Dec 2024) · **Kit Implementation**: `nocap-repo/kit/muon_opt.py`

### 2.1 What Is Muon?
Standard optimizers like AdamW update every neural network weight independently as scalar numbers. **Muon** (*Momentum Orthogonalized by Newton–Schulz*) treats hidden-layer 2D weight matrices as geometric grids. Each step computes an SGD-momentum update matrix $G$, then replaces $G$ with its nearest **semi-orthogonal matrix** $O$ before applying the weight update:

$$\text{Ortho}(G) = U V^T \quad \text{where } G = U S V^T \text{ (SVD)}$$

Orthogonalization drops the singular values $S \to I$, rebalancing parameter directions so that no single singular direction dominates the update.

### 2.2 Who Gets Muon vs. AdamW?
- **Muon**: Applied exclusively to 2D hidden weight matrices (internal transformer linear layers). For 4D convolution filters, spatial dimensions are flattened into 2D.
- **AdamW**: Applied to 1D parameters (biases, LayerNorm gains/biases), embeddings, and final classification heads.

### 2.3 Why Newton–Schulz Instead of SVD?
Computing exact Singular Value Decomposition (SVD) on GPU every training step is computationally prohibitive and slow. Muon uses a 5-step **Newton–Schulz (NS5)** polynomial iteration to approximate $U V^T$ using fast, native matrix multiplications (`torch.matmul`) in `bfloat16` precision.

The degree-5 polynomial mapping singular values $\sigma \in (0, 1]$ toward $1$ is defined as:

$$\phi(x) = a x + b x^3 + c x^5 \quad \text{where } a = 3.4445, \; b = -4.7750, \; c = 2.0315$$

### 2.4 PyTorch Kernel Implementation (`kit/muon_opt.py`)

```python
import torch

def zeropower_via_newtonschulz5(G: torch.Tensor, steps: int = 5, eps: float = 1e-7) -> torch.Tensor:
    """
    Computes nearest semi-orthogonal matrix Ortho(G) = U V^T using 5 Newton-Schulz steps.
    """
    assert G.ndim >= 2
    a, b, c = (3.4445, -4.7750, 2.0315)
    X = G.bfloat16()

    # Transpose tall matrices so matmul operates on smaller dimension
    if G.size(-2) > G.size(-1):
        X = X.mT

    # Step 1: Frobenius Normalize -> Places singular values in [0, 1]
    X = X / (X.norm(dim=(-2, -1), keepdim=True) + eps)

    # Step 2: 5 Iterative Matmul Polynomial Steps
    for _ in range(steps):
        A = X @ X.mT
        B = b * A + c * (A @ A)
        X = a * X + B @ X  # phi(X) = a*X + b*X^3 + c*X^5

    # Undo transpose if applied
    if G.size(-2) > G.size(-1):
        X = X.mT

    return X

def muon_update(grad: torch.Tensor, momentum: torch.Tensor, beta: float = 0.95, ns_steps: int = 5, nesterov: bool = True) -> torch.Tensor:
    """
    Computes Muon update: Nesterov momentum -> Newton-Schulz Orthogonalization -> Aspect Ratio Scaling.
    """
    momentum.lerp_(grad, 1 - beta)
    update = grad.lerp_(momentum, beta) if nesterov else momentum

    if update.ndim == 4:
        update = update.view(len(update), -1)

    update = zeropower_via_newtonschulz5(update, steps=ns_steps)
    
    # Aspect-ratio scaling adjustment: max(1, rows / cols)^0.5
    update *= max(1, update.size(-2) / update.size(-1)) ** 0.5
    return update
```

---

## Page 03: B32 Systems Path & Precision Architecture

> **Source Page**: [`b32-systems-explained.html`](file:///home/q/Downloads/software_i_built/nocap-benchmark/index.html#innovations)  
> **Component Screen**: `systems_v1`

### 3.1 Systems Optimization vs. Algorithmic Change
`B32×accum16` is a **systems throughput choice**, not a new learning algorithm. It changes how training tokens are packed into GPU memory per optimizer update.

### 3.2 Fixed Token Budget Invariant
To ensure a strictly fair clock, the total number of tokens processed per optimizer update is kept **strictly invariant** at **524,288 tokens**:

$$\text{Tokens Per Update} = \text{Microbatch Size} \times \text{Gradient Accumulation Steps} \times \text{Sequence Length}$$

- **Older B16 Baseline Path**: $16 \times 32 \times 1,024 = 524,288 \text{ tokens}$
- **Candidate B32 Path**: $32 \times 16 \times 1,024 = 524,288 \text{ tokens}$

### 3.3 Why B32 Saves Step Execution Time
Doubling microbatch size from 16 to 32 and halving gradient accumulation steps from 32 to 16 improves GPU Tensor Core occupancy on the NVIDIA A100 GPU. In matched `systems_v1` screens, this yielded a **~6% reduction in per-step core GPU execution time** with zero loss in mathematical precision.

---

## Page 04: Timing-Cheap Split-QKV Attention Mechanics

> **Source Page**: [`split-qkv-explained.html`](file:///home/q/Downloads/software_i_built/nocap-benchmark/index.html#innovations)  
> **Kit Implementation**: `nocap-repo/kit/muon_opt_split_qkv_cheap_v1.py`

### 4.1 Fused Joint QKV vs. Split-QKV
In standard transformer implementations, Query ($Q$), Key ($K$), and Value ($V$) projection weights are concatenated into a single joint matrix $W_{QKV} \in \mathbb{R}^{d \times 3d}$.

Under Muon, separating $W_Q, W_K, W_V \in \mathbb{R}^{d \times d}$ into three separate matrices allows Muon's Newton–Schulz kernel to orthogonalize each projection independently:

$$\text{Joint QKV (Stock Control)}: W_{QKV} \to \text{One joint Ortho update}$$
$$\text{Split-QKV (Candidate Stack)}: W_Q, W_K, W_V \to \text{Three independent Ortho updates}$$

### 4.2 Why Timing-Cheap Split-QKV Was Selected
During research, two split variants were tested:
1. **Dual-Polar Split-QKV**: Improved validation loss by ~+0.015, but added an expensive ~2.6% step-time overhead ($\text{ratio} \approx 1.026$). Recorded as **NO-GO** in the Failure Ledger.
2. **Timing-Cheap Split-QKV (`muon_split_qkv=1`, `match_joint=0`)**: Optimized memory layout to eliminate step-time penalties, achieving the quality boost without adding time overhead.

---

## Page 05: Fair-Clock Measurement Methodology

> **Source Page**: [`FAIR_CLOCK.html`](file:///home/q/Downloads/software_i_built/nocap-benchmark/index.html#calculator)

### 5.1 The Pure Training Timer (`train_time_ms`)
Speed is measured strictly using **training execution time** (`train_time_ms`). The benchmark harness pauses the clock during validation evaluation passes. Compile warmup time remains **inside** the measured clock.

### 5.2 Official Disjoint Validation Rule
The first-passage clock $T$ is defined strictly by the **first logged CSV row** where official disjoint validation cross-entropy loss satisfies:

$$\text{val\_disjoint} \le 3.3821$$

Sliding validation loss curves and Exponential Moving Average (EMA) curves are diagnostic tools only and **do not** define the official score.

### 5.3 Single-Seed Statistical Noise Prior (~6%)
Single-seed training runs carry an inherent statistical noise prior of **~6%**. Therefore, a speedup between 6–10% requires replication across **3 independent random seeds** (Seeds 1337, 2029, 4242) to prove validity beyond random noise.

---

## Page 06: Claim Boundaries & What Is Not Claimed

> **Source Page**: [`not-claimed-explained.html`](file:///home/q/Downloads/software_i_built/nocap-benchmark/index.html#overview)

To maintain scientific integrity, the public benchmark report enforces **three explicit claim boundary clauses**:

1. **Clause 01 — Under Sealed F1 Baseline**: All published speedup percentages are measured strictly against our locked AdamW baseline `F1_seal` (3.866 h) on the matched hardware pin.
2. **Clause 02 — Public % vs. F1_seal Only**: Percentages are not claimed against provisional historical notebook runs (~229.1 min) or unsealed baselines.
3. **Clause 03 — Matched Candidate-vs-Stock $\Delta T\% = \text{N/A}$**: The joint-QKV stock control run did not drop below the 3.3821 threshold on seeds 1337 and 2029 (ending at 3.3848 and 3.3840). Because the control did not cross the target line, an exact matched pair speedup cannot be computed, and is reported as **N/A**.

---

## Page 07: Scoreboard, Recipe & Numbers Canon

> **Source Pages**: [`NUMBERS_CANON.html`](file:///home/q/Downloads/software_i_built/nocap-benchmark/index.html#scoreboard) & [`RECIPE_CARD.html`](file:///home/q/Downloads/software_i_built/nocap-benchmark/index.html#recipe)

### 7.1 Scoreboard Master Table

| Series / Run Name | Optimizer Stack | First Cross Step | Val Loss | Train Time ($T_{\text{h}}$) | Time Saved ($\Delta T\%$) | Status |
| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
| **Candidate · Seed 1337** | Muon + B32 + Split-QKV | Step 4736 | 3.3741 | **3.553 h** | **+8.1%** | ✅ Passed Target |
| **Candidate · Seed 2029** | Muon + B32 + Split-QKV | Step 4736 | 3.3726 | **3.567 h** | **+7.7%** | ✅ Passed Target |
| **Candidate · Seed 4242** | Muon + B32 + Split-QKV | Step 4736 | 3.3733 | **3.511 h** | **+9.2%** | ✅ Passed Target |
| **Baseline (`F1_seal`)** | Standard AdamW | Step 4736 | 3.3815 | **3.866 h** | 0.0% (Ref) | 🔒 Locked Baseline |
| **Stock Control · Seed 1337** | Muon + Joint QKV | No Cross | 3.3848 | N/A | N/A | ❌ Missed Target |
| **Stock Control · Seed 2029** | Muon + Joint QKV | No Cross | 3.3840 | N/A | N/A | ❌ Missed Target |

### 7.2 Recipe Hyperparameter Specifications
- **Model**: GPT-2 (12 layers, 768 hidden dimension, 12 attention heads, ~124M parameters).
- **Dataset**: FineWeb raw tokens (1,024 sequence length).
- **Batching**: Microbatch size = 32, Gradient Accumulation = 16, Tokens per update = 524,288.
- **Optimizer Config**:
  - Muon (2D weights): Learning rate = 0.02, Momentum = 0.95, Newton–Schulz steps = 5.
  - AdamW (1D / Embeddings / Head): Learning rate = 0.0018, Betas = (0.9, 0.95), Weight Decay = 0.1.
- **Schedule**: Warmup-Stable-Warmdown (WSD) schedule: 476 warmup steps, 3,100 stable steps, 1,192 warmdown steps (Total horizon = 4,768 steps).

---

## Page 08: Failure & Retracted Claims Ledger

> **Source Page**: [`FAILURE_LEDGER.html`](file:///home/q/Downloads/software_i_built/nocap-benchmark/index.html#failures)

| Failed Experiment | Category | Impact / Result | Reason for Rejection |
| :--- | :--- | :--- | :--- |
| **Dual-Polar Split-QKV** | Architecture | +2.6% Step-Time Tax | Recomputing polar maps added execution overhead that canceled out learning gains. |
| **Naive Canon Architecture** | Architecture | +30% Step-Time Tax | Canonical block transformations added excessive per-step compute penalty. |
| **Short Warmdown (256 steps)** | Schedule | Final Loss ~3.405 | Terminated early without reaching the 3.3821 validation threshold. |
| **Retracted F2b Speed Claim** | Methodology | Retracted (-2.9%) | Retracted due to unsealed baseline stack drift and single-seed noise prior. |
| **Tier A Screen Candidates** | Optimizer | 5 / 5 No-Go | Five candidate variants plateaued early during screening proxy runs. |

---

## Page 09: Campaign Chronology & Timeline

> **Source Page**: [`CAMPAIGN_TIMELINE.html`](file:///home/q/Downloads/software_i_built/nocap-benchmark/index.html#overview)

1. **Wave 1 (Colab T4 Exploration)**: Explored initial sliding evaluations and warmdown behavior. Discovered naive canonical block tax (~30%).
2. **F2 & F2b Warmdown Tuning**: F2 (256 warmdown) failed at 3.405. F2b (1024 warmdown) reached 3.377, but speed framing was retracted due to unsealed baseline drift.
3. **Denominator Sealing (`F1_seal`)**: Established the locked baseline denominator at 3.866 hours on 1x A100 GPU (`F1_seal`).
4. **Systems & Component Validation**: Validated `B32×accum16` (~6% step gain) and timing-cheap Split-QKV as standalone components.
5. **Sealed 3-Seed Campaign**: Ran sealed candidate stack on seeds 1337, 2029, and 4242, achieving 3/3 target passes with mean 3.544 h (≈8.3% time saved).