docs(quant): add QUANTIZATION §4b tier trade-space

The fast/balanced/max composes aren't rungs on a single fidelity axis —
they're corners of a 3-axis trade-space (weight-fidelity / decode+context
/ prefill-TTFT). §4b names the axes, the corner map, the legitimacy rule
(a differentiator no instrument can measure isn't a tier), scheme-vs-
algorithm, and why "balanced" stays provisional. It fills the prefill
corner with the measured INT8 W8A8 vs FP8 result (v0.24.0): quality tied
107/150, W8A8 wins prefill/TTFT (native CUTLASS INT8), FP8 wins decode —
a prefill-vs-decode tradeoff, not a replacement. W8A8 qualifies as an
analyzed tier; shipping it as a slug is deferred (not committed).

- §4a "Tiering principle" now points into §4b (topology projection).
- DUAL_CARD balanced/max footnote cross-links §4b for the rationale.
- QUANTIZATION.md added to the docs/README.md index.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm

docs/QUANTIZATION.md

@@ -79,7 +79,76 @@ Method labels don't rank fidelity — **measure it.** The cleanest signal is **K
 
 **Where QAT fits:** quantization-*aware* training fine-tunes *with* quantization simulated, so weights adapt → near-BF16 at low bits. Its advantage is **concentrated at ≤4-bit** — at 8-bit, PTQ is already near-lossless, so QAT adds little. So: want **4-bit + max fidelity** → a **QAT-int4** (e.g. Google's Gemma QAT) beats AutoRound/AWQ int4; want **max fidelity, size-flexible** → just use **8-bit PTQ (FP8/INT8)**, no QAT needed. PTQ methods that *approach* QAT without retraining: AutoRound (learns rounding), AWQ (activation-aware), GPTQ (2nd-order), QuIP#/AQLM (codebook, 2-bit), SpinQuant/QuaRot (rotation); the GGUF analogue is **imatrix** (§3).
 
-**Tiering principle:** **dual-card = fidelity tier, single-card = fit/speed tier.** A dual's distinct value is *capacity* — spend it on a higher-fidelity quant (FP8/INT8/AWQ) than the single's small INT4, not the same one.
+**Tiering principle:** **dual-card = fidelity tier, single-card = fit/speed tier.** A dual's distinct value is *capacity* — spend it on a higher-fidelity quant (FP8/INT8/AWQ) than the single's small INT4, not the same one. This is the **topology projection** of the tier trade-space in **§4b** — a single card is forced into the fit/speed corner; the second card is what *buys* the higher-fidelity (and prefill) corners as real options.
+
+---
+
+## 4b. The tier trade-space — what fast / balanced / max actually optimize
+
+§4a ranks quants by **fidelity** (KLD vs bf16). That's the right instrument for *one* question — "how close to full precision?" — but it's a single axis, and the shipped **fast / balanced / max** composes are **not** three rungs on it. They're **corners of a trade-space**. On this stack's hardware (Ampere SM 8.6, PCIe, no native FP8, bandwidth-bound decode) there are **three independent axes**:
+
+| Axis | Bound by | Direction | Measured by | The tier it defines |
+|---|---|---|---|---|
+| **Weight fidelity** | bits/weight | more bits → closer to bf16 | **KLD** (§4a) | **max** |
+| **Decode speed + context** | memory **bandwidth** | fewer weight bytes → faster decode **and** bigger KV pool | **decode TPS**, KV-pool tokens | **fast** |
+| **Prefill / TTFT** | tensor-core **compute** | native INT8 skips the dequant step | **TTFT** | **prefill** |
+
+Two consequences fall straight out of this:
+
+- **Decode speed and context are the *same* axis here.** Decode is bandwidth-bound, so the quant with the fewest weight bytes (INT4) is *both* the fastest to decode *and* the one that leaves the most VRAM for KV → the biggest context pool. "fast" being the headroom leader too isn't a coincidence — it's the same byte count paying off twice.
+- **Prefill is a *different* axis from decode.** Prefill is compute-bound, and Ampere has native INT8 tensor cores but no native FP8. So an **INT8 W8A8** quant (8-bit *weights and activations*) runs prefill on the IMMA cores with no dequant → fast TTFT, while FP8 and INT4 must dequant to 16-bit first. A quant can be a mediocre decoder and the best prefiller; decode TPS tells you nothing about it.
+
+### Scheme vs algorithm — don't confuse the two
+
+A recurring trap: **`W4A16` / `W8A8` / `W8A16` / `FP8` are *schemes*** — how many bits, and whether the *activations* are quantized too. The scheme sets **which axis** a quant lives on. **`AutoRound` / `AWQ` / `GPTQ` are *algorithms*** — how the weights are rounded — and they set **accuracy *within*** a scheme. So "AutoRound vs AWQ vs a GPTQ-int4" is one *algorithm* comparison at a fixed scheme (all W4A16); "W4A16 vs W8A8" is a *scheme* comparison across axes. **The tier is set by the scheme; the algorithm only decides how good that tier's fidelity is.** (A practical corollary: a "better" W4 — say a code-calibrated AutoRound — is a *fast-tier* swap, not a move up a tier. It can't change axis by being marginally larger.)
+
+### The corner map
+
+Each tier maximizes one axis and accepts the cost on the others:
+
+| Tier | Optimizes | Measured differentiator | Sacrifices | Typical scheme |
+|---|---|---|---|---|
+| **fast** ⭐ | decode + context | TPS, KV-pool size | weight fidelity (lowest KLD rank) | W4A16 (AutoRound) |
+| **max** | weight fidelity | KLD vs bf16 | decode, pool, prefill | FP8 / INT8 (W8A16) |
+| **prefill** | TTFT | TTFT bench | some activation fidelity | INT8 **W8A8** |
+| **balanced** | *(see note)* | *(under review)* | *(see note)* | — |
+
+> The §4a *single-card vs dual-card* split is the **topology projection** of this map: a single 24 GB card is forced into the **fast** corner (VRAM picks the smallest quant), and the **second card is what *buys*** the **max** and **prefill** corners as real options. "Add a card" = "unlock the other corners." (See [`DUAL_CARD.md`](DUAL_CARD.md) for the per-model dual-card pick-table this maps onto.)
+
+### The rule that makes a tier legitimate
+
+**A tier earns its place only if it owns a corner no other tier dominates — with a *measurable* differentiator.** The instrument has to be able to actually *see* the thing the tier claims:
+
+- weight fidelity → **KLD** — but note KLD is **weights-only**; it cannot see activation-quant or KV-quant error,
+- decode / context → **TPS + KV-pool size**,
+- prefill → **TTFT**,
+- anything fidelity-related KLD can't see (activation quant, KV quant, calibration drift) → the **8-pack** quality suite ([QUALITY_TEST.md](QUALITY_TEST.md)).
+
+**A claimed differentiator no instrument can measure is not a tier — it's a guess.** This is the test that decides whether a candidate quant ships as its own tier or folds into an existing one.
+
+> **Worked example — why "balanced" is provisional.** "balanced" has historically meant *an INT4 weight + a higher-fidelity KV* (int8-per-token-head instead of fp8), betting the KV fidelity is worth a tier. On the measurements we have, it's **dominated by fast**: same-or-slower decode, a *smaller* KV pool (heavier weights leave less room), and a *tie* on the 8-pack. Its only claimed edge is a KV cache that is the **same byte size** as fast's and that the short-context 8-pack can't see. Until an instrument that *can* see it (long-context recall / NIAH) separates them, "balanced" isn't a real corner — it's fast with a more expensive KV. The honest options are three: prove it with the right instrument, redefine it onto a real axis, or retire it.
+
+### The prefill corner, filled — **INT8 W8A8** (measured 2026-06-30, vLLM v0.24.0)
+
+The third axis — **prefill / TTFT** — now has an occupant. An **INT8 W8A8** quant (INT8 weights *and* dynamic INT8 activations, vision kept fp16) was benched head-to-head against **FP8** at the 8-bit slot, *all else equal* (same int8-per-token-head KV → same 295K pool, same MTP n=3, same TP=2, same 262K ctx, same harness). vLLM picks the native **CUTLASS INT8** GEMM for it on Ampere sm_86; FP8 falls back to **Marlin weight-only dequant** (it logs *"may degrade performance for compute-heavy workloads"*).
+
+| | **W8A8** (CUTLASS INT8) | **FP8** (Marlin dequant) |
+|---|---|---|
+| 8-pack quality (`--full`, 150) | **107/150** | **107/150** |
+| Prefill t/s @10K → 90K | **2062 → 1021** | 1364 → 875 (**W8A8 +17–51%**) |
+| Short-prompt TTFT | **122 ms** | 158 ms |
+| Decode TPS (narr/code) | 77 / 96 | **83 / 108** |
+| KV pool | 295K / 1.13× | 295K / 1.13× |
+
+**Verdict: W8A8 *qualifies* as a legitimate new tier (by the legitimacy rule above), not a replacement** — but *qualifying* (this analysis) is separate from a *shipping* decision (a catalog call, deferred pending cross-rig numbers + the thinking-on arm; not committed as a shipped slug). It's a clean **prefill-vs-decode tradeoff at the 8-bit slot** — *equal quality (107=107, pack-for-pack ±1), equal pool*, but W8A8 wins **prefill/TTFT** (native INT8 IMMA, the compute-bound path FP8's Marlin dequant is explicitly worse at) while FP8 wins **decode** (its Marlin kernel is better-tuned for single-token). So:
+
+- **W8A8 = the prefill / agentic 8-bit corner** (long-prompt · RAG · tool-chains — TTFT-bound).
+- **FP8 = the decode-throughput 8-bit corner.**
+- Neither dominates; they're different corners. This is the legitimacy rule in action — W8A8 earns its corner with a **measurable** differentiator (prefill t/s + TTFT), exactly what "balanced" lacked.
+
+> **The activation-quant fear was unfounded.** W8A8 quantizes activations — §4a's weights-only KLD can't see that cost, which is *why* this needed the 8-pack. The 8-pack saw it: **zero cost** (107 = 107). 8-bit weight fidelity + dynamic per-token activation scales held tool-call / structured / numeric / agentic quality identical to near-lossless FP8.
+
+**Open refinements (don't block the analysis):** this run was **thinking-off** (`--full` default) — a thinking-**on** arm is the remaining two-arm check; and a **bf16-KV** W8A8 likely pushes prefill/TTFT further still, trading max-ctx. Both are follow-ups, not gates.
 
 ---