Populate envelopes.yml with the first memory-envelope rows, computed (not
guessed) from the kv-calc pool ceiling at model-max context:
vllm/dual @ rtx-5090 (2x32 GB): 2 -> 4 concurrent full-262K sessions
vllm/minimal @ rtx-5090 (1x32 GB): 1 -> 9 concurrent full-32K sessions
Both are the no-preemption ceiling: N *full-context* sequences whose KV
blocks fit the pool. Concurrency is a capacity question kv-calc answers
deterministically (arithmetic, arch-independent), and max_num_seqs is a cap
with graceful preemption (never OOM), so an over-optimistic ceiling costs an
occasional preempt, not a crash. kv-calc is sm_120-calibrated (disc #571
paulp83's real-5090 verify-stress PASS), so a 32 GB projection is trusted
arithmetic. A concurrency-soak upgrades a `computed` row to `validated`.
Guard: test-envelopes now accepts a `computed` (kv-calc basis) block as
provenance alongside `validated` (soak) — a computed row must name its basis
(invocation + PASS/cap boundary). Added a fixture proving injection is
provenance-agnostic (the launcher reads max_num_seqs; only the guard cares
about provenance).
Multi-GPU: the seam already fires on homogeneous >2-card rigs (collapses N
identical cards to one class; verified vllm/dual injects on a 4x5090 spec) and
kv-calc computes TP=4 pools; the experimental TP=4 slugs are documented as
computable-but-deferred rather than seeded. Heterogeneous rigs no-op by design.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Found testing vibethinker-3b (a reasoning model): with a small max_tokens
it spends the budget mid-<think> and returns HTTP 200 with 256 tokens but
EMPTY content (never emits the final answer). The old content-based check
wrongly flagged those streams as silent-empty failures. For a KV-pool
stress test the stream DID run (generated tokens, held KV) -> ok =
completion_tokens > 0; a true silent-empty is HTTP 200 with ZERO tokens
(what soak-test means). After the fix vibethinker sustains N=8 clean on a
3090 (8x the compose default, 0 post-warm growth).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
The #246 Phase 2 first pass — spend a bigger card's KV pool on
concurrency, weights-invariant. Ships the machinery + measurement; the
32 GB envelope rows land from volunteer probe runs (baselines-style
plumbing-first).
Why concurrency-only (agreed 2026-07-05, design doc): most headline
configs are already at 262K model-max on 24 GB (GGUF single, dual vLLM),
so the extra 32 GB pool serves MORE STREAMS, not more context. The
context lever (bigger max_model_len) only helps single-card vLLM and is
Cliff-2b-capped anyway — deferred.
- Composes: `--max-num-seqs` parametrized to `${MAX_NUM_SEQS:-N}` on the
two pilot slugs (dual=2, minimal=1). Weights-invariant, default kept.
- envelopes.yml (empty): measured per-(slug, card-class) `max_num_seqs`,
born-from-measurement (a `validated` block is required — no guesses).
- launch_compat.py `_envelope_env`: injects MAX_NUM_SEQS from a row when
it EXCEEDS the compose default; 24 GB / no-row / heterogeneous /
user-env-set -> no-op. Both launchers whitelist MAX_NUM_SEQS.
- test-envelopes: schema + injection contract (inject / no-row /
heterogeneous / user-env-wins / no-gain).
- concurrency-probe.sh: the measurement — N concurrent long streams x R
rounds, separating expected pool-fill from a real leak (post-warm
growth). Live-validated N=2 on the dual-3090 (0 MB post-warm growth,
all rounds clean — the shipped default is sound).
Reuses the Phase 1 injection seam; independent of the KV verdict.
Full scripts gate 67/67.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Adds a subsection to DTYPE_MATRIX's two-axes section clarifying that the
FMHA-kernel arch gating is a vLLM-family (vLLM + SGLang) phenomenon.
The llama.cpp family (mainline / ik-llama / beellama) is always-dequant:
KV quant is storage-only on every arch (dequant inside the FA kernel, no
FP8/FP4 tensor cores even on Hopper), so no consumer-vs-datacenter split
— q4_0 KV behaves the same on a 3090/4090/5090/Spark, which is why our
single-card GGUF configs hit 262K anywhere. GGUF weight quant is
dequant-to-FP16 too, so the native-FP8/NVFP4-weights win is vLLM-only.
Division of labor: native low-precision COMPUTE wins are vLLM-only (and
mostly datacenter for KV); the CAPACITY win (KV compression for long
ctx) is delivered arch-agnostically by the GGUF family — the right tool
for a consumer card that wants big context.
Verified: llama.cpp #22411 / #24109, ik_llama #1142.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Consolidates the validated findings from the #246 A/B + #571 volunteer
data + vLLM/TRT-LLM source review. New DTYPE_MATRIX section "Having the
Tensor Cores ≠ using them" makes the two load-bearing distinctions
explicit:
- Axis 1 — weights vs KV are gated SEPARATELY. FP8/FP4 *weights* work
wherever the TCs exist (sm_89+ FP8, all Blackwell FP4). KV is
different because it rides the attention (FMHA) kernel.
- Axis 2 — KV compute vs KV storage. Native FP8/FP4 *attention* compute
needs FA3 (Hopper sm_90 only) or trtllm-gen FMHA (datacenter Blackwell
sm_100/103 only). On EVERY consumer card — Ada 4090 (sm_89), consumer
Blackwell 5090/PRO-6000 (sm_120), DGX Spark GB10 (sm_121) — FP8 KV is
storage-only and nvfp4 KV doesn't work.
Consequences documented (both empirically confirmed): e4m3 ≡ e5m2 in
speed on consumer cards (86.73 vs 86.66 on a 5090, disc #571 — a
precision choice, not a perf lever); nvfp4 KV crashes on consumer
Blackwell (#43562). DGX Spark = same sm_12x family as the 5090.
Corrected now-wrong claims: DTYPE_MATRIX "Ada/Blackwell get a real win
on FP8 KV" (false — Hopper/DC-only) + the per-arch Ada/Blackwell-consumer
rows; QUANTIZATION + HARDWARE "native FP8 compute on sm_89+" → storage-
only + precision framing. Compute-win KV on consumer cards = INT8-PTH.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Two 5090 owners hit `--kv-cache-dtype nvfp4 requires sm100f` crashing
mid-boot on the #246 A/B (disc #571). Root-caused from vLLM #43562 /
TRT-LLM #10241: nvfp4 KV forces the trtllm-gen FP4 FMHA, built ONLY for
datacenter Blackwell sm_100/sm_103. Consumer Blackwell (sm_120/121 —
RTX 5090 / PRO 6000 Blackwell) is a HIGHER cc number but a different
family with no FMHA build. NVFP4 *weights* work there; only the KV path
doesn't. Our #246 gate had used a numeric ">=10.0" floor that wrongly
passed sm_120 — a floor can't express "sm_100/103 but not the
numerically-higher sm_120".
- gates.py: new `_ARCH_KERNEL_SM_FAMILY` allowlist ({nvfp4: sm_100/103});
dropped nvfp4 from the numeric `_ARCH_KERNEL_SM` floor; family-membership
reject with the FMHA reason + fp8_e4m3 fallback.
- arch-ab.sh: nvfp4 arm now refuses on consumer Blackwell (not just
<sm_10), naming the FMHA gap + the fp8_e4m3 path; dropped nvfp4 from the
recommended arms in help.
- hardware profiles: removed nvfp4 from rtx-5090 / rtx-6000-pro-blackwell
KV lists (both sm_120) + added a why-not note.
- docs (DTYPE_MATRIX / HARDWARE / KV_MATH / QUANTIZATION): corrected the
"Blackwell sm >= 10.0" framing to "datacenter sm_100/103 only".
- UPSTREAM.md: #43562 / TRT-LLM #10241 row + re-test trigger.
- test-arch-ab: nvfp4 refuses on sm_86 AND sm_120, allowed on sm_100;
the dual-5090 all-arms test drops nvfp4.
Full scripts gate 66/66.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
report.sh already printed both raw inputs (host capability sections +
the [nvlink] engagement trail added after #446/#488); what was missing
was the CROSS-REFERENCE. New scripts/lib/p2p-state.sh implements the
verdict matrix once, consumed by report.sh (full verdict line) and
preflight.sh (capability one-liner):
- <2 GPUs / no capability -> SILENT (stock-PCIe owners never nagged;
that's ~95% of dual-3090 rigs and us)
- capability + engaged -> one OK line
- NVLink bridge + P2P off -> WARN (bridge idle, ~15% decode on the
table per the #77 controlled A/B; names
the fix: launcher boot / force_on)
- P2P-capable driver + off -> INFO (launcher auto-engages since #291;
residual case = direct docker compose)
The lib is the read-only AUDITOR; scripts/detect_nvlink.sh stays the
boot-time DECIDER. Their capability probes mirror each other by design,
and test-p2p-state runs BOTH against shared faked-nvidia-smi fixtures
and asserts agreement, so they cannot drift apart silently. Engagement
classification is pure (stdin text: boot trail beats env fallback).
Live-validated on this rig (stock-PCIe dual): preflight and report both
correctly silent; verdict matrix + classifier + probes covered
hermetically. Full scripts gate 66/66.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Root cause of club-3090 #548, pinned from the stack trace: A1's
promotion accidentally committed the whole torch_compile cache (3,667
files, ~20 MB) compiled on THIS sm_86 rig, and the compose warm-start
mount fed it to every fresh pull. On sm_120 the AOT graph (baked
against Marlin-processed FP8 layers -> reads layer.workspace) loads
onto Cutlass-processed layers -> AttributeError -> restart loop.
vLLM's AOT cache key doesn't include arch/kernel selection, so the
cross-arch hit is silent (upstream issue to follow; UPSTREAM.md row
updated with the pinned mechanism).
Fix = the house pattern every other model already has: cache contents
gitignored (cache/.gitignore + README), directory kept for the mount,
local files untouched (our warm-start intact). Fresh users pay one
~60-90s compile on first boot and warm-start locally thereafter --
against THEIR OWN silicon's kernel selection.
Also retires the recurring dirty-tree noise from best_config files
updating during our own runs.
Full scripts gate 65/65.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Everything the #246 runner dogfood on this rig surfaced, in one PR:
- verify-stress STRESS_FAST=1 (opt-in; gates keep full mode): skips
probe 7's large fresh needles (near-duplicate the ladder's depths)
and caps the ceiling ladder at 2 rungs (~2/3 anchor + ceiling).
MEASURED on the identical vllm/dual boot: 861s -> 361s (-58%).
Depth coverage stays 4-anchor (bench probe 10K/90K + mid + ceiling).
- VRAM margin attribution: compose `count: all` (DeviceRequests
Count:-1) and VISIBLE_DEVICES=all now resolve as a DETERMINED
"all host GPUs" -- kills the spurious per-rung WARN on dual rigs.
- VRAM margin aggregation: MIN per-card free instead of sum. TP OOMs
on the first card to run out; the sum overstated dual margins ~2x
(906 MB reported where the honest figure was 453 MB). SEMANTIC
TIGHTENING: multi-GPU margin advisories now fire against the real
per-card floor. Ceiling test updated to the new contract.
- arch-ab: lean tier exports STRESS_FAST=1 (per-arm ~30 -> ~20 min);
margin advisory surfaced in the summary table + a mid-run "ladder
CLEAN, not a failure" note so first-time runners don't abort on the
rc=1 advisory; bundle print points at the #246 test thread.
Full scripts gate 65/65. The e4m3-below-sm_8.9 refusal is the prior
commit on this branch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
vLLM hard-rejects fp8_e4m3 KV below SM 8.9 at boot -- without the guard
a bare run on an Ampere rig (default arms include e4m3) sits in
switch.sh's ready-wait until the 10-minute timeout instead of getting a
clear refusal. The message says why: on Ampere there is no arch delta
to measure. --arms e5m2 (control-only) stays possible.
test-arch-ab: bare-run-on-3090 refusal + control-only-on-3090 pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
report.sh --full-calibration instead of the bare default: seconds, no
test re-runs, and the predicted-vs-actual VRAM matrix on the volunteer
card class is direct Phase 2 input (32 GB envelope calibration).
Deliberately NOT --full -- that re-runs the whole battery (~43 min)
against only the last arm's container; the arms already carry that data.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
report.sh default mode (~2 s, path/host/user redaction ON) rides in the
tarball — PCIe lane width / driver / power caps / WSL-vs-bare-metal is
the context that makes cross-rig A/B variance interpretable (the same
mechanism that auto-flagged the x4-lane slot in #158).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
One command per volunteer rig: runs the pilot variant once per KV-dtype
arm (fresh symmetric boot each), verify-full + bench n=5 + NIAH ladder
per arm -- soak and quality packs deliberately skipped (quality
consolidates at promotion time per the #246 test plan; ~25-30 min/arm).
Ends with a per-arm comparison table and ONE attachable tarball.
Arms: e5m2 (control) / e4m3 (native FP8 KV, sm_89+) / nvfp4 (Blackwell
opt-in, refused below sm 10.0) / fp8w (vllm/qwen-27b-dual-max STOCK --
FP8-weights checkpoints reject fp8 KV, so this arm measures the
native-FP8-WEIGHTS lift instead; dual-rig only).
Guard rails: variant auto-pick by GPU count (2+ -> vllm/dual, 1 ->
vllm/minimal); KV arms are explicit KV_CACHE_DTYPE pins (unambiguous on
any launcher version); the running container's --kv-cache-dtype is
asserted live before any bench time is spent; --dry-run / --resume /
--full escape hatch.
test-arch-ab: hermetic plan/refusal contract via CLUB3090_FAKE_GPUS.
Full scripts gate 65/65.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
launch.sh/switch.sh now detect GPU arch and export KV_CACHE_DTYPE=
fp8_e4m3 (native FP8 compute) on sm_89+ cards for the pilot slugs
(vllm/dual, vllm/minimal). The injected value is the hardware
profiles' dormant kv_format_default.balanced -- one source of truth
shared with the pull gates; the Ampere no-op is data equality
(3090-class balanced = fp8_e5m2 = compose default -> nothing emitted),
not a code branch.
Injection guards (all load-bearing):
- pilot allowlist only; expansion gated on the #246 cross-rig A/B
- per-variant: registry kv_format == fp8_e5m2 only (int8-PTH/TQ/bf16
slugs never touched -- compressed-tensors weights reject fp8 KV)
- vLLM-family variants only; explicit user KV_CACHE_DTYPE= wins;
unmapped cards / heterogeneous rigs / no nvidia-smi -> no injection
- direct `docker compose up` keeps Ampere-safe compose defaults
Rides the existing resolve-variant-pin export seam (new optional
--gpu-spec); VLLM_ATTENTION_BACKEND is whitelisted but ships no value
(vLLM auto-detect stays the default until measured). Preflight banner
names the detected arch class.
Consistency fixes the injection exposed:
- gates.py _ARCH_KERNEL_SM: fp8_e4m3 9.0 -> 8.9 (vLLM's real floor is
SM89+; we'd otherwise inject e4m3 on 4090s our own pull-gate calls
unloadable) + new nvfp4: 10.0 row (v0.24.0 literal had NO gate --
a 3090 pull of an nvfp4 config wouldn't have been rejected)
- Blackwell hardware profiles: nvfp4 declared as CANDIDATE capability
(engine list unchanged until validated -- gates take the intersection)
- kv-calc: projected nvfp4 rows (bytes/elem + activation coefs),
calibration unchanged
Docs: HARDWARE.md new section, KV_MATH/QUANTIZATION/DTYPE_MATRIX rows
(incl. retiring the Genesis-era "e4m3 undertuned per #51" advisory in
favor of the A/B).
Validated: 8-case injection matrix in test-launch-compat; live no-op
on the real 2x3090 (spec built, nothing injected); faked 4090 through
the real bash detection path emits e4m3; compose interpolation both
ways; kv-calc --calibration green; full scripts gate 64/64.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
The single-card Gemma-4 default was the catalog's only default with no
baseline row (wave-2 PRIORITY gap). Full gate on Anbeeld's OFFICIAL
v0.3.2-preview digest (the current beellama-local pin), rebench tag
gemma-dflash-regate:
- decode 44.91 narr / 79.76 code (n=5, CV 2.1%/5.6%), TTFT 115 ms
- NIAH ladder clean to 117,513 tok (91% of 128K); ceiling margin 929 MB
(< 1024 MB bar -> caveat stays, but up from ~177 MB pre-v0.3.2)
- 8-pack current harness: 108/150 off / 113/150 on (+5 in-band wash,
think-off default re-confirmed)
- soak PASS (p50 59.8, 100.6% retention, 0 growth, 0/100 silent-empty)
Row seeded via catalog-baseline.sh (first non-vLLM engine through the
producer flow); BENCHMARKS row + compose header (Quality + margin
numbers) updated to match.
Tooling fallout the gate exposed:
- rebench-full.sh hardcoded RUNS:-3/WARMUPS:-1, silently under-running
the bench protocol on every orchestrated gate. The A1 "n=3 env leak"
attribution was wrong -- it was this line (baselines comment
corrected). Fixed to protocol 5/3; n=3 had flattered gemma-dflash
code TPS by +8%.
- catalog-baseline.sh counted prefill-probe run-lines into its bench-n
gate (read n=8 for an n=5 log); now reads the summary headers.
Full scripts gate 64/64.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Reviewed dispositions of the wave-1 gap list (verdict sheet approved
2026-07-04):
- vllm/dual + vllm/qwen-27b-dual-fast (alias mirror): FRESH from the
2026-06-30 v0.24.0 pin-bump gate row (decode 70.7/93.5, NIAH->240K,
soak PASS) — measured on the current vllm-stable pin, so no
born-stale archaeology needed after all.
- ik-llama/iq4ks-mtp-vision: carried decode (60.39/72.4) from iq4ks-mtp
per the 2026-05-25 vision re-tune row; carried qualifies because the
vision delta doesn't touch the decode path.
- ik-llama/apex-fit-q8q5: 2026-05-28 row (decode 105.63/156.80, n=5),
NIAH clean@180K.
- vllm/gemma-26ba4b-single: 2026-06-06 row (decode 169.2/219.9) + full
8-pack (98 off / 109 think-on), NIAH clean@161K — FRESH (gemma-stable
still pins v0.22.0).
- vllm/agents-a1-dual: decode pair order corrected from the tag
artifact (narr 154.0 / code 153.8 — wave-1 had them swapped).
- Footer: SEED WAVE 2 list -> KNOWN GAPS dispositions (vllm/minimal,
gemma-12b pair, beellama/gemma-dflash PRIORITY re-gate, deckard).
catalog-baseline.sh: insertion anchor updated for the retitled footer
(accepts both titles); test-catalog-baseline.sh now guarantees the ADD
path (strips the seeded vllm/dual row from its copy) and asserts added
rows land before the footer — the marker-drift class this bump caught.
Guards: test-baselines 15 rows joined, same 4 wave-1 stale WARNs, all
5 new rows fresh. Full scripts gate 64/64.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Closes the rescore-materialization gap the induction tool's first live
run surfaced (#562): the benchlocal-cli 'rescore' subcommand ALREADY
supports write-back (--in-place / --output) — the gap was practice, not
a missing upstream feature. The T2 rescore ran to stdout only, so the
published A1 thinking-on 110/150 diverged from the tag artifact (108).
- A1's rescore MATERIALIZED into the tag JSONs (rescore --in-place; OFF
105 unchanged, ON 108→110) — induction now extracts 110 and the
regenerated corpus record carries 110; artifact, corpus, baseline row
and publication all agree. Pre-rescore JSON backed up outside the tag.
- docs/QUALITY_TEST.md: 'Rescoring saved results — MATERIALIZE, don't
just read' — the rule (a rescore that changes a published number must
be written back in the same session), the command, what rescore can't
re-run (sandbox packs), and the A1 case as the cautionary example.
- catalog-baseline.sh header: reads-artifacts-as-truth warning + the
header's gate text updated to the actual n>=3/warn-under-5 behavior.
- baselines.yml A1 comment: gap note -> materialized + doc pointer.
Guards green (test-catalog-baseline, test-baselines).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Decode-only TPS can't express real trade-offs anymore (the W8A8-vs-FP8
result was a prefill-corner-vs-decode-corner split) — this adds the
CANONICAL prefill/TTFT measurement to bench.sh per the design's sourcing
rule, with the protocol gotchas productized:
- bench.sh PREFILL PROBE (default-on; PREFILL_PROBE=0 / PREFILL_DEPTHS /
PREFILL_RUNS): warm + n measured per depth (10K + 90K anchors; 90K is
inside the DeltaNet degradation regime and pairs with the NIAH ladder's
~94K rung). CACHE-BUSTED: fresh salted haystack per request — composes
serve enable_prefix_caching, an identical prompt re-measures the CACHE
HIT (vLLM's prefix cache is block-chained; unique first line breaks the
chain). SELF-CALIBRATING: word-count heuristics overshoot tokens ~1.3x;
the warmup's reported prompt_toks scales the measured runs (target^2/
actual) to within ~4% of the requested depth. Depths exceeding the
served ctx SKIP with a note. DUAL METRIC, labeled: prompt_tokens/TTFT =
client-observed (user-truth: incl tokenization+transfer+scheduling) AND
the vLLM stats-log windowed rate = engine-internal (compute-truth) —
at 93K on A1 they differ by ~7s of non-prefill overhead (5.5K vs ~10K
t/s); never cross-compare kinds (stack LEARNINGS row added).
- measurement_record parser: per-block pass -> prefill_tps_by_ctx +
ttft_ms_by_ctx extensions; the canonical short-prompt ttft_s is
PROTECTED from the probe blocks (the old last-occurrence rule would
have swallowed the 90K block's 17s TTFT).
- catalog-baseline.sh: rows gain prefill_tps {10k: N, 90k: M} (parsed
via THE record parser, no second grammar) + ANCHOR CALIBRATION at
induction: the probe's deep anchor vs the NIAH ladder's nearest rung —
agreement (0.7-1.3) certifies the ladder's whole depth curve; A1 live:
probe 5459-5584 t/s @93K vs ladder 7403 @94K = ratio 0.74-0.75, OK.
Divergence warns with an investigate message (design: a finding).
- test-baselines schema: prefill_tps = dict of numeric depth points.
test-catalog-baseline fixture: probe blocks + TTFT-pollution guard +
anchor-OK assertion.
Live-validated 3x against the serving A1 (262K): 10K = 7977 t/s CV 1.1%
TTFT 1.25s; 93K = 5584 t/s CV 0.5% TTFT 16.1s; engine-log ~10K t/s.
Full scripts gate green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
The consumer half of the producer wiring: THIS RIG's corpus records now
render next to the shipped bar in the catalog preview — the design's
'c3 overlays the bar with your numbers, badged'.
- LocalMeasured projection + CatalogEntry.local_measurement: the newest
#249 corpus record per variant slug (decode point, both 8pk arms, pin,
date). One canonical-short decode point per record — the narr/code
pair split is a parser note for 2c.
- services.local_measurements(): pure-fs corpus scan; newest-per-slug by
the record's _recorded_at stamp (NEW — rebench-full now stamps it at
write; mtime+line-order fallback covers pre-stamp records); malformed
lines skipped. Joins in the same enrichment pass — still zero
subprocess on the catalog path.
- Preview rework: the measured line is the BAR with provenance
(date · rig · submitted_by from the joined baseline); a stale bar gets
an explicit detail line ('† bar measured on <pin> — current <pin>;
re-bench owed'); a slug this rig has gated adds
'yours ~<decode> · 8pk <P/150> (<date> · this rig)'.
Live-verified: the corpus's first record (A1, from 2a's live run) joins
and renders 'yours ~154 decode · 8pk 105/150'. Tests: overlay semantics
(newest-wins by stamp, malformed-skip, entry join, no-record → None) +
the three-state preview (fresh bar + provenance + yours · stale pin
detail · no-yours). c3 suite 766/766; scripts gate green serial.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
The producer wiring: a validated gate run now flows into the corpus
automatically and into the shipped bar via one reviewed command.
- scripts/catalog-baseline.sh (§2.3, one mechanism / three entry points):
gate-validates the tag dir (verify-full pass HARD · bench n>=3 HARD,
n<5 WARN vs the canonical target · quality present unless --tps-only),
extracts decode TPS/TTFT/8pk both arms/ctx_validated (NIAH-ladder
verdict incl. token count)/provenance (pin via the same resolution the
emit join uses; rig+power via nvidia-smi, all overridable), UPSERTS the
baselines.yml row textually (comments preserved), prints a unified
diff. Never commits — rows ship via PR. --baselines-file for tests.
- rebench-full.sh: on completion, EXACT-container-match the served
engine to a registry slug (identity semantics — never port/substring,
the F9 rule) and append a fingerprint-complete #249 record (engine_pin
via resolve_variant_pin/compose fallback · hardware · power-cap ·
quality_8pk extensions · soak status) to the gitignored corpus, then
print the catalog-baseline.sh induction prompt. BYO/swap serves skip
with a note (no registry identity to record against).
- measurement_record.py: --quality-8pk / --quality-8pk-think-on ->
measured_extensions (the designed extension namespace; frozen schema
untouched).
- test-catalog-baseline.sh: hermetic (synthetic tag dir + a COPY of
baselines.yml) — extraction, dry-run no-write, add->replace upsert,
refusals (missing verify/quality/unknown slug), real-file checksum
guard.
TRUTH CORRECTIONS the tool's first live run surfaced (run against the
real agents-a1-fp8-dual tag): the A1 gate's bench was n=3 (a RUNS=3 env
leak into the overnight gate — 'n=5' was misstated in publications), the
BENCHMARKS TPS pair was order-garbled (artifact: narrative 154.0 / code
153.8), and the #81-rescored think-on total (110) was never materialized
back into the tag JSON (reads 108) — materialization gap documented in
the row comment. baselines.yml + BENCHMARKS.md corrected.
Live-validated: rebench record block ran against the serving A1 → first
corpus record written (vllm-agents-a1-dual__e9d28e22.jsonl: pin v0.24.0,
rtx-3090, 370W, quality extensions); induction dry-run reproduces the
row from the tag. Deferred to 2b/2c: c3 local-overlay + richer badges;
the bench 10K/90K prefill/TTFT probe. Gates: test-catalog-baseline +
test-measurement-record + test-baselines green; full scripts gate green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
The catalog's measured columns (TPS / 8pk) now come from ONE productized
source: scripts/lib/profiles/baselines.yml — the shipped, PR-reviewed
'bar' (accepted display projection of a validated gate run, with pin/rig/
power provenance) — joined per-slug into registry-emit --json with an
emit-computed staleness verdict. Consumers never read baselines.yml or
BENCHMARKS.md directly (BENCHMARKS stays the public human/cross-rig
ledger). Design: the catalog-baselines note (2026-07-02, §2/§5 slice 1).
- baselines.yml SEED WAVE 1: 10 rows with airtight traceability only
(rebench tags on disk: agents-a1 golden specimen + gemma-31b-dual;
unambiguous decode-class BENCHMARKS rows for the rest). 4 rows are
HONESTLY born-stale with documented reasons (llamacpp rolling-tag-era
x2, beellama pre-#296 image, 35B v0.22.0) — the guard's demo cases.
9 slugs still owed rows are listed as wave-2 gaps, no guessed numbers.
- registry-emit join: per-variant 'baseline' field; current pin resolved
the way launchers actually resolve it (engine-profile install.spec,
compose-image-default fallback for ik/llama.cpp) → 'stale' =
measured-pin != current-pin, null when undeterminable.
- test-baselines.sh: schema + slug-membership + ctx-parity (compose ctx
default == registry max_ctx, functional slugs) RED; pin-staleness
WARN-only (pin bumps must not block on immediate re-bench — the debt
stays visible). test-registry-json gains the 'baseline' contract key.
- c3: enrich_measurements = pure in-memory map off the joined field —
deletes BOTH the per-slug --explain fan-out (~4s/slug, the #439
option-3 leg) and the BENCHMARKS.md scrape from the catalog path
(Explain modal + cross-rig explorer keep their readers). Stale rows
render a † on the TPS cell + a status-line legend; full badge/overlay
treatment is slice 2. Measurement gains source='baseline' + stale.
- results/baselines/README: the three-store relationship (regression
corpus vs measurement records vs display bar) + the no-drift rule the
slice-2 induction tool enforces.
Verified: guard test green (10 rows, 4 stale-warned); live join emits
57 variants / 10 baselines; live c3 catalog shows all 10 with daggers on
exactly the stale 4 (A1 serving during the check: 154/154 - 105/150);
c3 suite 764/764; full scripts gate green serial. (First gate pass
tripped classifier/dedup by running CONCURRENTLY with the c3 suite —
.pull-captures pollution class; serial rerun clean.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Investigation of a Discord cross-rig report (WSL, 3060Ti+4090: verify
fails on the pinned v0.3.2-preview digest, passes on the old noonghunna
snapshot; NOT reproducible on our 3090s — verify-full all-pass) surfaced
three doc-level truths worth recording:
- The 50-series gap is a TOOLCHAIN ceiling: Anbeeld's CI builds on the
Dockerfile-default CUDA 12.4, whose nvcc cannot target sm_120 (no
cubin, max PTX compute_90) — every official tag lacks Blackwell.
Upstream ask filed: Anbeeld#85 (CUDA_VERSION 12.8.1 + arch list incl
120); when it lands we retire the noonghunna snapshot entirely.
- Three registry status_notes claimed launchers inject 'server-cuda-
v0.3.0' — stale since the 2026-06-12 pin bump; they inject the
v0.3.2-preview digest from engines/beellama-local.yml install.spec.
Fixed all three (qwen dflash, gemma-12b, gemma dflash) + honest
labeling of the noonghunna snapshot as v0.3.0-feature-level and
unmaintained (predates KVarN + v0.3.1 fixes).
- The engine-notes self-build guidance was outdated: FA_ALL_QUANTS is
hardcoded in Anbeeld's cuda.Dockerfile since our PR Anbeeld#48, so a
self-build needs only CUDA_DOCKER_ARCH (+ CUDA_VERSION=12.8.1 for
sm_120). Recipe verified against his master Dockerfile 2026-07-04.
UPSTREAM.md beellama row updated (dated entry + next-triggers; the 'no
official image' claim struck through as historical). Gates: YAML +
registry import clean; status-drift / profiles-compat / switch-parity
green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
First cross-rig confirmation: decode 152.2/152.3 (CV 0.0%) within ~1% of
the gate rig despite one card on an OCuLink eGPU dock at PCIe 4.0 x4 and
both capped 250W — single-stream TP=2 decode is VRAM-bandwidth-bound.
Full battery mirrors the gate (NIAH 6/6 to 240K, soak 100% retention,
agentic TTFT sub-linear); boot ran custom all-reduce via the PCIe-P2P
auto-detect, clean throughout.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
The A1 compose shipped (#546) without the 'Hardware metadata' block every
sibling carries, so preflight_compose_gpu_fit logged 'WARN: compose has
no hardware metadata; allowing boot' and skipped enforcement — visible in
guybrush01's #548 report. Add the block (min-vram 24 / gpu-count 2 /
TP 2 / Requires-sm 8.0+ — Marlin FP8 weight-only needs sm_80, unlike the
sibling's INT4 7.5+); preflight now parses + enforces (live rc=0).
Also propagate the #548 finding as Caveats (4): Blackwell sm_120 hits an
upstream vLLM v0.24.0 kernel-selection bug at boot ('QKVParallelLinear'
has no attribute 'workspace' — Cutlass W8A8 selected, a Marlin-path
attribute expected). Workaround shipped as a commented env line
(VLLM_TEST_FORCE_FP8_MARLIN=1 — forces the exact weight-only path this
compose's gate validated; verified present in the v0.24.0 image) +
mirrored into the registry status_note so switch.sh's NOTE carries it.
Gates: compose_meta_get parses all 4 keys; docker compose config valid;
test-compose-status-drift / registry-disk / preflight-gpu-fit green;
full scripts gate green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
F2 — the Catalog preview strip clipped a WRAPPING caveat line: border
eats 2 rows of max-height 6 → 4 content lines; the dual-fast caveat
wrapped past that and lost its tail. max-height 6→8 + overflow-y auto
(height stays auto — short previews don't grow). Regression test pins a
wrapped-caveat entry to its full height with the tail visible.
F5 — the #544 deliberate deferrals:
- wait_gpu_vram_settle wired into ALL model/studio scene handlers
(27b, 35b-a3b, gemma-12b, deckard, ai-studio) — was gemma-int8 only;
every scene switch now lets the torn-down scene's VRAM release before
the next boot (#535 class).
- mode_off gains an engine-prefix CATCH-ALL: the enumerated stop_*
lists cover gpu-mode scenes, but a catalog-launched engine
(switch.sh <slug>) survived 'off' — caught LIVE during validation
when off left vllm-qwen36-27b-minimal serving and the 27b TP=2 scene
booted straight into its residue (the exact #535 failure). Any
remaining vllm-/llama-cpp-/ik-llama-/sglang-/beellama- container is
now stopped, with a named notice.
- c3 preflight-error visibility (the third residual): verified
already-plumbed — switch.sh's #544 refusal exits fast, its
[preflight] ERROR lines stream into the serve pane, serve_failed
stamps ✗ + [!] capture (pinned by existing tests). No change needed.
Validation: bash -n + full scripts gate green (by exit code); c3 suite
763/763; LIVE scene cycle off → 27b → off on the rig — boot ready
(qwen3.6-27b, 21.5 GiB/card), fixed off left ZERO engine containers
(verified with a non-enumerated probe container that the catch-all
stopped), both GPUs at 1 MiB after.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
F4 — the Logs drill had no copy affordance: LivePane is RichLog-family,
which captures the mouse and implements no text selection (Top/Config
are selectable Statics). LivePane (tui-core, shared) now keeps a
plain-text tail buffer (markup stripped, capped 2000 lines) exposed via
tail_text(); c3 routes [Y] to the visible pane's tail on the Containers
Logs drill and the ③ Gate run output. Placeholders and display-only
notes are NOT buffered (append_line buffer=False), so [Y] on an idle
drill falls through to the highlighted container name as before. The
transient #serve-live pane keeps the established slug-copy semantics.
F8 — the idle placeholder was hardcoded test-runner wording ('Ready.
Select a test and press Enter to run.') that leaked into the docker-logs
drill. The placeholder is now constructor-owned (neutral 'Ready.'
default); c3 mounts pass pane-specific copy (logs drill / gate output /
'' for the hidden serve pane). c3t is unaffected (own vendored pane).
Tests: 6 tui-core unit tests (tail buffer, unbuffered notes, cap,
placeholder contract; 76/76) + 4 headless (placeholders, tail copy wins
over row-primary on the Logs drill, idle fall-through to container name,
③ Gate tail); c3t 83/83; c3 suite 762/762.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
estate.yml is a desired-state PLAN, but report-state reported its
instances as 'active' without checking docker — so every consumer
treated a leftover ~/.club3090/estate.yml as live GPU claims. The T1.1
audit hit this on an EMPTY rig: the c3 serve confirm warned '⚠ Starting
this will STOP estate llama-gpu0 (GPU [0]), estate llama-gpu1 (GPU [1])'
with nothing running. Scary + wrong for a fresh user.
estate_cli (the layer that owns estate liveness — fix here, not a c3
filter):
- report-state --json: each instance gains 'container' (the
club3090-<name> it would own) + 'running' (docker probe: true / false
/ null when docker itself is unavailable), payload gains
'running_count'. Additive — existing keys unchanged.
- report-state text view: per instance '— running' / '— down (plan
only)' / '— liveness unknown'.
c3 reconcile gate (consumer): an estate instance is a claim unless
running == false. true / null / MISSING (older CLI output) all stay
claims — the dual-writer gate fails CLOSED on unknown liveness, and a
mid-boot instance (running container, VRAM not yet allocated) still
conflicts.
Tests: estate JSON contract test extended (running/container/
running_count; running asserted 'in (False, None)' so the test passes
with or without docker); 3 new c3 gate tests (stale plan on empty rig →
SAFE; running:true → claim even with idle GPUs; running:null → fail
closed; the legacy no-key fixtures double as the missing-key case).
Scripts gate all green; c3 suite 758/758. Live repro on this rig (which
has the exact stale estate.yml): running_count 0, reconcile now reports
estate_claims=[] while keeping the TRUE conflict (the actually-serving
vllm container).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
