The producer wiring: a validated gate run now flows into the corpus
automatically and into the shipped bar via one reviewed command.
- scripts/catalog-baseline.sh (§2.3, one mechanism / three entry points):
gate-validates the tag dir (verify-full pass HARD · bench n>=3 HARD,
n<5 WARN vs the canonical target · quality present unless --tps-only),
extracts decode TPS/TTFT/8pk both arms/ctx_validated (NIAH-ladder
verdict incl. token count)/provenance (pin via the same resolution the
emit join uses; rig+power via nvidia-smi, all overridable), UPSERTS the
baselines.yml row textually (comments preserved), prints a unified
diff. Never commits — rows ship via PR. --baselines-file for tests.
- rebench-full.sh: on completion, EXACT-container-match the served
engine to a registry slug (identity semantics — never port/substring,
the F9 rule) and append a fingerprint-complete #249 record (engine_pin
via resolve_variant_pin/compose fallback · hardware · power-cap ·
quality_8pk extensions · soak status) to the gitignored corpus, then
print the catalog-baseline.sh induction prompt. BYO/swap serves skip
with a note (no registry identity to record against).
- measurement_record.py: --quality-8pk / --quality-8pk-think-on ->
measured_extensions (the designed extension namespace; frozen schema
untouched).
- test-catalog-baseline.sh: hermetic (synthetic tag dir + a COPY of
baselines.yml) — extraction, dry-run no-write, add->replace upsert,
refusals (missing verify/quality/unknown slug), real-file checksum
guard.
TRUTH CORRECTIONS the tool's first live run surfaced (run against the
real agents-a1-fp8-dual tag): the A1 gate's bench was n=3 (a RUNS=3 env
leak into the overnight gate — 'n=5' was misstated in publications), the
BENCHMARKS TPS pair was order-garbled (artifact: narrative 154.0 / code
153.8), and the #81-rescored think-on total (110) was never materialized
back into the tag JSON (reads 108) — materialization gap documented in
the row comment. baselines.yml + BENCHMARKS.md corrected.
Live-validated: rebench record block ran against the serving A1 → first
corpus record written (vllm-agents-a1-dual__e9d28e22.jsonl: pin v0.24.0,
rtx-3090, 370W, quality extensions); induction dry-run reproduces the
row from the tag. Deferred to 2b/2c: c3 local-overlay + richer badges;
the bench 10K/90K prefill/TTFT probe. Gates: test-catalog-baseline +
test-measurement-record + test-baselines green; full scripts gate green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
1d974459c9