AI Studio generation bugs (e.g. #510 "ComfyUI generation error" on the HiDream lane) weren't
diagnosable — report.sh captured hardware/stack but not the ComfyUI log where the actual
workflow-execution error lives, so reporters had no easy way to include it.
Add an opt-in `--studio` flag (off by default — verbose, only relevant for studio bugs) that
appends the container log tails for the AI Studio stack: comfyui (400 lines — the generation
engine, carries the error trace) + studio-director / orchestrator / image-shim / tts /
step-voice / gallery (200 lines each). Each in a collapsible <details> block with its run
status, ANSI colour stripped, and piped through the same redact() as the rest of the report.
If no studio containers exist, it says so.
Also: add the `--studio` row to the bug-report issue template (the bug-type → command table)
so studio reporters reach for it.
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Co-authored-by: noonghunna <10742901+noonghunna@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The Profile (at-a-glance) header + required Status field is gate-tested
(test-compose-status-drift) but was documented only in CLAUDE.md/AGENTS.md
(the agent guide) — the human-contributor surfaces never stated it, so a
submission could omit Status and only fail in CI. Surface the requirement
(link, don't duplicate the schema) on every path:
- CONTRIBUTING: new "Profile header on every compose" rule next to one-per-PR.
- PR template: a "Profile header complete" checkbox for compose PRs.
- ADDING_MODELS Step 3: a "Profile header (mandatory)" subsection.
- BRING_YOUR_OWN §4: a header bullet for contributed composes.
Also reciprocal-link the two onboarding docs: ADDING_MODELS now points paths
1-2 (serve/tune/validate without the catalog) to BRING_YOUR_OWN, framing
itself as the promotion step after a config validates there.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Repo-wide follow-up to the compose quant-layer move (9821c94). The
mechanical move + registry/launch/test rewire covered the launchable
surface; a full-tree sweep found compose-path refs the move invalidated
in docs, two functional scripts, and one half-migrated mapping.
Functional fixes:
- bench-row-formatter.sh infer_compose_path(): 6 dual entries
(int8-tq3, tq3-mtp-genesis, tq3-nomtp, tq3-mtp, int8, bf16) were left
at bare dual/<file>.yml while the rest were migrated -> would emit
dead compose paths into BENCHMARKS rows.
- residency-instrument/run-instrumented-soak.sh: case->COMPOSE_FILE
paths (long-text, long-text-no-mtp, tools-text, dual default) now
resolve under <quant>/.
Docs: README layout line + tree, engine/model READMEs, patch-README
quick-recipes, diagnostics, FAQ/CLIFFS/KV_MATH/DTYPE/MULTI/SINGLE/
STRUCTURED_COT/TQ3/UPSTREAM, issue template, sglang cross-refs
(-> vllm prod path). Per-model targets: qwen-vllm->autoround-int4,
llama-cpp->unsloth-q4km, gemma defaults (bf16-mtp/fp8-mtp),
carnice->own slug dir.
Intentionally left as historical/append-only records: CHANGELOG x2,
BENCHMARKS row-labels (live paths already correct in row bodies),
calibration source: provenance citations, switch.sh/parity history
comments. Separate follow-ups: gpu-mode.sh (#417 deprecated-repo
repoint), bench-row-formatter compose_display() docker-compose.yml
branch (PR-B). Flagged pre-existing-stale: dual/int8-tq3.yml in
pr40798/pr40914 READMEs (predate this refactor; ambiguous target).
Guard tests (registry-disk, mounts-resolve, switch-parity,
launch-compat) all PASS post-edit. Leak-clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
We don't distribute a club-3090 vLLM image to users, so the
build-vllm-image.yml workflow + its surrounding docs are dead weight. The
workflow had also been failing on every tag push and the weekly cron because
its base nightly (nightly-1acd67a7) was purged from Docker Hub (#167/#407) —
that red CI on the v0.8.4 tag is what surfaced this.
Removed:
- .github/workflows/build-vllm-image.yml (the GHCR image builder)
- docker/vllm-club3090/Dockerfile (its build recipe)
- docs/CI_RUNNER_SETUP.md (build/distribution doc)
- README + UPSTREAM references to ghcr.io/noonghunna/vllm-club3090
- docs/README index link to the deleted CI doc
Kept: the generic VLLM_IMAGE override (now documented against
vllm/vllm-openai:latest — also the #167 workaround). The launch-compat test
still exercises that override, just with an upstream image.
Recoverable from history if we ever want to ship an image again.
NOT touched: patch_attribution.py still references the dockerfile_bake
delivery mode — left for a separate decision (internal, harmless).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The smoke-gating mechanism required a self-hosted GPU runner that we don't
maintain + administration:read permission on GITHUB_TOKEN that the default
scope lacks. Result: :latest never got promoted on the v0.7.0 release run.
Simplification:
- Remove detect-smoke-runner job (no permission dependency)
- Remove smoke job (no runner dependency)
- Add promote-aliases job: unconditionally promote :latest + :nightly-stable
to the just-built dated tag using docker buildx imagetools create
- Build job continues to produce dated + :club-vX.Y.Z (on tag pushes)
- retention job unchanged (still gated on schedule/dispatch)
- Workflow permissions reduced: actions:read no longer needed
The Docker Hub :latest convention is "most recent, no guarantees". Users who
want verified images pin to :club-vX.Y.Z. If smoke-gating becomes useful when
we get a registered runner, it can layer on top as a separate post-build job
without re-introducing the failure modes.
docs/CI_RUNNER_SETUP.md rewritten to reflect the simplified flow + manual
:latest bootstrap recipe. docs/UPSTREAM.md updated to point at :club-vX.Y.Z
as the recommended pin instead of :latest.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CHANGELOG.md is now auto-generated from commit messages by git-cliff in
the release workflow. Hand-edits below the static header will be wiped on
the next tag.
Workflow (`.github/workflows/release.yml`):
- On tag push (`v[0-9]+.[0-9]+.[0-9]+`):
1. Render GitHub Release body: `git-cliff --latest --strip header`
→ just the per-version section, no SemVer preamble repeat
2. Regenerate full CHANGELOG.md: `git-cliff` (default = all tags)
→ preserves header + all historical sections
3. Commit CHANGELOG.md back to master with `[skip ci]` marker
4. Publish GitHub Release with the latest-only body
Template (`cliff.toml`):
- `[changelog].header` now holds the SemVer preamble + CalVer→SemVer
mapping table (preserved across regens; stripped from GitHub Release
bodies via `--strip header`).
- `body` template now renders the **full commit message** (subject as
bold bullet, body indented below) instead of just the first line.
Rich narrative I write in commit message bodies (tables, validation
numbers, before/after diffs) now flows into both CHANGELOG.md and the
GitHub Release page from the same source.
- Per-release Pin/Diff footer guarded with `{% if version %}` so the
Unreleased section doesn't emit empty links.
CHANGELOG.md replaced with the auto-gen output. Past hand-written tables
and phase breakdowns are replaced by the corresponding commit messages
(those were already rich for commits that mattered — v0.3.1 soak-helper
fix has its Before/After table in the commit body and renders fine).
Going forward: just write rich commit messages and tag. Both surfaces
update automatically. No hand-edit of CHANGELOG.md required.
Four small fixes addressing low soak-test compliance in cross-rig bench
contributions. Audit of 5 recent #113/#107/#102/#104/#93 showed soak data
IS being run but it's hidden in the main report and the dedicated
template field comes out empty (template said "leave blank if you ran
--full"). Older BENCHMARKS rows often omit soak verdict entirely.
1. **`.github/ISSUE_TEMPLATE/numbers-from-your-rig.yml`**: replace optional
"soak summary" textarea with a required dropdown listing PASS / borderline /
FAIL / Skipped+reason / Not-yet-run. Verdict is now grep-able even when
the data is buried in the main report textarea.
2. **`scripts/soak-test.sh`**: add `--continuous` / `--quick` / `--fresh`
flags + `--help` + cleaner usage docs. Was 5 env vars to invoke
(`SOAK_MODE=continuous SOAK_SESSIONS=5 SOAK_TURNS=5 CONTAINER=... ENDPOINT=...`);
now `bash scripts/soak-test.sh --continuous` does the same with
auto-detect (existing logic preserved + exposed). Env vars still work
for back-compat.
3. **`scripts/report.sh`**: when `--bench` (or partial) ran without
`--soak`/`--full`, append a "⚠ Soak: not included" reminder block to
the report so contributors know what's missing before pasting into
the issue template.
4. **`BENCHMARKS.md`**: Notes-column convention — every row should start
with explicit `Soak: ✓ PASS` / `⚠ borderline` / `✗ FAIL` / `—` so
readers can grep at a glance. Updated 2 recent rows (ygafarov #113,
JDWarner #107) to use the convention. Older rows backfill as the
convention spreads.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Release Drafter only catches PRs; this repo's workflow is mostly
direct-to-master commits per the auto-mode pattern, so 80%+ of
substantive changes were invisible to it.
git-cliff is commit-based: parses every commit since the last tag,
categorizes by conventional-commit prefix (`docs:`, `scripts:`,
`composes:`, `models:`, `fix:`, `chore:`) with keyword-based fallback
for un-prefixed commits (`Document Cliff 1`, `Add Gemma 4 compose`,
`power-cap-sweep:`, `verify-full.sh:`). Squash-merged PRs flow
through the same parsers since their squashed title becomes the
commit message.
Workflow triggers on `v[0-9]+.[0-9]+.[0-9]+` tag push, runs git-cliff
with `--latest`, creates a GitHub Release with the categorized body.
Tested locally on 274 commits since repo init: 49 land in catch-all
"Other" (genuinely unconventional one-off commits); rest distribute
across 9 categories. Existing v2026.05.09 release stays as-is
(hand-written); next CalVer tag onwards uses this pipeline.
Future cadence: `git tag v$(date +%Y.%m.%d) && git push origin v...`
— workflow does the rest.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Maintains a draft GitHub release continuously, categorizing changes by
PR label or conventional-commit prefix in the title. Categories:
- 🎯 New models + serving paths
- 🔧 Pin bumps + upstream
- 📊 Benchmarks + cross-rig data
- ⚠️ Cliffs, gotchas, regressions
- 🛠️ Scripts + tooling
- 🐛 Bug fixes
- 📝 Documentation
- 🧹 Maintenance
Versioning is CalVer (year.month.day) — this repo is a rolling config
stack, not a versioned API, so date tags are honest about that. Tag
manually via `git tag v2026.05.09 && git push origin v2026.05.09`;
Release Drafter populates the body, you click "Publish" in the UI.
Autolabeler maps existing commit-prefix conventions (docs:, scripts:,
composes:, models:, fix:, chore:) to categories without per-PR label
discipline. Files-based fallback catches direct-to-master doc edits
that touch BENCHMARKS.md / HARDWARE.md / CLIFFS.md.
Closes the loop on @laurimyllari's "include git commit in output"
suggestion (issue #112) by giving cross-rig contributors a stable
version tag to cite.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Compose files now live under `<model>/<engine>/compose/<topology>/<file>.yml`,
with topology as a folder rather than a filename prefix. Solves all 7
inconsistencies surfaced in the post-rename audit (single-card composes
without `single-` prefix, unsuffixed `docker-compose.yml` ambiguity,
fine-tunes encoding model name in filename, etc.) by making the directory
hierarchy enforce the convention.
Layout:
models/<model>/<engine>/compose/<topology>/<feature>.yml
Where:
- <model>: qwen3.6-27b, gemma-4-31b
- <engine>: vllm, llama-cpp, sglang
- <topology>: single, dual, multi3, multi4, multi8
- <feature>: docker-compose.yml (default) | turbo.yml | dflash.yml | etc.
Each topology subdir has a `docker-compose.yml` for the recommended
starter — bare `cd <topology> && docker compose up` works because
docker compose finds that filename automatically. Variants drop the
`docker-compose.` prefix since they're invoked via `-f` flag.
27 compose file moves total:
- 18 Qwen vLLM composes redistributed across single/dual/multi4
- 2 Qwen llama-cpp composes into single/
- 6 Gemma vLLM composes redistributed across single/dual
- 1 untracked qwopus-bf16mtp moved to dual/
Inside each compose: relative paths to `../patches/` and `../cache/`
bumped to `../../patches/` / `../../cache/`, and `../../../../models-cache`
to `../../../../../models-cache` (one extra `..` for the new depth).
Reference updates across 148 files (BENCHMARKS, all docs, CHANGELOGs,
sibling-table cross-references in compose headers, scripts, patch
READMEs, .github issue templates, tools/residency-instrument).
scripts/switch.sh VARIANTS map updated; tags themselves unchanged
(`vllm/dual` → `dual/docker-compose.yml`, `vllm/dual4` → `multi4/docker-compose.yml`,
`vllm/gemma-mtp` → `gemma-4-31b/.../dual/docker-compose.yml`, etc.).
AGENTS.md "Compose layout" section rewritten to describe the new
hierarchy, with concrete examples and the fine-tune exception
(`dual/carnice-bf16mtp.yml` carries the fine-tune name as a filename
prefix until the fine-tune graduates to its own model directory).
All switch.sh paths verified to resolve to actual files post-move.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After shipping report.sh --full / --soak / --stress flags in 8a29b95, the
bug report flow + troubleshooting guidance still pointed at older patterns.
Updated:
- .github/ISSUE_TEMPLATE/bug-report.yml — replaced single-flag suggestion
with a "pick the flag that matches your bug" decision table:
- boot crash / wrong output / tool-call regression → --verify (~2 min)
- OOM mid-conversation / agentic cliff → --soak (~25 min) — only test
that catches Cliff 2b
- TPS regression / cross-rig perf → --bench (~5 min)
- not sure / capture everything → --full (~35 min)
- docs/FAQ.md "Found a bug — what should I include?" — replaced the generic
"verify-full.sh output" note with the same flag decision table; updated
the troubleshooting-ladder pointers to use --verify (boot path) and
--full (multi-card TQ3+Genesis intersection bugs).
- docs/MULTI_CARD.md cross-rig contribution callouts (×2) — recommended
command upgraded from --bench to --full (or --bench fallback if soak
time-budget is tight, with explicit caveat that --bench skips Cliff 2b).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/report.sh now has four optional sections:
--verify verify-full.sh (~1-2 min)
--stress verify-stress.sh (~5-10 min, 7/7 incl. Cliff 2 needles)
--soak SOAK_MODE=continuous + summary.md embed (~25 min, catches Cliff 2b)
--bench bench.sh (~3 min, canonical TPS)
--full all four (~35 min total, the canonical cross-rig pass)
Why soak as its own flag: verify-full + verify-stress + bench all PASS on
configs that FAIL the multi-turn continuous soak (Cliff 2b at ~25K accumulated
tokens). Until upstream lands a fix, soak is the only test that catches the
agentic-workload failure mode that bit issues #41 / #42 / #43 / #45.
Propagated --full as the recommended single-command path through:
- README.md (bug/bench callout)
- CONTRIBUTING.md (Numbers from your rig + new compose variant gate)
- .github/PULL_REQUEST_TEMPLATE.md (one checkbox covers verify+stress+soak+bench)
- .github/ISSUE_TEMPLATE/numbers-from-your-rig.yml (single paste includes soak)
- BENCHMARKS.md ("How to add a row for your rig")
Backward compatible: existing --verify and --bench flags unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add .github/PULL_REQUEST_TEMPLATE.md with the rig-report / verify /
soak-continuous / bench / BENCHMARKS row checklist that PR #44 surfaced
as missing. New compose variants now have an explicit gate list.
- CONTRIBUTING.md: new "Submitting a new compose variant — full gate list"
section explaining the why behind each gate, cross-linked to docs/CLIFFS.md
and #41 for the soak-continuous rationale.
- numbers-from-your-rig.yml: optional v2 SOAK_MODE=continuous summary field
for cross-rig Cliff 2b validation.
- docs/STRUCTURED_COT.md: replace "Phase 2 pending" stub with measured
results — Holiday tagline (4/6 rescue, 23-token think), DeepSeek scratchpad
(5/6 rescue, 387-token think), PROMPT_TERSE (5/6 rescue at 75 tokens with
no FSM mask). Headline reframe: PROMPT_TERSE rescuing 5/6 with the same
G/A/E shape as current suggests FSM enforcement is the mechanism causing
those regressions, not absence of structure. Phase 3 plan included.
- tools/grammar-eval: deepseek-scratchpad.gbnf (PLAN/NOTE×0-15/VERDICT)
and subset-bench.py wired for the 5-condition harness.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related improvements after [@noonghunna]'s observation that the
ladder we just added (commit 9560efd) was structurally OK but easy to
miss:
1. **FAQ ladder gets an at-a-glance table** at the top of
"Before symptom-matching" section. Five rows, one per step, showing
variant name + what each step adds + what it tests. Plus a one-line
path-finder ("if single-card, run 1-3; if dual, layer-by-layer
tells you which intersection breaks"). Readers no longer have to
scroll through 80 lines before they see the full ladder shape.
2. **Bug report template leads with the ladder** instead of jumping
straight to the report.sh ask. New intro:
"Before filing — try the 5-step triage ladder first"
Links into the FAQ section anchor. Acknowledges that "a lot of
'should I file a bug' questions resolve at step 1 or 2 (often
re-running setup.sh is the fix)." Worth 15 min before opening an
issue. The report.sh paste field stays — for users who've done the
ladder and have a real bug to file.
3. **Issue chooser config gets a third option** above
"General Q&A / discussion":
"Troubleshooting — try the 5-step triage ladder first"
Clicking "Open an issue" → users now see Troubleshooting / Q&A /
Bug-report / Bench-contribution as four distinct paths, with the
ladder explicitly named as a self-help option BEFORE the bug-report
template appears.
Result: users with budget / boot / MTP-class issues get pointed at
the ladder twice (once in the chooser, once in the bug template intro)
before they ever fill out the form. Reduces the "filed a bug that
turned out to be a partial-pull / setup-not-rerun / config-too-tight
issue" pattern that's eaten the last few triage rounds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bug report and numbers-from-your-rig issue templates previously asked for
6+ separate fields each (docker logs, verify-full, nvidia-smi, GPU config,
compose variant, commit, etc.). The new scripts/report.sh captures all
of that in one paste-ready dump, plus the data we kept asking individually
(power caps + default vs current, NVLink topology, OS, system RAM, idle
GPU VRAM detection, container runtime versions, cached vLLM image SHAs).
Templates restructured to lead with `bash scripts/report.sh > my-rig.md`:
- bug-report.yml: 6 required + 2 optional fields → 3 required (what
happened, repro, rig report) + 1 optional (extras / fallback). Net -38%
lines, but more importantly: one command instead of six manual asks.
- numbers-from-your-rig.yml: 5 required + 2 optional fields → 1 required
(rig + bench report via `--bench`) + 1 optional (notes / fallback). Net
-38% lines.
Both templates retain manual-fallback guidance for users who can't run
report.sh (no shell access, different rig environment, etc.).
CONTRIBUTING.md "Numbers from your rig" + "Bug reports" entries updated
to reference report.sh as the primary path. README.md repo layout adds
report.sh to scripts/ list, plus a one-liner in the docs-and-extras
paragraph pointing affected users at the script.
Net effect: future cross-rig contributors share more standardized data
with less effort. Triage threads stop bouncing on "could you also send
me X?" follow-ups for the surface report.sh covers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- setup.sh: GENESIS_PIN now defaults to commit bf667c7 (Genesis HEAD as of
2026-04-27, semver "v7.54"). This is the exact tree our published TPS
numbers were measured against; tagged v7.51-stable was one minor older
but came up first because the SHA isn't durable. Switch to commit pin
removes the doc-vs-runtime mismatch. Clone strategy adjusted since
--branch + --depth 1 doesn't accept SHAs.
- .env.example: documents MODEL_DIR / HF_TOKEN / CUDA_VISIBLE_DEVICES /
MEM_UTIL / MAX_MODEL_LEN / GENESIS_PIN / SKIP_GENESIS / URL / WARMUPS /
RUNS with the same defaults the composes ship. Pure opt-in.
- .github/ISSUE_TEMPLATE/: bug-report.yml requires docker logs --tail 100,
verify-full.sh output, nvidia-smi, GPU config, compose variant, repo
commit. numbers-from-your-rig.yml structures cross-rig TPS contributions
with rig spec, bench output, VRAM, max ctx, and notes. config.yml
routes Q&A to Discussions.
- .gitignore: drop trailing slash on genesis pattern so it also ignores
local symlinks that some of us point at out-of-tree clones.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
