80f75c0df2
NVFP4 runs on sub-sm_90 cards via the Marlin W4A16 weight-only fallback
since vLLM v0.24 (floor = capability 7.5, marlin_utils_fp4). Live-confirmed
on 2x RTX 3090 sm_86 2026-07-11: the shipped dual/nvfp4/mtp.yml boots at
262K + fp8 KV + MTP n=3 (accept 97%+), 69.7/85.5 decode TPS, and scores
8-pack think-off 110/150 — a statistical tie with the fp8 production
tier's 109. c3 hiding these slugs as "incompatible-hw" was factually wrong.
Semantic split — required_sm stays the NATIVE floor; new registry field
`fallback_sm` = the weight-only-fallback floor:
- kv-calc: in the band [fallback_sm, required_sm) the fit is PRICED
normally and annotated with `hw_fallback` {required_sm, card_sm, note}
instead of returning incompatible-hw (both --fit and --fit-all).
- c3: FitVerdict carries hw_fallback; fits-* + fallback -> "⚑" glyph,
VISIBLE by default. incompatible-hw hiding stays for true incompatibles.
- gates.py/compat.py: the hard SM floor becomes fallback_sm when present
(else required_sm, unchanged).
- Registry: fallback_sm=7.5 on the 4 NVFP4 slugs + status_note honesty
pass ("REFUSES on Ampere" -> fallback reality + measured numbers);
same corrections in the 4 compose headers.
- test-kv-calc-fit section (f) rewritten to the new contract.
On Ampere these slugs still carry the honest downside in the note: no
speed edge (~20% slower than the AutoRound tier for the same model) —
their sub-sm_90 value is models where NVFP4 is the only quant.
Guards green: kv-calc-fit, registry-disk, status-drift, switch/launch
parity, profiles-compat, pullgate, pull, registry-json/emit, diagnose,
launch-compat, default resolvers, kv-calc --calibration; c3 248 fast
tests + headless incompatible-hw hide tests.
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Co-authored-by: noonghunna <10742901+noonghunna@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
club-3090 serve cockpit (c3)
A lazydocker-style terminal UI for the club-3090 AI inference stack — the front door for
discover → serve → operate → validate on a consumer NVIDIA rig. Wraps the same registry, launchers,
and scripts you'd run by hand (switch.sh, launch.sh, gpu-mode, health.sh, the verify/bench
suite) behind one keyboard-driven screen.
Status: working — read paths (catalog, estate, containers, health) are live; write paths (serve / scene-switch / downloads) are guarded by a confirm + a single-writer reconcile lease.
What's inside
Two modes, shown as a tab bar; the producer mode is hidden in the lean view.
1Run & Operate (always shown)- Catalog — the full registry of model variants; filter, inspect, and serve one (
⏎). - Orchestration — live GPU cards, the
gpu-modescenes (incl.ai-studio), and supporting services; switch scene / stop. - Containers — running/stopped services with engine + port; drill into Logs / Top / Config, start a stopped one.
- Doctor — "is it serving correctly?" —
health.shlive +verify/verify-fullreads, basic/full reports, and the power-cap sweep.
- Catalog — the full registry of model variants; filter, inspect, and serve one (
2Bring & Validate (producer lane — hidden in lean view)- The add-a-model pipeline: ① Bring (fit-check an HF repo) → ② Serve (generate a compose + serve untested) → … → ⑤ Promote.
Launch shows both modes by default; c3 --lean (or [C] in-app) gives the consumer view
(Run & Operate only).
How to run
The cockpit depends on an in-repo sibling package, club3090-tui-core (tools/tui-core/), which
is not on PyPI — install both from the checkout.
With uv (recommended — one command; the local path is wired in pyproject.toml):
uv pip install -e tools/serve-cockpit
c3
With plain pip (install the core first, then the cockpit):
pip install -e tools/tui-core
pip install -e tools/serve-cockpit
c3
Either way the launch is c3 (or python -m club3090_cockpit). The app finds the repo root from its
own location; override with C3_REPO_ROOT=/path/to/club-3090 if you installed it elsewhere.
First run — set your Model Dir + HF token: press S to open Settings, set your Model Dir
(where weights download to) and your HuggingFace token (needed for gated / private repos), then
Ctrl+S to save (HF_HOME is auto-derived under the model dir). Then hit r to browse the catalog.
Keeping current: the cockpit moves fast — after a git pull, re-run the install
(uv pip install -e tools/serve-cockpit) to pick up new deps (e.g. PyYAML) and UI changes.
Keybindings
| Key | Action |
|---|---|
1 / 2 |
Run & Operate · Bring & Validate |
↑ ↓ ← → |
move within / between the tab bar and content |
⏎ |
primary action for the focused row (serve / start / download / confirm) |
k |
stop a service / cancel a download |
f |
force-start (experimental — skips the fit gate) |
r |
refresh the catalog (re-reads the registry) |
S |
settings — set Model Dir + HF token (Ctrl+S saves) |
N |
new pod — Operate · Orchestration: compose a model + GPU set (fit-checked, gated) |
Y |
copy the focused context to the clipboard |
. |
toggle the left rail (full-width content) |
C |
toggle lean view (hide / restore the Bring & Validate mode) |
? |
help · q quit |
Pods (multiple models on one host)
A pod = one model pinned to a chosen GPU set + port (an estate instance). The Operate · Orchestration tab shows a pod view — each pod with its GPUs stacked and a placement health badge (✓ placed / ⚠ PLACEMENT MISMATCH, so you see where models actually landed) — and [N] opens a New-pod modal (name · slug · GPU set, prefilled with the free GPUs). The create is fit-checked against the selected GPUs and routed through the confirm gate.
Same capability headless via the CLI: bash scripts/pod.sh create/list/status/up/down/rm (--json on reads). Full guide → docs/PODS.md.
Running tests
Fully headless — no TTY, GPU, Docker, or script calls (a conftest blocks any real spawn).
uv pip install -e "tools/serve-cockpit[dev]" # or: pip install pytest pytest-asyncio
cd tools/serve-cockpit && pytest