nvidia/Qwen3.6-27B-NVFP4 (modelopt v0.45 MIXED_PRECISION: NVFP4 gs16 FFN +
FP8-static attention + FP8 KV scales baked + unquantized mtp.* head, 21.9 GB,
full VLM) — the first catalog tier this dev rig CANNOT run: NVIDIA supports
Hopper + Blackwell only, so both slugs are AUTHORED BLIND and gated
required_sm=9.0. The first community boot is the validation (funnel /
numbers-from-your-rig); compose headers carry the exact ask + known-unknowns
(MTP-on-modelopt, Cliff 2 on Blackwell, froggeric template).
- vllm/qwen-27b-single-nvfp4 (:8076, 🧪) — 131K default sized for the
smallest target (5090 32 GB: kv-calc 28.5/29.4 GB PASS @97%); GB10 128 GB
runs the FULL 262K at 29% budget (4 streams @262K = 50%) — the single-card
home; H100 @262K = 46%. MAX_MODEL_LEN env raises it.
- vllm/qwen-27b-dual-nvfp4 (:8077, 🧪) — TP=2 @262K full ctx, mirrors the
production dual-max shape with ~11 GB/card weights (vs 14.5 FP8) → 2x 5090
primary target; kv-calc 22.1 GB/card (75% budget).
Wiring, per ADDING_MODELS + the compat model:
- weights entry (format=modelopt, hf_repo, 22 GB) + weights.py aliases
- engines/vllm-stable.yml supported_weight_formats += modelopt (stock vLLM
supports --quantization modelopt; the FP4 KERNELS need sm>=9.0 which is
encoded per-slug via required_sm, not at the engine — loader is
arch-agnostic)
- kv-calc: weights_nvfp4_gb in the qwen spec, nvfp4 branch in
_weights_per_card_gb, weights_variant pass-through for qwen3.6-27b compose
cfgs, nvfp4-single/nvfp4-dual aliases, CLI --weights-variant choice
- registry: both entries kvcalc_key-wired (NOT SKIP — kv-calc predicts fit
on hardware we don't own, which is the point), no DEFAULTS rows
- KV is fp8_e4m3 NOT nvfp4-KV (consumer Blackwell has no FP4 FMHA — see
hardware/rtx-5090.yml + vllm#43562); this checkpoint bakes FP8 KV scales
Gates: full scripts/tests suite green (registry-disk counts 59/60,
profiles-compat + diagnose-profile fit on the 5090 canonical scenarios),
kv-calc --calibration green. Honest refusal verified on this rig: C3
(sm >= 9 floor) + C5 (fp8_e4m3 KV) reject on rtx-3090; switch.sh --list
shows both as (NA: experimental).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
34b1797568