Promote the 35B-A3B MoE dual compose out of preview after full live validation
on 2× RTX 3090 (2026-05-30): max-ctx probe (262K fits, 7.81× concurrency, 2.05M-
token KV pool), bench (178/174 wall · 182 decode TPS, CV <0.5%), verify-stress
(NIAH-clean to 240K), soak-continuous (0 growth / 0 err / 0 silent / 100%
retention), quality --full (det 79/90 = 88%, aider 13/30 — ≥ ik-llama ref), and a
live vision smoke (read a test image correctly @ 262K).
- compose: preview.yml → fp8.yml (serving-stack filename per layout convention),
preview-mtp.yml → fp8-mtp.yml; fp8.yml = 262K + vision-on + no-MTP, vLLM v0.22.0
stable (no overlays), ✅ Production header.
- registry: slug vllm/qwen-a3b-preview → vllm/qwen-35b-a3b-dual, max_ctx
16384 → 262144, compose_path, DEFAULTS, kvcalc alias (kv-calc.py).
- profile_runtime capture re-synced to fp8.yml; test-pull slug updated.
- BENCHMARKS row + ADDING_MODELS pointer.
- MTP intentionally OFF: built-in MTP is net-negative on this MoE under vLLM TP=2
at BOTH n=2 (≈−45%) and n=3 (≈−51%) — the draft forward pays inter-GPU sync the
acceptance can't amortize (ik-llama avoids this single-card). fp8-mtp.yml kept as
the A/B reference only.
GATED: the registry-wide test-diagnose-profile fails for this entry because
kv-calc's qwen-MoE branch does NOT divide weights by TP (a deliberate hack:
inflated weights ≈ the vLLM-filled peak, which coincidentally passes at 16K but
starves the 262K KV in the fit-check). The config is empirically validated; this
is a kv-calc MoE-VRAM-model limitation. Lands AFTER the kv-calc fix PR (separate)
that distinguishes diagnose fit-need from the calibration filled-peak for cheap-KV
MoE. Calibration anchor left untouched here (the re-baseline belongs to that PR).
Co-authored-by: noonghunna <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
