alex/club-3090
1
0
Fork 0
Code Issues Pull Requests Actions 38 Packages Projects Releases Wiki Activity

Merge pull request #592 from noonghunna/record/multi-fast-4card-quality

Browse Source 
Record multi-fast measured 4-card quality (108/150, @ryanmpelletier #584)
This commit is contained in:
noonghunna
2026-07-06 14:43:22 +05:00
committed by GitHub
parent d0e878c2f1 cbe544a3c6
commit fa83277b9e
2 changed files with 9 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
models/qwen3.6-27b/vllm/compose/multi4/autoround-int4/mtp.yml
View File

@@ -11,8 +11,9 @@
# Quality: = the dual fast proxy (vllm/dual): 109/150 (--full, current harness 2026-06-07);
# 3-way A/B fast 109 · balanced 105 · max 110 = TIE. (Earlier 129/150 on the
# 2026-05-09 harness is not comparable — verifier fixes since.) Quality is
# TP-invariant (same weights/KV/sampling as vllm/dual, only TP differs); a
# 4-card 8-pack confirmation is the open follow-up (report.sh runs no 8-pack).
# TP-invariant (same weights/KV/sampling as vllm/dual, only TP differs) — and
# now MEASURED on real 4-card hardware: @ryanmpelletier ran the full 8-pack
# thinking-off = 108/150 (#584, 2026-07-06), within ±1 of the 2-card carry.
# Best for: 4-card Qwen "fast" tier (vllm/qwen-27b-multi-fast) — AutoRound int4 + fp8 KV + MTP, TP=4 ⭐
# ---------------------------------------------------------------------------
# Quad RTX 3090 — Qwen3.6-27B "fast" tier at TP=4 (vllm/qwen-27b-multi-fast).

9
scripts/lib/profiles/baselines.yml
View File

@@ -368,13 +368,16 @@ baselines:
# clean to 240K · soak-continuous PASS (0 err / 0 silent / <200 MiB) ·
# bench n=5 CV 4.2%/3.0%. KV pool 1.77M tok / 6.77× @262K (vs 2-card
# vllm/dual 622K/2.37×) — the 4-card win is concurrency, not single-stream
# speed (decode ~flat, TP=4 all-reduce over PCIe). TPS-only: report.sh
# runs no 8-pack; quality is TP-invariant, carried from the vllm/dual
# proxy (109/150). ⚠ ceiling margin thin (851 MB @240K on driver 595, #149).
# speed (decode ~flat, TP=4 all-reduce over PCIe). ⚠ ceiling margin thin
# (851 MB @240K on driver 595, #149). QUALITY confirmed 2026-07-06 (#584
# follow-up): full 8-pack thinking-off 108/150 — within ±1 of the carried
# vllm/dual proxy (109), so TP-invariance is now MEASURED, not assumed.
narr_tps: 74.76
code_tps: 90.83
ttft_ms: 141
prefill_tps: { 10k: 1288, 90k: 1175 }
quality_8pk: "108/150"
quality_env: { harness: "benchlocal-cli @ club-3090 087d094 (sandboxes rebuilt 2026-07-06)" }
ctx_validated: { tokens: 240633, niah: "clean@240K" }
date: 2026-07-05
engine_pin: "vllm/vllm-openai:v0.24.0"