vllm/gemma-31b-qat-w4a16-dual A/B vs the autoround-int4 default (gemma-int8-mtp),
same int8-PTH KV + assistant MTP n=4 (byte-identical speculative-config):
- 8-pack 109/150 vs 105 (+4, within +/-5-7 noise = quality tie; real instructfollow
edge IF 15-vs-8 offset by hermes/cli/TC/RM).
- WEAKER spec-decode: MTP accept-len ~2.4 vs autoround's ~3.9 (per-position
acceptance lower at every position) -> ~30% lower TPS (71.6/87.8 @370W <
autoround 106/139 @230W despite MORE power; AL is the clean power-indep signal).
- Boots clean on stock vllm-gemma-stable (no #44494 workaround - tower arch).
Comparable quality but slower -> autoround-int4 stays the default; stays Experimental.
Updates the compose Quality field + registry status_note + a BENCHMARKS row.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
