Gemma-4-31B onto stock vLLM v0.24.0, overlay-free on vllm-stable — the
first slice of the gemma consolidation (Option A1).
- New dual/qat-awq-int4/int8.yml: cyankiwi QAT-AWQ-INT4 weights (lm_head
excluded → dodges the v0.24.0 tie_weights crash that autoround hits),
int8-PTH KV @262K, util 0.965 (bf16 lm_head margin), native gemma4
tool+reasoning parsers. Overlay-free: #40391 (KV page-align) is native
in v0.24.0; #42006 N/A.
- MTP DISABLED with a documented re-enable caveat: Gemma-4 MTP x tool-calling
is broken on v0.24.0 (upstream vLLM #39043; MTP fix#42006 closed-unmerged).
A/B-proven on-rig: MTP-off tools PASS, MTP-on tools FAIL. Re-enable when a
stable vLLM ships the fix.
- Registry: vllm/gemma-31b-dual on vllm-stable; qat-awq-int4 weights map.
- vllm-stable.features.int8_per_token_head -> true: #40391 is native in
v0.24.0, so gemma int8-PTH no longer needs the overlay-carrying engine.
Live-validated dual on stock v0.24.0: 267K KV pool, coherent, streaming
multi-tool 3/3 (all args, no leak). Suite green (registry/compat/diagnose).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
