Repoint the single-card Gemma-4-26B-A4B slug in place from the bf16/16K
AWQ+MTP path to an INT8-per-token-head long-context path. Same weights
(cyankiwi AWQ-4bit, Marlin WNA16 MoE), same external MTP drafter (n=4),
same gemma4 tool-call; only the KV format changes (bf16 -> int8_per_token_head)
via the vendored vLLM PR #40391 hybrid-SWA KV page-size overlay. The dual
(vllm/gemma-26ba4b-dual) is untouched.
INT8-PTH (1 byte/token) lifts the single-card ceiling from bf16 16K to a
176K default. Live-validated on 1x RTX 3090 (2026-06-06): #40391 applies
cleanly, int8_per_token_head KV inits, Marlin WNA16 MoE backend, MTP
SpeculativeConfig active; KV pool 183,357 tok >= 176K; coherent generation
(post cudagraph-warmup) + clean gemma4 tool-call. mem_util 0.94, not 0.96:
0.96 passed the upfront KV check but OOM'd in the drafter's later cudagraph
capture (240 MiB free, needed 256) on a single 24 GB card.
- engine vllm-gemma-stable: + gemma4-swa-moe family, + awq/compressed-tensors
weight formats
- patches.yml: + gemma-a4b-vllm-pr40391-rebased (model-scoped copy of the
patch dir; reaches the compose, passes test-patch-attribution)
- compose single/awq/mtp.yml -> single/awq/int8.yml (full rewrite to the
#40391 overlay + entrypoint + int8 KV; Status: Experimental)
- registry + profile_runtime: engine/kv/max_ctx/mem_util/compose_path/entrypoint
- test-generate-from-profile: repoint the clean-derived-seed fixture off this
slug (now overlay-carrying) to vllm/gemma-26ba4b-dual + a tp= override
Status stays Experimental; rebench-full + soak are the #464 follow-up. Full
gate suite green (test-submit-bench pre-existing/environmental, fails on
baseline too).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
