gemma-4-12b: add single-card vLLM INT8 compose (full 256K on one 3090)
New compose vllm/gemma-12b-int8 — Intel/gemma-4-12B-it-int8-AutoRound (W8A16) on
the stock gemma4-unified image, TP=1. Validated single 3090 sm_86 2026-06-04:
- int8 AutoRound loads on sm_86, coherent output
- KV pool ~433K tokens (gemma4 KV ~32 KB/tok) holds the FULL 262144 — NIAH
exact-recall 140K/170K/200K/230K/241K, 0 crashes
- bench 50.0/50.1 TPS narr/code, TTFT 65 ms, ~22.5 GB peak (one card)
- the high-fidelity single-card vLLM path (INT8 ≈ Q8); ≈ dual bf16's TPS, no TP-comm
Plumbs INT8 AutoRound as a first-class weight format end-to-end (it wasn't wired):
- engines/vllm-gemma4-unified.yml: + autoround to supported_weight_formats
- models/gemma-4-12b.yml: + autoround-int8 weights variant (size_gb 13)
- compose_registry.py: + vllm/gemma-12b-int8 entry (tp=1, 262144, port 8037)
- kv-calc.py: g12spec gets weights_int8_gb; gemma4-swa-dense selector + int8
branch; --weights-variant CLI gains int8; --compose alias gemma-single-int8
- compat.py: _kv_calc_weights_variant maps autoround-int8 -> int8 (was falling
to the bf16 fallback -> false 122%-budget FAIL)
- test-compose-registry-disk.sh: 56/57 -> 57/58
Also refreshes the BENCHMARKS Gemma-4-12B dual rows + section header, which still
described the p-RoPE overlay dropped in PR #309 (config fix supersedes).
Guard suite 41/41 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
