Rename vllm/gemma-int8→gemma-int8-mtp + vllm/gemma-mtp→gemma-bf16-mtp (both carry MTP n=4; only KV format differs); kv-calc alias fix. Ladder gemma-bf16-mtp default 32K→131K (measured BF16 KV pool 196,527 tok @ 0.95; verify-stress 8/8, NIAH→120K, bench 118.8/154.3, zero TPS regression). Codify the dual-card max-context-priority rule in DUAL_CARD.md. Suite 41/41.
Every gemma-4-31b vLLM compose pinned a purged Docker Hub nightly
(e47c98ef / bf610c2f), un-bootable for fresh users (#250, #167 class).
Prune the 9-variant set to 3 and repoint survivors onto the immutable
stable release tag vllm/vllm-openai:v0.21.0 (never :latest).
Survivors: vllm/gemma-mtp (bf16 dual, default), vllm/gemma-int8 (int8-PTH
dual), vllm/gemma-mtp-tp1 (fp8 single). Dropped: gemma-dflash,
gemma-dflash-int8, gemma-int8-tq3, gemma-bf16, gemma-awq; the 262K path
folds into gemma-int8 via a CTX env override. Gemma is decoupled from the
shared Qwen nightly profiles via a new vllm-gemma-stable engine profile;
Qwen nightly profiles are untouched.
Diagnose decoupling: the pruned gemma dflash patch dirs doubled as the
diagnose tool's cross-model disk-source proxies for two Qwen overlays
(vllm-pr41703-dflash, vllm-pr42102-dflash-kv-quant). Those capabilities
are image-baked in the pinned nightly, not mountable files, so mark them
image_baked and have diagnose skip the disk-source check for image-baked
overlays. Runtime-neutral (no Qwen compose mounts them).
switch.sh launcher: handle the new VLLM_IMAGE engine-pin export (the twin
loop in launch.sh had it; switch.sh did not -> "unexpected engine pin
export"), and guard the VLLM_NIGHTLY_SHA echo for the image-only stable
profile (was unbound under set -u).
Validated on 2x RTX 3090 (Ampere, v0.21.0): full scripts/tests/*.sh suite
35/0; live boots of gemma-mtp (bf16) and gemma-int8 serve coherent output,
qwen vllm/dual regression-clean. gemma-mtp-tp1 (fp8_e4m3) is correctly
SM-gated (required_sm=9.0) -- confirmed Ampere-incompatible (Triton
"fp8e4nv not supported on sm_86"), a documented 32 GB+/Hopper variant;
beellama remains the single-card gemma path on Ampere.
Refs #451, #250, #167.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
STEP 3: scripts/generate-compose.sh (+ scripts/lib/generate_compose.py).
Implements the brief's steps 1-10: scope gates first (type!=vllm /
genesis_equipped -> clean refuse), engine-pin loads:true validation
(image NEVER rewritten), arch via model_slugs/arch_model_xref, tp/kv
validation, trc {true,unverified} security refusal, compose-keyed patch
selection, delivery-gap-before-drift-guard, graded drift-guard
(capability-scoped -> OMIT+DEGRADED+--accept-degraded, foundational ->
hard-refuse, never repair, never wire a failed patch). Emits from the
captured compose_service_template: param-slots/constants verbatim, image
expression passed through verbatim, --trust-remote-code never emitted
in-scope (governed slot, locked §88), wiring re-derived only at the two
named insertion points, synthesizes nothing else. 3-category provenance
header above services: so STEP-2 service_body() discards it.
STEP 4: scripts/tests/test-generate-compose.sh. 5 golden triples
(vllm/minimal, vllm/dual, vllm/gemma-mtp, vllm/gemma-int8 [full,
multi-file overlay], vllm/gemma-dflash [dflash]) — all verified
genesis_equipped:false. Per triple: semantic diff vs shipped confined to
the two insertion points (image + constants verbatim), selected+wired
subset-of-shipped, wired pass reaches() on the GENERATED compose,
selected-but-undelivered NOT reachable, 3-category header, no
--trust-remote-code emitted. Plus the refusal/degraded matrix and
kv_arg() unit table. Imports (does not reimplement) patch_attribution.
test-patch-attribution.sh stays byte-identical (61 patch / 11 arch / 18
calibration, same 15 known-gap lines); all other test-*.sh remain RC=0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
