llama.cpp single: thinking-off policy alignment + MTP profile family
Closes the dataextract 0/15 quality regression caused by mounting the
vLLM-only froggeric Jinja template on the llama.cpp engine, which
silently suppressed --reasoning off. Native template + --reasoning off
restores it. Stack-wide Qwen3.6 thinking-off-by-default policy now
shipped on the llama.cpp path (matches 22/24 vLLM composes).
Adds three named single-card profiles:
- llamacpp/default (262K vanilla, unchanged) — cliff-immune fallback
- llamacpp/mtp (NEW: 131K + MTP n=2) — single-card workhorse, ~60 TPS code
- llamacpp/mtp-vision (NEW: 49K + MTP + vision) — first stack profile
combining MTP + vision on build 9235 (the older "strip mmproj when
MTP" rule was obsolete; sweep-verified MTP + vision coexist)
Retires single/concurrent.yml — single-card concurrency is anti-value
(per-slot ctx ~48K, worse than every other profile). Concurrency
belongs on dual.
Fixes a latent user-facing bug: engines/llama-cpp-mainline.yml had
supported_drafters: [draft-mtp] (the llama.cpp CLI flag value) while
the compat-layer C7 gate compares against drafter spec_method (mtp).
Net effect pre-fix: llamacpp/mtp would have been silently filtered
out of launch.sh candidate lists. Fixed [draft-mtp] -> [mtp].
Lowers -ub default 2048 -> 1024 on the llama.cpp single composes. The
per-pass activation peak halves; verify-stress goes 5/7 -> 7/7
including the 60K + 91K needle rungs previously treated as
architectural Cliff 2 territory. Cliff 2 single-prompt at 50-60K was
config-driven on llama.cpp, not architectural; CLIFFS.md note added
(vLLM Cliff 2 narrative unchanged — different kernel-level failure).
Migrates 6 stale call sites for the engine-id rename
(llama-cpp-mainline -> llama-cpp-local) that earlier work lagged.
Measured impact, Config A (llamacpp/mtp, 131K, MTP n=2, ub=1024):
- bench: 51.28 narr / 59.72 code decode TPS (n=3, CV 1.9%/0.5%)
- verify-stress: 7/7 PASS (incl. 60K + 91K needle recall)
- quality 8-pack: 102/150 (68%) — beats every Qwen vLLM dual config in
discussion #119 by 6-16 pp
- aider-polyglot-30: 17/30 (56.7%) — matches Qwen vLLM bf16 dual
exactly (17/30) on half the hardware
- per-GPU code TPS (59.72) ~equal to vLLM dual configs (60-63),
confirming the engine-side per-card rate is identical and vLLM
dual's aggregate advantage is purely from the second card
Config B (llamacpp/mtp-vision, 49K, MTP + vision):
- multimodal vision probe passed
- bench: 56.52 narr / 66.17 code decode TPS (n=5)
- verify-stress: 7/7 PASS
Tests green: test-launch-compat, test-profiles-compat,
test-switch-registry-parity, test-pullgate-gates, test-patch-attribution.
Leak-grep clean. YAML lint clean. Performance charts regenerated to
include the two new entries.
Doc alignment: SINGLE_CARD, CLIFFS, FAQ, EXAMPLES, README, llama-cpp
README, COMPOSE_GENERATOR, PULL_GATE all reflect the new family.
CHANGELOG.md + UPSTREAM.md left intact per append-only history rule.
Followups (queued, not blocking): #400 MTP retune A/B (n=4 + p-min=0
on the fixed config), #403 soak-test container glob (unlocks Cliff 2b
validation on llama.cpp), #405 benchlocal-cli port-offset for parallel
rebench, #401 + #404 ik_llama + Tom's TurboQuant trackers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ed1507122c