alex/club-3090
1
0
Fork 0
Code Issues Pull Requests Actions 38 Packages Projects Releases Wiki Activity

tess-dual-mtp: default REASONING_BUDGET=16384 (inert while thinking off)

Browse Source 
Caps the think phase so reasoning can't consume the entire completion
budget when a user enables REASONING=on — the token_limit truncation
class @seanyourhighness eliminated in #665 (2 -> 0 on his 4090 run,
think-on 8-pack 125/150). Inert under the shipped thinking-off default.
Expected to mitigate the streaming-toolcall+thinking caveat (same
mechanism); first-party think-on validation running now — Quality line
update follows when it lands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
This commit is contained in:
noonghunna
2026-07-11 14:43:10 +00:00
parent 2d7c1d06d7
commit e3f6f02356
1 changed files with 9 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
models/tess-4-27b/llama-cpp/compose/dual/migtissera-q4km/mtp.yml
View File

@@ -64,6 +64,14 @@
# SERVED_NAME --alias value (default: tess-4-27b)
# REASONING thinking gate (default: off — stack-wide policy)
# REASONING_FORMAT reasoning routing (default: deepseek — hygiene)
# REASONING_BUDGET think-token cap (default: 16384; inert while
# REASONING=off). Caps the think phase so long reasoning
# can't eat the whole completion budget — kills the
# token_limit truncation class on think-ON runs
# (@seanyourhighness #665, cross-rig 4090; expected to
# mitigate the streaming-toolcall+thinking ⚠️ caveat —
# first-party think-ON validation in flight 2026-07-11).
# -1 = unrestricted (the raw llama.cpp default).
# REPEAT_PENALTY repetition penalty (default: 1.0 — validated neutral)
# DRY_MULTIPLIER DRY sampler strength (default: 0.0 = OFF). For severe long-ctx
# loops set 0.8 (strongest loop-breaker; opt-in, #517).
@@ -111,6 +119,7 @@ services:
--jinja
--reasoning ${REASONING:-off}
--reasoning-format ${REASONING_FORMAT:-deepseek}
--reasoning-budget ${REASONING_BUDGET:-16384}
--temp ${TEMP:-${TEMPERATURE:-0.6}}
--top-p ${TOP_P:-0.95}
--top-k ${TOP_K:-20}