feat(qwen3.6-27b): thinking OFF by default across all 21 composes
Previously, all 19 vLLM Qwen composes used --reasoning-parser qwen3 (parses
<think>...</think> output blocks) but did NOT explicitly disable the
thinking template. That meant Qwen3 default — thinking ON — applied
across the board. This burns hidden token budget on internal CoT for
every request, hurts latency, and creates a "Qwen looks much slower
than Gemma" gap on agentic benchmarks (which is exactly what we just
hit on aider-polyglot — Qwen exceeded the 1500s timeout, Gemma
finished in 19 min).
Change:
- All vLLM composes (18 of them) now pass `--chat-template-kwargs
'{"enable_thinking": false}'` after `--reasoning-parser qwen3`.
- llama.cpp single/docker-compose.yml: DISABLE_THINKING default flipped
0 → 1 (thinking now OFF by default; opt back in via DISABLE_THINKING=0).
- llama.cpp single/concurrent.yml: gained `--chat-template-kwargs` flag
with default `{"enable_thinking":false}` (overridable via
CHAT_TEMPLATE_KWARGS env).
NOT changed:
- bounded-thinking.yml — that's the structured-CoT compose where thinking
IS the feature. Reverted my initial blanket change for that one.
- qwopus-bf16mtp.yml — already had enable_thinking=false (preview compose).
Users who want thinking ON can:
- For vLLM: pass `chat_template_kwargs: {enable_thinking: true}` in the
per-request body (works fine).
- For llama.cpp: set `DISABLE_THINKING=0` in compose/.env.
Aligns with Gemma 4's "thinking off by default" (it ships that way upstream)
and removes the Qwen vs Gemma framework-bench skew.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This commit is contained in:
noonghunna
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
3909c2d6b8
commit
29d17ed82d
@@ -66,6 +66,7 @@ services:
|
||||
--cont-batching
|
||||
--jinja
|
||||
--reasoning-format ${REASONING_FORMAT:-none}
|
||||
--chat-template-kwargs ${CHAT_TEMPLATE_KWARGS:-{"enable_thinking":false}}
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
|
||||
@@ -60,7 +60,7 @@
|
||||
# Override to `auto` to get separate `reasoning_content` field
|
||||
# (Qwen3.6 thinking trace) — useful for clients that render
|
||||
# reasoning_content (most don't). Issue: club-3090#97.
|
||||
# DISABLE_THINKING set to 1 in .env to add `--chat-template-kwargs '{"enable_thinking":false}'`
|
||||
# DISABLE_THINKING default: 1 (thinking OFF). Set to 0 to opt INTO thinking
|
||||
# which forces empty <think></think> blocks in responses. Useful for
|
||||
# clients (e.g. opencode) that display <think> content as the response.
|
||||
# Tradeoff: applies to ALL clients on this server — Hermes/agents that
|
||||
@@ -92,7 +92,7 @@ services:
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
# DISABLE_THINKING=1 in compose/.env appends --chat-template-kwargs to disable
|
||||
# DISABLE_THINKING=1 (default) appends --chat-template-kwargs to disable
|
||||
# Qwen3 thinking server-side. Forces the chat template to insert empty
|
||||
# <think></think> blocks → output goes straight to the response. Useful for
|
||||
# clients (e.g. opencode) that display <think> content as the response.
|
||||
@@ -100,7 +100,7 @@ services:
|
||||
# use thinking lose reasoning capability. See docs/HARDWARE.md and disc club-3090#97.
|
||||
# Note: $$VAR is YAML-escape for $VAR (compose passes literal $ to bash).
|
||||
EXTRA_ARGS=()
|
||||
if [ "$${DISABLE_THINKING:-0}" = "1" ]; then
|
||||
if [ "$${DISABLE_THINKING:-1}" = "1" ]; then
|
||||
EXTRA_ARGS+=("--chat-template-kwargs" '{"enable_thinking":false}')
|
||||
echo "[entrypoint] DISABLE_THINKING=1 — chat template will produce empty <think></think>"
|
||||
fi
|
||||
|
||||
@@ -127,6 +127,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_xml
|
||||
|
||||
@@ -118,6 +118,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -141,6 +141,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -130,6 +130,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -154,6 +154,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -149,6 +149,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -257,6 +257,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -139,6 +139,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -237,6 +237,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -134,6 +134,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -138,6 +138,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -247,6 +247,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -357,6 +357,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -374,6 +374,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -274,6 +274,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -97,6 +97,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
@@ -157,6 +157,8 @@ services:
|
||||
- --trust-remote-code
|
||||
- --reasoning-parser
|
||||
- qwen3
|
||||
- --chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
- --enable-auto-tool-choice
|
||||
- --tool-call-parser
|
||||
- qwen3_coder
|
||||
|
||||
Reference in New Issue
Block a user