From 71e5954ea987ece658ecfa1f232ccf3506a41fbf Mon Sep 17 00:00:00 2001 From: noonghunna <10742901+noonghunna@users.noreply.github.com> Date: Fri, 8 May 2026 09:26:30 +0000 Subject: [PATCH] power-cap-sweep: sum delta.reasoning_content alongside delta.content MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bench script's TPS counter was reading only delta.content from streaming chat completions. When the server routes thinking tokens to delta.reasoning_content (--reasoning-format auto, or extra_body preserve_thinking), the counter saw 0 tokens for the full bench window even though the GPU was generating fine — produced 0.00 TPS readings across all caps with otherwise valid sampling data. Surfaced in @laurimyllari's 4090 sweep (disc club-3090#62) — entire 38-cap decode-single sweep returned 0.00 TPS while sampler captured correct power, SM clock, mem clock, throttle %, pstate. Same root cause class as syangsao's opencode hang (#97): client-side parser only knows about `content`, server routes to `reasoning_content`. Fix: sum both fields' text length when present. Backwards-compatible — when REASONING_FORMAT=none (default), reasoning_content is empty so behavior is unchanged. Validated against running container: smoke test shows 15 content chunks / 0 reasoning_content chunks at default config, matching prior behavior. Reasoning-format=auto rigs would now produce valid TPS readings instead of 0.00. Applied to both decode-single (~line 416) and decode-concurrent (~538) bench paths since both use the same SSE parsing pattern. Co-Authored-By: Claude Opus 4.7 (1M context) --- scripts/power-cap-sweep.sh | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/scripts/power-cap-sweep.sh b/scripts/power-cap-sweep.sh index 2ef9484b..b1aa8a41 100755 --- a/scripts/power-cap-sweep.sh +++ b/scripts/power-cap-sweep.sh @@ -413,7 +413,13 @@ try: text = "" delta = choice.get("delta") if isinstance(delta, dict): - text = delta.get("content") or "" + # Sum BOTH content and reasoning_content fields. Some llama.cpp + # configs (--reasoning-format auto, or chat completions with + # preserve_thinking) route thinking tokens to reasoning_content + # instead of content. Counting only content silently produces + # 0 TPS readings even though the GPU is generating fine. See + # disc club-3090#62 (laurimyllari 4090 sweep, 2026-05-08). + text = (delta.get("content") or "") + (delta.get("reasoning_content") or "") if not text: text = choice.get("text") or "" if text: @@ -535,7 +541,13 @@ try: text = "" delta = choice.get("delta") if isinstance(delta, dict): - text = delta.get("content") or "" + # Sum BOTH content and reasoning_content fields. Some llama.cpp + # configs (--reasoning-format auto, or chat completions with + # preserve_thinking) route thinking tokens to reasoning_content + # instead of content. Counting only content silently produces + # 0 TPS readings even though the GPU is generating fine. See + # disc club-3090#62 (laurimyllari 4090 sweep, 2026-05-08). + text = (delta.get("content") or "") + (delta.get("reasoning_content") or "") if not text: text = choice.get("text") or "" if text: