alex/club-3090
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

power-cap-sweep: sum delta.reasoning_content alongside delta.content

Browse Source 
Bench script's TPS counter was reading only delta.content from streaming
chat completions. When the server routes thinking tokens to
delta.reasoning_content (--reasoning-format auto, or extra_body
preserve_thinking), the counter saw 0 tokens for the full bench window
even though the GPU was generating fine — produced 0.00 TPS readings
across all caps with otherwise valid sampling data.

Surfaced in @laurimyllari's 4090 sweep (disc club-3090#62) — entire
38-cap decode-single sweep returned 0.00 TPS while sampler captured
correct power, SM clock, mem clock, throttle %, pstate. Same root cause
class as syangsao's opencode hang (#97): client-side parser only knows
about `content`, server routes to `reasoning_content`.

Fix: sum both fields' text length when present. Backwards-compatible —
when REASONING_FORMAT=none (default), reasoning_content is empty so
behavior is unchanged. Validated against running container: smoke test
shows 15 content chunks / 0 reasoning_content chunks at default config,
matching prior behavior. Reasoning-format=auto rigs would now produce
valid TPS readings instead of 0.00.

Applied to both decode-single (~line 416) and decode-concurrent (~538)
bench paths since both use the same SSE parsing pattern.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This commit is contained in:
noonghunna
2026-05-08 09:26:30 +00:00
co-authored by Claude Opus 4.7
parent 8f103f33ec
commit 71e5954ea9
1 changed files with 14 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/power-cap-sweep.sh
+14 -2
View File
@@ -413,7 +413,13 @@ try:
text = ""
delta = choice.get("delta")
if isinstance(delta, dict):
text = delta.get("content") or ""
# Sum BOTH content and reasoning_content fields. Some llama.cpp
# configs (--reasoning-format auto, or chat completions with
# preserve_thinking) route thinking tokens to reasoning_content
# instead of content. Counting only content silently produces
# 0 TPS readings even though the GPU is generating fine. See
# disc club-3090#62 (laurimyllari 4090 sweep, 2026-05-08).
text = (delta.get("content") or "") + (delta.get("reasoning_content") or "")
if not text:
text = choice.get("text") or ""
if text:
@@ -535,7 +541,13 @@ try:
text = ""
delta = choice.get("delta")
if isinstance(delta, dict):
text = delta.get("content") or ""
# Sum BOTH content and reasoning_content fields. Some llama.cpp
# configs (--reasoning-format auto, or chat completions with
# preserve_thinking) route thinking tokens to reasoning_content
# instead of content. Counting only content silently produces
# 0 TPS readings even though the GPU is generating fine. See
# disc club-3090#62 (laurimyllari 4090 sweep, 2026-05-08).
text = (delta.get("content") or "") + (delta.get("reasoning_content") or "")
if not text:
text = choice.get("text") or ""
if text: