concurrency-probe: success = tokens generated, not non-empty content

Found testing vibethinker-3b (a reasoning model): with a small max_tokens
it spends the budget mid-<think> and returns HTTP 200 with 256 tokens but
EMPTY content (never emits the final answer). The old content-based check
wrongly flagged those streams as silent-empty failures. For a KV-pool
stress test the stream DID run (generated tokens, held KV) -> ok =
completion_tokens > 0; a true silent-empty is HTTP 200 with ZERO tokens
(what soak-test means). After the fix vibethinker sustains N=8 clean on a
3090 (8x the compose default, 0 post-warm growth).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
This commit is contained in:
noonghunna
2026-07-05 05:10:29 +00:00
parent 65c150d562
commit dcf10b6cdc

View File

@@ -83,9 +83,13 @@ def one(stream, rnd):
try:
r = json.load(urllib.request.urlopen(req, timeout=600))
toks = (r.get("usage") or {}).get("completion_tokens", 0)
content = (r.get("choices") or [{}])[0].get("message",{}).get("content","") or ""
ok = toks > 0 and len(content.strip()) > 0
return {"ok": ok, "toks": toks, "silent": toks == 0 or not content.strip(),
# KV-pool stress semantics: a stream "ran" if it GENERATED tokens
# (held KV, decoded) — content-emptiness is irrelevant here (reasoning
# models legitimately return empty content when max_tokens truncates
# mid-<think>). silent-empty = HTTP 200 but ZERO tokens (the real
# failure soak-test flags).
ok = toks > 0
return {"ok": ok, "toks": toks, "silent": toks == 0,
"err": None, "dt": time.time()-t0}
except Exception as e:
return {"ok": False, "toks": 0, "silent": False, "err": str(e)[:80], "dt": time.time()-t0}