* concurrency-probe: surface aggregate tok/s in verdict, RESULT, and sweep
The probe already computed aggregate throughput (summed completion
tokens / wall) per round but only printed it in the round table — the
verdict, RESULT line, and sweep knee reported per-stream only. Users
asking "what total throughput at N agents?" (the Bruno question) had
no first-party number to point at.
- verdict: aggregate tok/s (N streams) next to per-stream
- RESULT: agg_tps= field (machine-readable, sweep-parseable)
- SWEEP: per-N summary table (per-stream vs aggregate) + aggregate in
the knee line
Knee logic unchanged (still per-stream-floor based). Offline guard
test-concurrency-probe passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
* concurrency-probe: re-resolve served model after each SWEEP boot
The top-of-script MODEL autodetect runs before SWEEP boots anything on
$URL, so it silently falls back to the default (qwen3.6-27b) — and every
request against any other model 404s, reading as errors=N per round (hit
live sweeping vllm/qwen-35b-a3b-dual on :8051; the 27B sweep only worked
because the fallback equals its served name). Re-resolve from /v1/models
after each boot; an explicit caller MODEL= pin is respected (MODEL_PINNED).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
* FAQ + BENCHMARKS: first-party concurrency/aggregate-throughput matrix
Measured via the extended concurrency-probe (this PR) on 2x3090:
- dense 27B dual-fast: batching knee N=2 (~104 decode-agg @16K ctx),
collapses to ~54 by N=8; generation shape 211 agg @ N=8
- 35B-A3B MoE dual: ~250-270 decode-agg FLAT N=2..16 @16K ctx;
generation shape 1,037 tok/s aggregate @ N=16 (92.8/stream, clean)
New FAQ entry: how to serve multiple coding agents (MAX_NUM_SEQS,
ctx-vs-concurrency, model pick by architecture, honest aggregate-vs-
per-stream caveats, prefill-bound note, probe invocation).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
---------
Co-authored-by: noonghunna <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
