0bdcb69fa3
Closes the "dual variants not yet re-benched on v0.20" caveat from PR #23. Confirms no v0.20 regression on fp8 / FP16 paths — code TPS within bench variance of chart values across all 3. n=5 measured + 3 warmup per prompt: | Variant | Chart (prior) narr / code | v0.20 measured narr / code | Δ | |--------------------|----------------------------|------------------------------|-------------------------| | dual.yml | 69.05 / 88.58 | 68.61 / 90.71 (CV 1.8% both) | narr -0.6% / code +2.4% | | dual-dflash.yml | 81.94 / 124.93 | 77.12 / 125.97 (CV 2-4%) | narr -5.9% / code +0.8% | | dual-dflash-noviz | 78.19 / 126.99 | 78.94 / 123.18 (CV 2-3%) | narr +1.0% / code -3.0% | dual-dflash narrative is the only delta outside CV (-5.9%); could be substrate (different driver / power state at chart capture) or a minor DFlash N=5 spec-decode regression on v0.20. Chart value stays — within ±5pp of measured, within bench noise band. Both dual.yml and dual-dflash* are "Genesis-less by design" (zero Genesis env vars). The migration only bumped the image SHA on these — no env-var changes — which is consistent with the flat result. The +50% TPS jump on TQ k8v4 we reported to Sander in discussion #19 came from enabling his full PROD env-var stack on the TQ KV path; fp8 / FP16 dual paths don't share the same patches and don't exhibit a similar bump. Updated `docs/DUAL_CARD.md` performance summary table with the new measured numbers + a per-variant Δ column. Per-config summaries written to `results/v0.20-migration/dual-{yml,dflash,dflash-noviz}.summary`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
27 lines
686 B
Plaintext
27 lines
686 B
Plaintext
variant: dual-dflash-noviz.yml
|
|
date: 2026-05-01 PM
|
|
context: 200000
|
|
gpu_mem_util: tbd-from-compose
|
|
kv_dtype: auto (FP16 forced by DFlash)
|
|
tp: 2
|
|
spec_decode: dflash_n5
|
|
language_model_only: true
|
|
narr_wall_tps: 78.94
|
|
narr_wall_cv: 3.3
|
|
narr_decode_tps: 79.80
|
|
code_wall_tps: 123.18
|
|
code_wall_cv: 2.8
|
|
code_decode_tps: 126.54
|
|
ttft_narr_ms: 136
|
|
ttft_code_ms: 145
|
|
dflash_al: 4.14-4.38
|
|
dflash_avg_accept: 0.629-0.676
|
|
chart_value_narr: 78.19
|
|
chart_value_code: 126.99
|
|
delta_vs_chart_narr: +1.0%
|
|
delta_vs_chart_code: -3.0%
|
|
notes: |
|
|
No env-var changes vs dev205. Image bump only. Both narr + code within
|
|
bench variance of chart values. Confirms no v0.20 regression on the
|
|
text-only DFlash path.
|