docs(benchmarks): K8V4 result + P2P-CNS finding on lucebox-hub dual-GPU (#229)

K8V4 same-card bench (-ctk q8_0 -ctv q4_0): 74.68 mean tok/s vs 73.97
default KV — basically identical (+1%). KV-format optimization doesn't
help at HumanEval-scale (<150-tok prompts × 128-tok gen) where the KV
pool isn't the bottleneck. Asymmetric quant available via PR #56/#54
(merged 2026-04-28 in lucebox-hub).

P2P-CNS finding (more important): the dual-GPU split bench from
cb089e1 ran on a chipset that reports "Chipset Not Supported" for
GPU↔GPU peer access (PHB topology, common consumer-board limitation).
The lucebox-hub split path requires CUDA P2P for direct draft-feature
transfers; without it, falls back to host-staging copies (CPU↔GPU
bouncing). The +1.7% we observed is therefore NOT a fair test of the
split's value — it's measuring same-card vs same-card-with-host-staging
overhead.

@weicj's published 51.86 tok/s on dual 2080 Ti 22GB (PR #80) presumably
ran with P2P available. Our negative result on PHB-only consumer boards
is rig-specific, not a refutation of the technique. Updated BENCHMARKS
row to reflect this honestly.

Setup gotcha now documented in BENCHMARKS: check `nvidia-smi topo -p2p r`
before configuring --target-gpu / --draft-gpu. NVLink-bonded setups
would typically expose P2P (cross-rig confirmation needed on lucebox
specifically; @JusefPol's NVLink win was on vLLM TP=2, not lucebox).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
noonghunna
2026-05-04 14:49:26 +00:00
parent 030f780f24
commit e78eaa1148
2 changed files with 29 additions and 2 deletions

View File

@@ -0,0 +1,24 @@
[bench] target = /mnt/models/gguf/qwen3.5-27b/Qwen3.5-27B-Q4_K_M.gguf
[bench] draft = /mnt/models/huggingface/qwen3.5-27b-dflash/model.safetensors
[bench] bin = /opt/ai/lucebox-hub/dflash/build/test_dflash
[bench] tmp = /tmp/dflash_bench
[bench] skipping tokenize (reusing /tmp/dflash_bench)
[bench] mode=fast n_gen=128
prompt steps AL pct% tok/s
--------------------------------------------------------------
has_close_elements 28 4.57 28.8 55.54
separate_paren_groups 20 6.40 40.9 75.72
truncate_number 22 5.82 38.1 70.09
below_zero 18 7.11 45.1 82.68
mean_absolute_deviation 24 5.33 33.3 63.61
intersperse 28 4.57 29.2 54.61
parse_nested_parens 13 9.85 63.5 109.09
filter_by_substring 21 6.10 38.4 72.09
sum_product 16 8.00 55.5 91.51
rolling_max 21 6.10 41.4 71.83
--------------------------------------------------------------
MEAN 6.38 41.4 74.68
commit/step range: 4.57 - 9.85
tok/s range: 54.6 - 109.1