docs(benchmarks): K8V4 result + P2P-CNS finding on lucebox-hub dual-GPU (#229)
K8V4 same-card bench (-ctk q8_0 -ctv q4_0): 74.68 mean tok/s vs 73.97
default KV — basically identical (+1%). KV-format optimization doesn't
help at HumanEval-scale (<150-tok prompts × 128-tok gen) where the KV
pool isn't the bottleneck. Asymmetric quant available via PR #56/#54
(merged 2026-04-28 in lucebox-hub).
P2P-CNS finding (more important): the dual-GPU split bench from
cb089e1 ran on a chipset that reports "Chipset Not Supported" for
GPU↔GPU peer access (PHB topology, common consumer-board limitation).
The lucebox-hub split path requires CUDA P2P for direct draft-feature
transfers; without it, falls back to host-staging copies (CPU↔GPU
bouncing). The +1.7% we observed is therefore NOT a fair test of the
split's value — it's measuring same-card vs same-card-with-host-staging
overhead.
@weicj's published 51.86 tok/s on dual 2080 Ti 22GB (PR #80) presumably
ran with P2P available. Our negative result on PHB-only consumer boards
is rig-specific, not a refutation of the technique. Updated BENCHMARKS
row to reflect this honestly.
Setup gotcha now documented in BENCHMARKS: check `nvidia-smi topo -p2p r`
before configuring --target-gpu / --draft-gpu. NVLink-bonded setups
would typically expose P2P (cross-rig confirmation needed on lucebox
specifically; @JusefPol's NVLink win was on vLLM TP=2, not lucebox).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
24
results/lucebox-k8v4-20260504-144647/phase-c-k8v4.log
Normal file
24
results/lucebox-k8v4-20260504-144647/phase-c-k8v4.log
Normal file
@@ -0,0 +1,24 @@
|
||||
[bench] target = /mnt/models/gguf/qwen3.5-27b/Qwen3.5-27B-Q4_K_M.gguf
|
||||
[bench] draft = /mnt/models/huggingface/qwen3.5-27b-dflash/model.safetensors
|
||||
[bench] bin = /opt/ai/lucebox-hub/dflash/build/test_dflash
|
||||
[bench] tmp = /tmp/dflash_bench
|
||||
[bench] skipping tokenize (reusing /tmp/dflash_bench)
|
||||
|
||||
[bench] mode=fast n_gen=128
|
||||
prompt steps AL pct% tok/s
|
||||
--------------------------------------------------------------
|
||||
has_close_elements 28 4.57 28.8 55.54
|
||||
separate_paren_groups 20 6.40 40.9 75.72
|
||||
truncate_number 22 5.82 38.1 70.09
|
||||
below_zero 18 7.11 45.1 82.68
|
||||
mean_absolute_deviation 24 5.33 33.3 63.61
|
||||
intersperse 28 4.57 29.2 54.61
|
||||
parse_nested_parens 13 9.85 63.5 109.09
|
||||
filter_by_substring 21 6.10 38.4 72.09
|
||||
sum_product 16 8.00 55.5 91.51
|
||||
rolling_max 21 6.10 41.4 71.83
|
||||
--------------------------------------------------------------
|
||||
MEAN 6.38 41.4 74.68
|
||||
|
||||
commit/step range: 4.57 - 9.85
|
||||
tok/s range: 54.6 - 109.1
|
||||
Reference in New Issue
Block a user