K8V4 same-card bench (-ctk q8_0 -ctv q4_0): 74.68 mean tok/s vs 73.97
default KV — basically identical (+1%). KV-format optimization doesn't
help at HumanEval-scale (<150-tok prompts × 128-tok gen) where the KV
pool isn't the bottleneck. Asymmetric quant available via PR #56/#54
(merged 2026-04-28 in lucebox-hub).
P2P-CNS finding (more important): the dual-GPU split bench from
cb089e1 ran on a chipset that reports "Chipset Not Supported" for
GPU↔GPU peer access (PHB topology, common consumer-board limitation).
The lucebox-hub split path requires CUDA P2P for direct draft-feature
transfers; without it, falls back to host-staging copies (CPU↔GPU
bouncing). The +1.7% we observed is therefore NOT a fair test of the
split's value — it's measuring same-card vs same-card-with-host-staging
overhead.
@weicj's published 51.86 tok/s on dual 2080 Ti 22GB (PR #80) presumably
ran with P2P available. Our negative result on PHB-only consumer boards
is rig-specific, not a refutation of the technique. Updated BENCHMARKS
row to reflect this honestly.
Setup gotcha now documented in BENCHMARKS: check `nvidia-smi topo -p2p r`
before configuring --target-gpu / --draft-gpu. NVLink-bonded setups
would typically expose P2P (cross-rig confirmation needed on lucebox
specifically; @JusefPol's NVLink win was on vLLM TP=2, not lucebox).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
e78eaa1148