docs(benchmarks): add lucebox-hub DFlash dual-GPU bench — no-op on 24 GB cards (#229)
First cross-engine bench on this stack against the lucebox-hub DFlash dual-GPU split (PR #80 from @weicj, merged 2026-05-04). Test: HumanEval 10 prompts × n_gen=128 on Qwen3.5-27B Q4_K_M target + z-lab DFlash draft. Result: dual-GPU split is essentially a no-op on 24 GB / 3090. Same-card baseline (target + draft on GPU 0): 73.97 tok/s mean (range 52.7–108.7) Dual-GPU split (--target-gpu 0 --draft-gpu 1): 75.24 tok/s mean (range 54.2–110.0) Delta: +1.7% (within run-to-run noise) AL (6.39) and accept (41.3%) identical between phases — same draft, same prompts. Why this matches expectation in retrospect: PR #80's split frees draft VRAM from competing with target activation budget. On 22 GB cards (where @weicj measured 51.86 tok/s) target Q4 (~16 GB) + draft (~3.5 GB) + KV genuinely fight for the budget. On 24 GB cards there's already 4 GB headroom, so the bottleneck the split addresses isn't binding. Peer-copy overhead from --draft-feature-mirror is roughly canceled by the freed budget. Implication: don't ship lucebox-hub dual-GPU as a 24 GB-class default. The 51.86 tok/s on dual 2080 Ti 22GB result still holds for tighter-VRAM Ampere. PFlash phase-split (PR #78) is a separate, untested question — that's about long-context prefill compression (24K → 262K passing NIAH), not decode TPS. Tracked at task #230 if pursued. Today's DFlash-decode result doesn't refute or confirm the PFlash long-context story. Adds: - BENCHMARKS.md "Cross-engine — Luce DFlash (lucebox-hub) on Qwen3.5-27B" subsection with both phases' numbers and the calibrated framing - results/lucebox-dual-gpu-20260504-142832/{phase-a,phase-b}.log raw output Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -103,6 +103,17 @@ Not TPS, but load-bearing. Every shipped variant is validated against:
|
||||
|
||||
The single-card vLLM Cliff 2b status is canonicalized in [#41](https://github.com/noonghunna/club-3090/issues/41) — fix is gated on upstream [Sandermage genesis-vllm-patches#19](https://github.com/Sandermage/genesis-vllm-patches/issues/19). See [docs/CLIFFS.md](docs/CLIFFS.md) for the byte-level explanation.
|
||||
|
||||
### Cross-engine — Luce DFlash (lucebox-hub) on Qwen3.5-27B
|
||||
|
||||
Not directly comparable to vLLM rows above (different engine, different bench script, different model — Qwen3.5-27B not 3.6 because the 3.6 DFlash draft is still under training as of 2026-05-04). Bench harness: `lucebox-hub/dflash/scripts/bench_he.py`, HumanEval 10 prompts, n_gen=128.
|
||||
|
||||
| Config | Rig | Mean tok/s | AL | Accept % | Notes |
|
||||
|---|---|---:|---:|---:|---|
|
||||
| Same-card (target + draft both on GPU 0) | @noonghunna (1× 3090) | 73.97 | 6.39 | 41.3% | Range 52.7–108.7 across 10 HE prompts. AL is the strongest for this draft on this model class. Bench 2026-05-04. |
|
||||
| **Dual-GPU split** ([PR #80](https://github.com/Luce-Org/lucebox-hub/pull/80) `--target-gpu 0 --draft-gpu 1 --draft-feature-mirror`) | @noonghunna (2× 3090, no NVLink) | 75.24 | 6.39 | 41.3% | Range 54.2–110.0. **+1.7% over same-card on 24 GB cards** — essentially noise. The split's value is freeing draft from competing with target activation budget; on 24 GB / 3090 there's already 4 GB of headroom so the bottleneck the split addresses isn't binding. **PR #80's 51.86 tok/s on dual 2080 Ti 22GB result still holds** for tighter-VRAM Ampere — that hardware class genuinely needs the split. |
|
||||
|
||||
PFlash phase-split ([PR #78](https://github.com/Luce-Org/lucebox-hub/pull/78)) is a separate question — that's about long-context prefill compression (24K → 262K passing NIAH source ctx claimed), not decode TPS. Not benched in this run; tracked at task #230 if pursued.
|
||||
|
||||
---
|
||||
|
||||
## See also
|
||||
|
||||
Reference in New Issue
Block a user