From cb089e1e36c3cc37db45da630331d0a5bfa10215 Mon Sep 17 00:00:00 2001
From: noonghunna <10742901+noonghunna@users.noreply.github.com>
Date: Mon, 4 May 2026 14:33:28 +0000
Subject: [PATCH] =?UTF-8?q?docs(benchmarks):=20add=20lucebox-hub=20DFlash?=
 =?UTF-8?q?=20dual-GPU=20bench=20=E2=80=94=20no-op=20on=2024=20GB=20cards?=
 =?UTF-8?q?=20(#229)?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

First cross-engine bench on this stack against the lucebox-hub DFlash dual-GPU
split (PR #80 from @weicj, merged 2026-05-04). Test: HumanEval 10 prompts ×
n_gen=128 on Qwen3.5-27B Q4_K_M target + z-lab DFlash draft.

Result: dual-GPU split is essentially a no-op on 24 GB / 3090.

  Same-card baseline (target + draft on GPU 0):  73.97 tok/s mean (range 52.7–108.7)
  Dual-GPU split (--target-gpu 0 --draft-gpu 1): 75.24 tok/s mean (range 54.2–110.0)
  Delta:                                          +1.7% (within run-to-run noise)

AL (6.39) and accept (41.3%) identical between phases — same draft, same prompts.

Why this matches expectation in retrospect: PR #80's split frees draft VRAM from
competing with target activation budget. On 22 GB cards (where @weicj measured
51.86 tok/s) target Q4 (~16 GB) + draft (~3.5 GB) + KV genuinely fight for the
budget. On 24 GB cards there's already 4 GB headroom, so the bottleneck the split
addresses isn't binding. Peer-copy overhead from --draft-feature-mirror is
roughly canceled by the freed budget.

Implication: don't ship lucebox-hub dual-GPU as a 24 GB-class default. The
51.86 tok/s on dual 2080 Ti 22GB result still holds for tighter-VRAM Ampere.

PFlash phase-split (PR #78) is a separate, untested question — that's about
long-context prefill compression (24K → 262K passing NIAH), not decode TPS.
Tracked at task #230 if pursued. Today's DFlash-decode result doesn't refute
or confirm the PFlash long-context story.

Adds:
- BENCHMARKS.md "Cross-engine — Luce DFlash (lucebox-hub) on Qwen3.5-27B"
  subsection with both phases' numbers and the calibrated framing
- results/lucebox-dual-gpu-20260504-142832/{phase-a,phase-b}.log raw output

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---
 BENCHMARKS.md | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/BENCHMARKS.md b/BENCHMARKS.md
index 540f139b..e4869c7b 100644
--- a/BENCHMARKS.md
+++ b/BENCHMARKS.md
@@ -103,6 +103,17 @@ Not TPS, but load-bearing. Every shipped variant is validated against:
 
 The single-card vLLM Cliff 2b status is canonicalized in [#41](https://github.com/noonghunna/club-3090/issues/41) — fix is gated on upstream [Sandermage genesis-vllm-patches#19](https://github.com/Sandermage/genesis-vllm-patches/issues/19). See [docs/CLIFFS.md](docs/CLIFFS.md) for the byte-level explanation.
 
+### Cross-engine — Luce DFlash (lucebox-hub) on Qwen3.5-27B
+
+Not directly comparable to vLLM rows above (different engine, different bench script, different model — Qwen3.5-27B not 3.6 because the 3.6 DFlash draft is still under training as of 2026-05-04). Bench harness: `lucebox-hub/dflash/scripts/bench_he.py`, HumanEval 10 prompts, n_gen=128.
+
+| Config | Rig | Mean tok/s | AL | Accept % | Notes |
+|---|---|---:|---:|---:|---|
+| Same-card (target + draft both on GPU 0) | @noonghunna (1× 3090) | 73.97 | 6.39 | 41.3% | Range 52.7–108.7 across 10 HE prompts. AL is the strongest for this draft on this model class. Bench 2026-05-04. |
+| **Dual-GPU split** ([PR #80](https://github.com/Luce-Org/lucebox-hub/pull/80) `--target-gpu 0 --draft-gpu 1 --draft-feature-mirror`) | @noonghunna (2× 3090, no NVLink) | 75.24 | 6.39 | 41.3% | Range 54.2–110.0. **+1.7% over same-card on 24 GB cards** — essentially noise. The split's value is freeing draft from competing with target activation budget; on 24 GB / 3090 there's already 4 GB of headroom so the bottleneck the split addresses isn't binding. **PR #80's 51.86 tok/s on dual 2080 Ti 22GB result still holds** for tighter-VRAM Ampere — that hardware class genuinely needs the split. |
+
+PFlash phase-split ([PR #78](https://github.com/Luce-Org/lucebox-hub/pull/78)) is a separate question — that's about long-context prefill compression (24K → 262K passing NIAH source ctx claimed), not decode TPS. Not benched in this run; tracked at task #230 if pursued.
+
 ---
 
 ## See also