docs(benchmarks): PFlash long-context bench — 131K source ceiling on 1× 3090 (#230)
Closes task #230. Measured PFlash NIAH compression at 16K-260K source contexts on 1× 24 GB / 3090 single-card. Result: PFlash works flawlessly up to 131K source. Compresses 131,068 tokens to 6,524 (5%) in 10.8s with NIAH key + answer both retained. Vanilla llama.cpp pp131072 takes ~257s per Luce's published numbers, so PFlash alone is ~24× faster at this context. End-to-end TTFT (PFlash + target prefill on 6.5K) would be ~12-13s vs ~257s = ~20×. Above 131K, drafter ephemeral forward-pass tensors (K_curr/V_curr/Q_last at full sequence length) exceed 24 GB. K-cache quantization (--pflash-k-type q8_0) doesn't help — the failing allocs are forward-pass not cache, confirmed by separate bench at 200K/260K with identical OOM at the same layer numbers. @weicj's PR #78 claim of 24K → 262K dual-GPU phase split is neither refuted nor reproduced. Their setup was 2× 22 GB Ti with target also loaded co-resident on one card; the "24K" was target+drafter combined. Our 131K is drafter-alone on 24 GB. Reproducing 262K specifically would require investigation of their drafter config (chunk_size, lookahead, BSA window) — drafter activation footprint at 200K+ is the binding constraint regardless of GPU count. Practical recommendation for 24 GB / 3090 single-card users: PFlash is shippable for source contexts ≤ 131K. The ~24× TTFT speedup is genuine and quality holds. Above 131K, fall back to vanilla llama.cpp prefill or wait for upstream drafter optimizations. Adds: - BENCHMARKS.md "PFlash long-context compression on 1× 3090" subsection with full per-context table + drafter ceiling explanation - results/lucebox-pflash-niah-20260504-150321/ (BF16 K cache run) - results/lucebox-pflash-niah-q8k-20260504-150600/ (q8_0 K cache run) This closes our active investigation of the Luce surface — three benches done (DFlash same-card 73.97 mean, K8V4 same-card 74.68 mean, PFlash compression ceiling 131K). Recommendation surface narrows to: PFlash at ≤131K is the one piece of Luce that beats vLLM dual.yml on TTFT for that workload class. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
noonghunna
@@ -113,7 +113,26 @@ Not directly comparable to vLLM rows above (different engine, different bench sc
|
||||
| Same-card, **K8V4** (`-ctk q8_0 -ctv q4_0`) | @noonghunna (1× 3090) | 74.68 | 6.38 | 41.4% | Range 54.6–109.1. **+1% over default KV** — basically identical. KV-format optimization doesn't help at HE-scale (<150-tok prompts × 128-tok gen) where KV pool isn't the bottleneck. Asymmetric quant via [PR #56/#54](https://github.com/Luce-Org/lucebox-hub/pull/56) merged 2026-04-28. |
|
||||
| Dual-GPU split ([PR #80](https://github.com/Luce-Org/lucebox-hub/pull/80) `--target-gpu 0 --draft-gpu 1 --draft-feature-mirror`) | @noonghunna (2× 3090, no NVLink, **P2P "Chipset Not Supported"**) | 75.24 | 6.39 | 41.3% | Range 54.2–110.0. **+1.7% over same-card** — but **NOT a fair test of the split's value**. CUDA P2P access is disabled at the chipset level on this rig (PHB topology, consumer-board limitation). The lucebox dual-GPU code path requires P2P for direct draft-feature transfers; without it, falls back to host-staging copies (CPU↔GPU bouncing). The published 51.86 tok/s on dual 2080 Ti 22GB ([PR #80](https://github.com/Luce-Org/lucebox-hub/pull/80)) presumably ran with P2P available. **Verdict for our hardware class**: dual-GPU split needs a P2P-capable interconnect (NVLink or peer-supported chipset) to deliver its value. PHB+CNS rigs see no benefit. |
|
||||
|
||||
PFlash phase-split ([PR #78](https://github.com/Luce-Org/lucebox-hub/pull/78)) is a separate question — that's about long-context prefill compression (24K → 262K passing NIAH source ctx claimed), not decode TPS. Not benched in this run; tracked at task #230 if pursued.
|
||||
### PFlash long-context compression on 1× 3090 — measured ceiling 131K source
|
||||
|
||||
Bench harness: `lucebox-hub/dflash/scripts/phase_split_dual_gpu.py bench-niah` (PFlash drafter only, no target loaded — measures the prefill compression phase). Drafter: `Qwen3-0.6B-BF16.gguf`, BSA enabled, keep_ratio=0.05.
|
||||
|
||||
| Source ctx | Compressed | Ratio | PFlash time | tok/s | Key + answer retained |
|
||||
|---:|---:|---:|---:|---:|:---:|
|
||||
| 16,372 | 788 | 0.048 | 1.08 s | 15,117 | ✓ ✓ |
|
||||
| 32,764 | 1,628 | 0.050 | 1.80 s | 18,205 | ✓ ✓ |
|
||||
| 65,524 | 3,252 | 0.050 | 4.37 s | 15,009 | ✓ ✓ |
|
||||
| **131,068** | **6,524** | **0.050** | **10.80 s** | **12,135** | **✓ ✓** |
|
||||
| 199,996 | OOM at layer 25 (390 MiB ephemeral alloc) | — | — | — | ✗ ✗ |
|
||||
| 259,996 | OOM at layer 18 (507 MiB ephemeral alloc) | — | — | — | ✗ ✗ |
|
||||
|
||||
**Headline**: PFlash works flawlessly up to **131K source on 1× 24 GB / 3090** — compresses to 6.5K (5%) in 10.8s with NIAH key + answer retained. Vanilla llama.cpp pp131072 takes ~257s per Luce's published numbers, so PFlash compression alone is **~24× faster** at this context. Adding target prefill on the compressed 6.5K would be ~1-2s, total ~12-13s end-to-end TTFT — matches the README claim of "~25s @ 128K, 10× speedup" within hardware variance.
|
||||
|
||||
Above 131K source the drafter's **ephemeral forward-pass tensors** (`K_curr/V_curr/Q_last` per layer at full sequence length) exceed 24 GB. K-cache quantization (`--pflash-k-type q8_0`) didn't help — the failing allocs are forward-pass not cache. Bench `lucebox-pflash-niah-q8k-20260504-150600/` confirmed identical OOM at 200K and 260K with both BF16 and q8_0 K cache.
|
||||
|
||||
**On the @weicj 24K → 262K phase-split claim** ([PR #78](https://github.com/Luce-Org/lucebox-hub/pull/78)): not refuted but not reproduced on our hardware class either — their setup was 2× 22 GB Ti with **target also loaded** on one card; "24K single-card" was target+drafter co-resident. Our 131K is drafter-alone on 24 GB, which already passes their dual-GPU 262K-style scaling sanity-check. Reproducing 262K specifically would need investigation of their drafter config (chunk_size, lookahead, BSA window) — drafter activation footprint at 200K+ is the binding constraint regardless of how many GPUs are present.
|
||||
|
||||
**Practical recommendation for 24 GB / 3090 single-card**: PFlash is shippable for source contexts ≤ 131K. The ~24× TTFT compression is genuine and the NIAH retention holds. Above 131K, fall back to vanilla llama.cpp prefill or wait for upstream drafter optimizations.
|
||||
|
||||
**Setup gotcha** for anyone re-running on consumer rigs: check `nvidia-smi topo -p2p r` before configuring `--target-gpu` / `--draft-gpu`. If the matrix shows `CNS` (Chipset Not Supported), the dual-GPU split won't deliver its claimed uplift on that hardware regardless of whether you have multiple GPUs. NVLink-bonded setups would also typically expose P2P (a different cross-rig contributor would need to confirm on lucebox specifically; @JusefPol's [#31](https://github.com/noonghunna/club-3090/pull/31) NVLink win was measured on vLLM TP=2, not lucebox). PHB-only consumer boards typically lack P2P.
|
||||
|
||||
|
||||
345
results/lucebox-pflash-niah-20260504-145351/gpu_monitor.csv
Normal file
345
results/lucebox-pflash-niah-20260504-145351/gpu_monitor.csv
Normal file
@@ -0,0 +1,345 @@
|
||||
ts,phase,index,temp_c,fan_pct,power_w,power_limit_w,mem_used_mib,mem_total_mib,util_pct
|
||||
1777906480.048,pflash_load,0,48,0,28.86,230.00,1,24576,0
|
||||
1777906480.048,pflash_load,1,53,0,22.93,230.00,1,24576,0
|
||||
1777906481.099,pflash_load,0,52,0,62.34,230.00,1700,24576,0
|
||||
1777906481.099,pflash_load,1,53,0,25.88,230.00,1,24576,0
|
||||
1777906482.142,pflash_load,0,52,0,128.56,230.00,1700,24576,0
|
||||
1777906482.142,pflash_load,1,53,0,26.76,230.00,1,24576,0
|
||||
1777906483.184,pflash_load,0,52,0,127.68,230.00,1700,24576,0
|
||||
1777906483.184,pflash_load,1,53,0,24.05,230.00,1,24576,0
|
||||
1777906484.226,pflash_load,0,53,0,127.82,230.00,1700,24576,0
|
||||
1777906484.226,pflash_load,1,53,0,23.36,230.00,1,24576,0
|
||||
1777906485.268,pflash_load,0,53,0,128.01,230.00,1700,24576,0
|
||||
1777906485.268,pflash_load,1,53,0,23.20,230.00,1,24576,0
|
||||
1777906486.310,pflash_load,0,53,57,128.18,230.00,1700,24576,0
|
||||
1777906486.310,pflash_load,1,53,0,23.31,230.00,1,24576,0
|
||||
1777906487.352,pflash_load,0,53,51,129.74,230.00,1700,24576,0
|
||||
1777906487.352,pflash_load,1,53,0,23.34,230.00,1,24576,0
|
||||
1777906488.407,pflash_load,0,53,53,130.16,230.00,1700,24576,0
|
||||
1777906488.407,pflash_load,1,52,0,23.22,230.00,1,24576,0
|
||||
1777906489.454,pflash_load,0,53,55,129.28,230.00,1700,24576,0
|
||||
1777906489.454,pflash_load,1,53,0,23.16,230.00,1,24576,0
|
||||
1777906490.496,pflash_load,0,53,57,129.52,230.00,1700,24576,0
|
||||
1777906490.496,pflash_load,1,53,0,23.53,230.00,1,24576,0
|
||||
1777906491.539,pflash_load,0,53,57,129.64,230.00,1700,24576,0
|
||||
1777906491.539,pflash_load,1,53,0,23.48,230.00,1,24576,0
|
||||
1777906492.580,pflash_load,0,54,58,129.72,230.00,1700,24576,0
|
||||
1777906492.580,pflash_load,1,52,0,23.42,230.00,1,24576,0
|
||||
1777906493.622,pflash_load,0,54,58,129.79,230.00,1700,24576,0
|
||||
1777906493.622,pflash_load,1,52,0,23.36,230.00,1,24576,0
|
||||
1777906494.663,pflash_load,0,54,58,129.85,230.00,1700,24576,0
|
||||
1777906494.663,pflash_load,1,52,0,23.27,230.00,1,24576,0
|
||||
1777906495.706,pflash_load,0,54,58,129.95,230.00,1700,24576,0
|
||||
1777906495.706,pflash_load,1,52,0,23.16,230.00,1,24576,0
|
||||
1777906496.748,pflash_load,0,54,58,129.99,230.00,1700,24576,0
|
||||
1777906496.748,pflash_load,1,52,0,23.13,230.00,1,24576,0
|
||||
1777906497.791,pflash_load,0,54,59,129.97,230.00,1700,24576,0
|
||||
1777906497.791,pflash_load,1,52,0,23.31,230.00,1,24576,0
|
||||
1777906498.835,pflash_load,0,54,59,130.06,230.00,1700,24576,0
|
||||
1777906498.835,pflash_load,1,52,0,23.47,230.00,1,24576,0
|
||||
1777906499.877,pflash_load,0,54,59,130.10,230.00,1700,24576,0
|
||||
1777906499.877,pflash_load,1,52,0,23.37,230.00,1,24576,0
|
||||
1777906500.918,pflash_load,0,54,59,129.95,230.00,1,24576,0
|
||||
1777906500.918,pflash_load,1,52,0,23.31,230.00,1,24576,0
|
||||
1777906501.962,pflash_load,0,54,59,132.29,230.00,1,24576,0
|
||||
1777906501.962,pflash_load,1,52,0,23.21,230.00,1,24576,0
|
||||
1777906503.007,pflash_load,0,51,59,52.60,230.00,1,24576,0
|
||||
1777906503.007,pflash_load,1,52,0,22.91,230.00,1,24576,0
|
||||
1777906504.057,pflash_load,0,50,59,32.71,230.00,1,24576,0
|
||||
1777906504.057,pflash_load,1,52,0,22.86,230.00,1,24576,0
|
||||
1777906505.109,pflash_load,0,50,59,31.68,230.00,1,24576,0
|
||||
1777906505.109,pflash_load,1,52,0,22.98,230.00,1,24576,0
|
||||
1777906506.176,pflash_load,0,50,59,30.25,230.00,1,24576,0
|
||||
1777906506.176,pflash_load,1,52,0,23.00,230.00,1,24576,0
|
||||
1777906507.238,pflash_load,0,50,59,29.75,230.00,1,24576,0
|
||||
1777906507.238,pflash_load,1,52,0,22.97,230.00,1,24576,0
|
||||
1777906508.288,pflash_load,0,49,59,29.76,230.00,1,24576,0
|
||||
1777906508.288,pflash_load,1,52,0,22.87,230.00,1,24576,0
|
||||
1777906509.339,pflash_load,0,49,59,29.91,230.00,1,24576,0
|
||||
1777906509.339,pflash_load,1,52,0,22.95,230.00,1,24576,0
|
||||
1777906510.388,pflash_load,0,49,59,29.77,230.00,1,24576,0
|
||||
1777906510.388,pflash_load,1,52,0,23.15,230.00,1,24576,0
|
||||
1777906511.438,pflash_load,0,49,59,29.69,230.00,1,24576,0
|
||||
1777906511.438,pflash_load,1,52,0,23.05,230.00,1,24576,0
|
||||
1777906512.488,pflash_load,0,49,59,29.61,230.00,1,24576,0
|
||||
1777906512.488,pflash_load,1,52,0,23.06,230.00,1,24576,0
|
||||
1777906513.538,pflash_load,0,49,59,29.74,230.00,1,24576,0
|
||||
1777906513.538,pflash_load,1,52,0,23.10,230.00,1,24576,0
|
||||
1777906514.585,pflash_load,0,49,59,29.81,230.00,1,24576,0
|
||||
1777906514.585,pflash_load,1,52,0,22.93,230.00,1,24576,0
|
||||
1777906515.634,pflash_load,0,49,59,29.43,230.00,1,24576,0
|
||||
1777906515.634,pflash_load,1,52,0,22.90,230.00,1,24576,0
|
||||
1777906516.683,pflash_load,0,48,59,29.47,230.00,1,24576,0
|
||||
1777906516.683,pflash_load,1,52,0,23.08,230.00,1,24576,0
|
||||
1777906517.733,pflash_load,0,48,59,29.72,230.00,1,24576,0
|
||||
1777906517.733,pflash_load,1,52,0,23.15,230.00,1,24576,0
|
||||
1777906518.787,pflash_load,0,48,59,29.76,230.00,1,24576,0
|
||||
1777906518.787,pflash_load,1,52,0,23.12,230.00,1,24576,0
|
||||
1777906519.834,pflash_load,0,48,59,29.68,230.00,1,24576,0
|
||||
1777906519.834,pflash_load,1,52,0,23.20,230.00,1,24576,0
|
||||
1777906520.884,pflash_load,0,48,59,29.67,230.00,1,24576,0
|
||||
1777906520.884,pflash_load,1,52,0,23.02,230.00,1,24576,0
|
||||
1777906521.933,pflash_load,0,48,59,29.66,230.00,1,24576,0
|
||||
1777906521.933,pflash_load,1,52,0,22.85,230.00,1,24576,0
|
||||
1777906522.984,pflash_load,0,48,59,29.83,230.00,1,24576,0
|
||||
1777906522.984,pflash_load,1,52,0,22.95,230.00,1,24576,0
|
||||
1777906524.036,pflash_load,0,48,59,29.70,230.00,1,24576,0
|
||||
1777906524.036,pflash_load,1,52,0,22.90,230.00,1,24576,0
|
||||
1777906525.102,pflash_load,0,48,59,29.72,230.00,1,24576,0
|
||||
1777906525.102,pflash_load,1,52,0,22.82,230.00,1,24576,0
|
||||
1777906526.160,pflash_load,0,48,59,29.68,230.00,1,24576,0
|
||||
1777906526.160,pflash_load,1,52,0,22.88,230.00,1,24576,0
|
||||
1777906527.210,pflash_load,0,47,59,29.60,230.00,1,24576,0
|
||||
1777906527.210,pflash_load,1,52,0,22.77,230.00,1,24576,0
|
||||
1777906528.259,pflash_load,0,47,59,29.63,230.00,1,24576,0
|
||||
1777906528.259,pflash_load,1,52,0,22.74,230.00,1,24576,0
|
||||
1777906529.307,pflash_load,0,47,59,29.63,230.00,1,24576,0
|
||||
1777906529.307,pflash_load,1,52,0,22.78,230.00,1,24576,0
|
||||
1777906530.358,pflash_load,0,47,59,29.61,230.00,1,24576,0
|
||||
1777906530.358,pflash_load,1,52,0,22.95,230.00,1,24576,0
|
||||
1777906531.408,pflash_load,0,47,58,29.59,230.00,1,24576,0
|
||||
1777906531.408,pflash_load,1,52,0,22.83,230.00,1,24576,0
|
||||
1777906532.458,pflash_load,0,47,58,29.58,230.00,1,24576,0
|
||||
1777906532.458,pflash_load,1,52,0,22.59,230.00,1,24576,0
|
||||
1777906533.507,pflash_load,0,47,58,29.59,230.00,1,24576,0
|
||||
1777906533.507,pflash_load,1,52,0,22.78,230.00,1,24576,0
|
||||
1777906534.557,pflash_load,0,47,58,29.53,230.00,1,24576,0
|
||||
1777906534.557,pflash_load,1,52,0,22.69,230.00,1,24576,0
|
||||
1777906535.607,pflash_load,0,47,58,29.58,230.00,1,24576,0
|
||||
1777906535.607,pflash_load,1,52,0,22.60,230.00,1,24576,0
|
||||
1777906536.657,pflash_load,0,47,58,29.68,230.00,1,24576,0
|
||||
1777906536.657,pflash_load,1,52,0,22.83,230.00,1,24576,0
|
||||
1777906537.707,pflash_load,0,47,58,29.70,230.00,1,24576,0
|
||||
1777906537.707,pflash_load,1,52,0,22.84,230.00,1,24576,0
|
||||
1777906538.757,pflash_load,0,47,58,29.62,230.00,1,24576,0
|
||||
1777906538.757,pflash_load,1,52,0,22.89,230.00,1,24576,0
|
||||
1777906539.806,pflash_load,0,46,58,29.59,230.00,1,24576,0
|
||||
1777906539.806,pflash_load,1,52,0,23.07,230.00,1,24576,0
|
||||
1777906540.856,pflash_load,0,46,58,29.60,230.00,1,24576,0
|
||||
1777906540.856,pflash_load,1,52,0,22.84,230.00,1,24576,0
|
||||
1777906541.906,pflash_load,0,46,58,29.58,230.00,1,24576,0
|
||||
1777906541.906,pflash_load,1,52,0,22.67,230.00,1,24576,0
|
||||
1777906542.955,pflash_load,0,46,58,29.58,230.00,1,24576,0
|
||||
1777906542.955,pflash_load,1,52,0,22.70,230.00,1,24576,0
|
||||
1777906544.006,pflash_load,0,46,58,29.60,230.00,1,24576,0
|
||||
1777906544.006,pflash_load,1,52,0,22.60,230.00,1,24576,0
|
||||
1777906545.055,pflash_load,0,46,58,29.55,230.00,1,24576,0
|
||||
1777906545.055,pflash_load,1,52,0,22.52,230.00,1,24576,0
|
||||
1777906546.105,pflash_load,0,46,58,29.57,230.00,1,24576,0
|
||||
1777906546.105,pflash_load,1,52,0,22.57,230.00,1,24576,0
|
||||
1777906547.163,pflash_load,0,46,58,29.81,230.00,1,24576,0
|
||||
1777906547.163,pflash_load,1,52,0,22.54,230.00,1,24576,0
|
||||
1777906548.211,pflash_load,0,46,58,29.60,230.00,1,24576,0
|
||||
1777906548.211,pflash_load,1,52,0,22.46,230.00,1,24576,0
|
||||
1777906549.264,pflash_load,0,46,58,29.34,230.00,1,24576,0
|
||||
1777906549.264,pflash_load,1,52,0,22.59,230.00,1,24576,0
|
||||
1777906550.313,pflash_load,0,46,58,29.31,230.00,1,24576,0
|
||||
1777906550.313,pflash_load,1,52,0,22.76,230.00,1,24576,0
|
||||
1777906551.361,pflash_load,0,46,58,29.33,230.00,1,24576,0
|
||||
1777906551.361,pflash_load,1,52,0,22.85,230.00,1,24576,0
|
||||
1777906552.409,pflash_load,0,46,58,29.39,230.00,1,24576,0
|
||||
1777906552.409,pflash_load,1,52,0,22.53,230.00,1,24576,0
|
||||
1777906553.459,pflash_load,0,46,58,29.31,230.00,1,24576,0
|
||||
1777906553.459,pflash_load,1,52,0,22.40,230.00,1,24576,0
|
||||
1777906554.508,pflash_load,0,45,58,29.36,230.00,1,24576,0
|
||||
1777906554.508,pflash_load,1,52,0,22.68,230.00,1,24576,0
|
||||
1777906555.557,pflash_load,0,45,58,29.51,230.00,1,24576,0
|
||||
1777906555.557,pflash_load,1,52,0,22.66,230.00,1,24576,0
|
||||
1777906556.609,pflash_load,0,45,58,29.32,230.00,1,24576,0
|
||||
1777906556.609,pflash_load,1,52,0,22.43,230.00,1,24576,0
|
||||
1777906557.656,pflash_load,0,45,58,29.29,230.00,1,24576,0
|
||||
1777906557.656,pflash_load,1,51,0,22.53,230.00,1,24576,0
|
||||
1777906558.705,pflash_load,0,45,58,29.43,230.00,1,24576,0
|
||||
1777906558.705,pflash_load,1,51,0,22.52,230.00,1,24576,0
|
||||
1777906559.756,pflash_load,0,45,58,29.25,230.00,1,24576,0
|
||||
1777906559.756,pflash_load,1,51,0,22.50,230.00,1,24576,0
|
||||
1777906560.806,pflash_load,0,45,58,29.29,230.00,1,24576,0
|
||||
1777906560.806,pflash_load,1,51,0,22.55,230.00,1,24576,0
|
||||
1777906561.856,pflash_load,0,45,58,29.52,230.00,1,24576,0
|
||||
1777906561.856,pflash_load,1,51,0,22.46,230.00,1,24576,0
|
||||
1777906562.904,pflash_load,0,45,58,29.49,230.00,1,24576,0
|
||||
1777906562.904,pflash_load,1,51,0,22.61,230.00,1,24576,0
|
||||
1777906563.953,pflash_load,0,45,58,29.14,230.00,1,24576,0
|
||||
1777906563.953,pflash_load,1,51,0,22.70,230.00,1,24576,0
|
||||
1777906565.009,pflash_load,0,45,58,29.18,230.00,1,24576,0
|
||||
1777906565.009,pflash_load,1,51,0,22.50,230.00,1,24576,0
|
||||
1777906566.058,pflash_load,0,45,58,29.24,230.00,1,24576,0
|
||||
1777906566.058,pflash_load,1,51,0,22.42,230.00,1,24576,0
|
||||
1777906567.108,pflash_load,0,45,58,29.25,230.00,1,24576,0
|
||||
1777906567.108,pflash_load,1,51,0,22.56,230.00,1,24576,0
|
||||
1777906568.160,pflash_load,0,45,57,29.01,230.00,1,24576,0
|
||||
1777906568.160,pflash_load,1,51,0,22.61,230.00,1,24576,0
|
||||
1777906569.209,pflash_load,0,45,57,29.00,230.00,1,24576,0
|
||||
1777906569.209,pflash_load,1,51,0,22.49,230.00,1,24576,0
|
||||
1777906570.257,pflash_load,0,45,57,29.16,230.00,1,24576,0
|
||||
1777906570.257,pflash_load,1,51,0,22.45,230.00,1,24576,0
|
||||
1777906571.306,pflash_load,0,45,57,29.01,230.00,1,24576,0
|
||||
1777906571.306,pflash_load,1,51,0,22.55,230.00,1,24576,0
|
||||
1777906572.353,pflash_load,0,44,57,29.03,230.00,1,24576,0
|
||||
1777906572.353,pflash_load,1,51,0,22.71,230.00,1,24576,0
|
||||
1777906573.401,pflash_load,0,44,57,29.17,230.00,1,24576,0
|
||||
1777906573.401,pflash_load,1,51,0,22.70,230.00,1,24576,0
|
||||
1777906574.450,pflash_load,0,44,57,29.03,230.00,1,24576,0
|
||||
1777906574.450,pflash_load,1,51,0,22.61,230.00,1,24576,0
|
||||
1777906575.498,pflash_load,0,44,57,29.07,230.00,1,24576,0
|
||||
1777906575.498,pflash_load,1,51,0,22.62,230.00,1,24576,0
|
||||
1777906576.549,pflash_load,0,44,57,29.32,230.00,1,24576,0
|
||||
1777906576.549,pflash_load,1,51,0,22.79,230.00,1,24576,0
|
||||
1777906577.597,pflash_load,0,44,57,29.25,230.00,1,24576,0
|
||||
1777906577.597,pflash_load,1,51,0,22.97,230.00,1,24576,0
|
||||
1777906578.646,pflash_load,0,44,57,29.22,230.00,1,24576,0
|
||||
1777906578.646,pflash_load,1,51,0,22.86,230.00,1,24576,0
|
||||
1777906579.696,pflash_load,0,44,56,29.31,230.00,1,24576,0
|
||||
1777906579.696,pflash_load,1,51,0,22.65,230.00,1,24576,0
|
||||
1777906580.745,pflash_load,0,44,51,29.08,230.00,1,24576,0
|
||||
1777906580.745,pflash_load,1,51,0,22.70,230.00,1,24576,0
|
||||
1777906581.796,pflash_load,0,44,45,28.88,230.00,1,24576,0
|
||||
1777906581.796,pflash_load,1,51,0,22.82,230.00,1,24576,0
|
||||
1777906582.852,pflash_load,0,44,0,28.72,230.00,1,24576,0
|
||||
1777906582.852,pflash_load,1,51,0,22.94,230.00,1,24576,0
|
||||
1777906583.903,pflash_load,0,44,0,28.17,230.00,1,24576,0
|
||||
1777906583.903,pflash_load,1,51,0,23.00,230.00,1,24576,0
|
||||
1777906584.951,pflash_load,0,44,0,28.00,230.00,1,24576,0
|
||||
1777906584.951,pflash_load,1,51,0,22.82,230.00,1,24576,0
|
||||
1777906586.001,pflash_load,0,44,0,27.84,230.00,1,24576,0
|
||||
1777906586.001,pflash_load,1,51,0,22.83,230.00,1,24576,0
|
||||
1777906587.050,pflash_load,0,44,0,27.80,230.00,1,24576,0
|
||||
1777906587.050,pflash_load,1,51,0,22.98,230.00,1,24576,0
|
||||
1777906588.100,pflash_load,0,44,0,27.92,230.00,1,24576,0
|
||||
1777906588.100,pflash_load,1,51,0,22.77,230.00,1,24576,0
|
||||
1777906589.148,pflash_load,0,44,0,27.83,230.00,1,24576,0
|
||||
1777906589.148,pflash_load,1,51,0,22.63,230.00,1,24576,0
|
||||
1777906590.196,pflash_load,0,44,0,27.96,230.00,1,24576,0
|
||||
1777906590.196,pflash_load,1,51,0,22.70,230.00,1,24576,0
|
||||
1777906591.245,pflash_load,0,44,0,28.11,230.00,1,24576,0
|
||||
1777906591.245,pflash_load,1,51,0,22.63,230.00,1,24576,0
|
||||
1777906592.296,pflash_load,0,44,0,27.91,230.00,1,24576,0
|
||||
1777906592.296,pflash_load,1,51,0,22.74,230.00,1,24576,0
|
||||
1777906593.345,pflash_load,0,44,0,27.89,230.00,1,24576,0
|
||||
1777906593.345,pflash_load,1,51,0,22.82,230.00,1,24576,0
|
||||
1777906594.395,pflash_load,0,44,0,28.09,230.00,1,24576,0
|
||||
1777906594.395,pflash_load,1,51,0,22.88,230.00,1,24576,0
|
||||
1777906595.445,pflash_load,0,44,0,27.99,230.00,1,24576,0
|
||||
1777906595.445,pflash_load,1,51,0,22.94,230.00,1,24576,0
|
||||
1777906596.495,pflash_load,0,44,0,27.99,230.00,1,24576,0
|
||||
1777906596.495,pflash_load,1,51,0,22.82,230.00,1,24576,0
|
||||
1777906597.545,pflash_load,0,45,0,28.17,230.00,1,24576,0
|
||||
1777906597.545,pflash_load,1,51,0,22.75,230.00,1,24576,0
|
||||
1777906598.594,pflash_load,0,45,0,27.97,230.00,1,24576,0
|
||||
1777906598.594,pflash_load,1,51,0,22.75,230.00,1,24576,0
|
||||
1777906599.642,pflash_load,0,45,0,27.94,230.00,1,24576,0
|
||||
1777906599.642,pflash_load,1,51,0,22.60,230.00,1,24576,0
|
||||
1777906600.695,pflash_load,0,45,0,28.17,230.00,1,24576,0
|
||||
1777906600.695,pflash_load,1,51,0,22.45,230.00,1,24576,0
|
||||
1777906601.747,pflash_load,0,45,0,28.09,230.00,1,24576,0
|
||||
1777906601.747,pflash_load,1,51,0,22.55,230.00,1,24576,0
|
||||
1777906602.795,pflash_load,0,45,0,28.17,230.00,1,24576,0
|
||||
1777906602.795,pflash_load,1,51,0,22.50,230.00,1,24576,0
|
||||
1777906603.845,pflash_load,0,45,0,28.27,230.00,1,24576,0
|
||||
1777906603.845,pflash_load,1,51,0,22.42,230.00,1,24576,0
|
||||
1777906604.893,pflash_load,0,45,0,28.04,230.00,1,24576,0
|
||||
1777906604.893,pflash_load,1,51,0,22.60,230.00,1,24576,0
|
||||
1777906605.940,pflash_load,0,45,0,28.06,230.00,1,24576,0
|
||||
1777906605.940,pflash_load,1,51,0,22.83,230.00,1,24576,0
|
||||
1777906606.989,pflash_load,0,45,0,28.08,230.00,1,24576,0
|
||||
1777906606.989,pflash_load,1,51,0,22.83,230.00,1,24576,0
|
||||
1777906608.038,pflash_load,0,45,0,28.27,230.00,1,24576,0
|
||||
1777906608.038,pflash_load,1,51,0,22.70,230.00,1,24576,0
|
||||
1777906609.086,pflash_load,0,45,0,28.03,230.00,1,24576,0
|
||||
1777906609.086,pflash_load,1,51,0,22.66,230.00,1,24576,0
|
||||
1777906610.136,pflash_load,0,45,0,27.99,230.00,1,24576,0
|
||||
1777906610.136,pflash_load,1,51,0,22.78,230.00,1,24576,0
|
||||
1777906611.185,pflash_load,0,45,0,28.31,230.00,1,24576,0
|
||||
1777906611.185,pflash_load,1,51,0,22.81,230.00,1,24576,0
|
||||
1777906612.234,pflash_load,0,45,0,28.25,230.00,1,24576,0
|
||||
1777906612.234,pflash_load,1,51,0,22.60,230.00,1,24576,0
|
||||
1777906613.284,pflash_load,0,45,0,28.25,230.00,1,24576,0
|
||||
1777906613.284,pflash_load,1,51,0,22.57,230.00,1,24576,0
|
||||
1777906614.332,pflash_load,0,45,0,28.37,230.00,1,24576,0
|
||||
1777906614.332,pflash_load,1,51,0,22.59,230.00,1,24576,0
|
||||
1777906615.383,pflash_load,0,45,0,28.17,230.00,1,24576,0
|
||||
1777906615.383,pflash_load,1,51,0,22.78,230.00,1,24576,0
|
||||
1777906616.432,pflash_load,0,45,0,28.17,230.00,1,24576,0
|
||||
1777906616.432,pflash_load,1,51,0,22.84,230.00,1,24576,0
|
||||
1777906617.485,pflash_load,0,45,0,28.29,230.00,1,24576,0
|
||||
1777906617.485,pflash_load,1,51,0,22.82,230.00,1,24576,0
|
||||
1777906618.544,pflash_load,0,45,0,28.08,230.00,1,24576,0
|
||||
1777906618.544,pflash_load,1,51,0,22.84,230.00,1,24576,0
|
||||
1777906619.594,pflash_load,0,45,0,28.10,230.00,1,24576,0
|
||||
1777906619.594,pflash_load,1,51,0,22.64,230.00,1,24576,0
|
||||
1777906620.644,pflash_load,0,45,0,28.29,230.00,1,24576,0
|
||||
1777906620.644,pflash_load,1,51,0,22.60,230.00,1,24576,0
|
||||
1777906621.692,pflash_load,0,45,0,28.11,230.00,1,24576,0
|
||||
1777906621.692,pflash_load,1,51,0,22.68,230.00,1,24576,0
|
||||
1777906622.741,pflash_load,0,45,0,28.04,230.00,1,24576,0
|
||||
1777906622.741,pflash_load,1,51,0,22.53,230.00,1,24576,0
|
||||
1777906623.789,pflash_load,0,45,0,28.18,230.00,1,24576,0
|
||||
1777906623.789,pflash_load,1,51,0,22.50,230.00,1,24576,0
|
||||
1777906624.837,pflash_load,0,45,0,28.07,230.00,1,24576,0
|
||||
1777906624.837,pflash_load,1,51,0,22.60,230.00,1,24576,0
|
||||
1777906625.886,pflash_load,0,45,0,28.17,230.00,1,24576,0
|
||||
1777906625.886,pflash_load,1,51,0,22.63,230.00,1,24576,0
|
||||
1777906626.935,pflash_load,0,45,0,28.36,230.00,1,24576,0
|
||||
1777906626.935,pflash_load,1,51,0,22.84,230.00,1,24576,0
|
||||
1777906627.982,pflash_load,0,45,0,28.24,230.00,1,24576,0
|
||||
1777906627.982,pflash_load,1,51,0,22.97,230.00,1,24576,0
|
||||
1777906629.030,pflash_load,0,46,0,28.31,230.00,1,24576,0
|
||||
1777906629.030,pflash_load,1,51,0,22.74,230.00,1,24576,0
|
||||
1777906630.080,pflash_load,0,46,0,28.18,230.00,1,24576,0
|
||||
1777906630.080,pflash_load,1,51,0,22.61,230.00,1,24576,0
|
||||
1777906631.129,pflash_load,0,46,0,28.22,230.00,1,24576,0
|
||||
1777906631.129,pflash_load,1,51,0,22.70,230.00,1,24576,0
|
||||
1777906632.178,pflash_load,0,46,0,28.39,230.00,1,24576,0
|
||||
1777906632.178,pflash_load,1,51,0,22.55,230.00,1,24576,0
|
||||
1777906633.228,pflash_load,0,46,0,28.26,230.00,1,24576,0
|
||||
1777906633.228,pflash_load,1,51,0,22.63,230.00,1,24576,0
|
||||
1777906634.278,pflash_load,0,46,0,28.21,230.00,1,24576,0
|
||||
1777906634.278,pflash_load,1,51,0,22.88,230.00,1,24576,0
|
||||
1777906635.327,pflash_load,0,46,0,28.24,230.00,1,24576,0
|
||||
1777906635.327,pflash_load,1,51,0,22.74,230.00,1,24576,0
|
||||
1777906636.385,pflash_load,0,46,0,28.09,230.00,1,24576,0
|
||||
1777906636.385,pflash_load,1,51,0,22.86,230.00,1,24576,0
|
||||
1777906637.433,pflash_load,0,46,0,28.15,230.00,1,24576,0
|
||||
1777906637.433,pflash_load,1,51,0,22.83,230.00,1,24576,0
|
||||
1777906638.483,pflash_load,0,46,0,28.27,230.00,1,24576,0
|
||||
1777906638.483,pflash_load,1,51,0,22.84,230.00,1,24576,0
|
||||
1777906639.532,pflash_load,0,46,0,28.09,230.00,1,24576,0
|
||||
1777906639.532,pflash_load,1,51,0,22.92,230.00,1,24576,0
|
||||
1777906640.580,pflash_load,0,46,0,28.14,230.00,1,24576,0
|
||||
1777906640.580,pflash_load,1,51,0,22.84,230.00,1,24576,0
|
||||
1777906641.630,pflash_load,0,46,0,28.45,230.00,1,24576,0
|
||||
1777906641.630,pflash_load,1,51,0,22.86,230.00,1,24576,0
|
||||
1777906642.679,pflash_load,0,46,0,28.33,230.00,1,24576,0
|
||||
1777906642.679,pflash_load,1,51,0,23.08,230.00,1,24576,0
|
||||
1777906643.727,pflash_load,0,46,0,28.27,230.00,1,24576,0
|
||||
1777906643.727,pflash_load,1,51,0,22.96,230.00,1,24576,0
|
||||
1777906644.777,pflash_load,0,46,0,28.46,230.00,1,24576,0
|
||||
1777906644.777,pflash_load,1,51,0,22.77,230.00,1,24576,0
|
||||
1777906645.824,pflash_load,0,46,0,28.40,230.00,1,24576,0
|
||||
1777906645.824,pflash_load,1,51,0,22.96,230.00,1,24576,0
|
||||
1777906646.873,pflash_load,0,46,0,28.47,230.00,1,24576,0
|
||||
1777906646.873,pflash_load,1,51,0,22.97,230.00,1,24576,0
|
||||
1777906647.922,pflash_load,0,46,0,28.35,230.00,1,24576,0
|
||||
1777906647.922,pflash_load,1,51,0,22.92,230.00,1,24576,0
|
||||
1777906648.970,pflash_load,0,46,0,28.48,230.00,1,24576,0
|
||||
1777906648.970,pflash_load,1,51,0,22.96,230.00,1,24576,0
|
||||
1777906650.020,pflash_load,0,46,0,28.33,230.00,1,24576,0
|
||||
1777906650.020,pflash_load,1,51,0,22.71,230.00,1,24576,0
|
||||
1777906651.068,pflash_load,0,46,0,28.42,230.00,1,24576,0
|
||||
1777906651.068,pflash_load,1,51,0,22.62,230.00,1,24576,0
|
||||
1777906652.116,pflash_load,0,46,0,28.54,230.00,1,24576,0
|
||||
1777906652.116,pflash_load,1,51,0,22.77,230.00,1,24576,0
|
||||
1777906653.167,pflash_load,0,46,0,28.36,230.00,1,24576,0
|
||||
1777906653.167,pflash_load,1,51,0,23.03,230.00,1,24576,0
|
||||
1777906654.222,pflash_load,0,46,0,28.37,230.00,1,24576,0
|
||||
1777906654.222,pflash_load,1,51,0,23.16,230.00,1,24576,0
|
||||
1777906655.273,pflash_load,0,46,0,28.48,230.00,1,24576,0
|
||||
1777906655.273,pflash_load,1,51,0,22.95,230.00,1,24576,0
|
||||
1777906656.323,pflash_load,0,46,0,28.35,230.00,1,24576,0
|
||||
1777906656.323,pflash_load,1,51,0,22.90,230.00,1,24576,0
|
||||
1777906657.372,pflash_load,0,46,0,28.40,230.00,1,24576,0
|
||||
1777906657.372,pflash_load,1,51,0,22.75,230.00,1,24576,0
|
||||
1777906658.422,pflash_load,0,46,0,28.56,230.00,1,24576,0
|
||||
1777906658.422,pflash_load,1,51,0,22.79,230.00,1,24576,0
|
||||
1777906659.471,pflash_load,0,46,0,28.40,230.00,1,24576,0
|
||||
1777906659.471,pflash_load,1,51,0,22.91,230.00,1,24576,0
|
||||
|
@@ -0,0 +1,17 @@
|
||||
[transformers] PyTorch was not found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
|
||||
[transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (251956 > 131072). Running this sequence through the model will result in indexing errors
|
||||
Traceback (most recent call last):
|
||||
File "/opt/ai/lucebox-hub/dflash/scripts/phase_split_dual_gpu.py", line 525, in <module>
|
||||
main()
|
||||
File "/opt/ai/lucebox-hub/dflash/scripts/phase_split_dual_gpu.py", line 521, in main
|
||||
args.func(args)
|
||||
File "/opt/ai/lucebox-hub/dflash/scripts/phase_split_dual_gpu.py", line 478, in run_bench_niah
|
||||
run_cases(args, cases)
|
||||
File "/opt/ai/lucebox-hub/dflash/scripts/phase_split_dual_gpu.py", line 332, in run_cases
|
||||
ready_s = daemon.start()
|
||||
^^^^^^^^^^^^^^
|
||||
File "/opt/ai/lucebox-hub/dflash/scripts/phase_split_dual_gpu.py", line 124, in start
|
||||
self.log.wait_for("[pflash-daemon] ready", 180)
|
||||
File "/opt/ai/lucebox-hub/dflash/scripts/phase_split_dual_gpu.py", line 90, in wait_for
|
||||
raise TimeoutError(f"timed out waiting for {needle!r}; tail={tail[-12:]}")
|
||||
TimeoutError: timed out waiting for '[pflash-daemon] ready'; tail=['ggml_cuda_init: found 1 CUDA devices (Total VRAM: 24126 MiB):', ' Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, VRAM: 24126 MiB']
|
||||
@@ -0,0 +1,2 @@
|
||||
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 24126 MiB):
|
||||
Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, VRAM: 24126 MiB
|
||||
41
results/lucebox-pflash-niah-20260504-150321/gpu_monitor.csv
Normal file
41
results/lucebox-pflash-niah-20260504-150321/gpu_monitor.csv
Normal file
@@ -0,0 +1,41 @@
|
||||
ts,phase,index,temp_c,fan_pct,power_w,power_limit_w,mem_used_mib,mem_total_mib,util_pct
|
||||
1777907036.255,pflash_load,0,51,0,28.92,230.00,1,24576,0
|
||||
1777907036.255,pflash_load,1,52,0,23.08,230.00,1,24576,0
|
||||
1777907037.309,niah_ctx16372,0,54,0,61.36,230.00,4348,24576,1
|
||||
1777907037.309,niah_ctx16372,1,52,0,25.83,230.00,1,24576,0
|
||||
1777907038.355,niah_ctx32764,0,58,57,210.91,230.00,6972,24576,100
|
||||
1777907038.355,niah_ctx32764,1,52,0,27.22,230.00,1,24576,0
|
||||
1777907039.401,niah_ctx32764,0,59,54,223.68,230.00,6984,24576,99
|
||||
1777907039.401,niah_ctx32764,1,52,0,23.30,230.00,1,24576,0
|
||||
1777907040.447,niah_ctx65524,0,58,56,211.54,230.00,11004,24576,100
|
||||
1777907040.447,niah_ctx65524,1,52,0,23.00,230.00,1,24576,0
|
||||
1777907041.492,niah_ctx65524,0,59,59,224.04,230.00,10954,24576,99
|
||||
1777907041.492,niah_ctx65524,1,52,0,22.92,230.00,1,24576,0
|
||||
1777907042.536,niah_ctx65524,0,60,62,227.43,230.00,10954,24576,99
|
||||
1777907042.536,niah_ctx65524,1,52,0,22.91,230.00,1,24576,0
|
||||
1777907043.583,niah_ctx65524,0,60,63,226.14,230.00,11482,24576,100
|
||||
1777907043.583,niah_ctx65524,1,52,0,22.81,230.00,1,24576,0
|
||||
1777907044.715,niah_ctx131068,0,59,64,190.93,230.00,18702,24576,100
|
||||
1777907044.715,niah_ctx131068,1,52,0,22.77,230.00,1,24576,0
|
||||
1777907045.761,niah_ctx131068,0,60,65,221.91,230.00,18944,24576,100
|
||||
1777907045.761,niah_ctx131068,1,52,0,22.94,230.00,1,24576,0
|
||||
1777907046.805,niah_ctx131068,0,60,65,227.38,230.00,19138,24576,99
|
||||
1777907046.805,niah_ctx131068,1,52,0,22.93,230.00,1,24576,0
|
||||
1777907047.850,niah_ctx131068,0,61,65,227.67,230.00,18944,24576,100
|
||||
1777907047.850,niah_ctx131068,1,52,0,22.81,230.00,1,24576,0
|
||||
1777907048.894,niah_ctx131068,0,61,66,227.86,230.00,19138,24576,99
|
||||
1777907048.894,niah_ctx131068,1,52,0,22.87,230.00,1,24576,0
|
||||
1777907049.945,niah_ctx131068,0,61,66,227.93,230.00,19138,24576,100
|
||||
1777907049.945,niah_ctx131068,1,52,0,22.92,230.00,1,24576,0
|
||||
1777907051.001,niah_ctx131068,0,61,66,227.80,230.00,18944,24576,100
|
||||
1777907051.001,niah_ctx131068,1,52,0,23.00,230.00,1,24576,0
|
||||
1777907052.057,niah_ctx131068,0,61,66,228.89,230.00,19138,24576,99
|
||||
1777907052.057,niah_ctx131068,1,51,0,22.97,230.00,1,24576,0
|
||||
1777907053.102,niah_ctx131068,0,60,67,219.70,230.00,21024,24576,94
|
||||
1777907053.102,niah_ctx131068,1,51,0,22.80,230.00,1,24576,0
|
||||
1777907054.223,niah_ctx131068,0,60,67,191.47,230.00,21024,24576,93
|
||||
1777907054.223,niah_ctx131068,1,51,0,22.93,230.00,1,24576,0
|
||||
1777907055.359,niah_ctx131068,0,59,67,190.35,230.00,2046,24576,92
|
||||
1777907055.359,niah_ctx131068,1,51,0,22.99,230.00,1,24576,0
|
||||
1777907056.402,cleanup,0,59,67,178.23,230.00,4080,24576,100
|
||||
1777907056.402,cleanup,1,51,0,22.87,230.00,1,24576,0
|
||||
|
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,5 @@
|
||||
Below is a long passage. Keep important facts from the passage.
|
||||
|
||||
The grass is green. The sky is blue. The sun is yellow. Here we go. sun is yellow. Here we go. There and back again. The special magic keymark0zeta number is 0438574. sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again. The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again.
|
||||
|
||||
Question: What is the special magic keymark0zeta number?
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,92 @@
|
||||
[transformers] PyTorch was not found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
|
||||
[transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (251956 > 131072). Running this sequence through the model will result in indexing errors
|
||||
[transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (199996 > 131072). Running this sequence through the model will result in indexing errors
|
||||
{
|
||||
"date": "2026-05-04",
|
||||
"mode": "dual_gpu_pflash_phase_split",
|
||||
"pflash_gpu": 0,
|
||||
"pflash_daemon_ready_s": 0.6323865110753104,
|
||||
"pflash_drafter": "/mnt/models/gguf/qwen3-draft-0.6b/Qwen3-0.6B-BF16.gguf",
|
||||
"tokenizer": "Qwen/Qwen3-0.6B",
|
||||
"keep_ratio": 0.05,
|
||||
"lookahead": 2,
|
||||
"chunk_size": 32,
|
||||
"pool_kernel": 13,
|
||||
"pflash_k_type": "compute",
|
||||
"cases": [
|
||||
{
|
||||
"name": "niah_ctx16372",
|
||||
"source_tokens": 16372,
|
||||
"compressed_tokens": 788,
|
||||
"compress_wall_s": 1.0830025799805298,
|
||||
"compress_tok_s": 15117.230838262947,
|
||||
"compression_ratio": 0.04813095528951869,
|
||||
"retained_key": true,
|
||||
"retained_answer": true
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx32764",
|
||||
"source_tokens": 32764,
|
||||
"compressed_tokens": 1628,
|
||||
"compress_wall_s": 1.799773955019191,
|
||||
"compress_tok_s": 18204.50835430088,
|
||||
"compression_ratio": 0.04968868270052497,
|
||||
"retained_key": true,
|
||||
"retained_answer": true
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx65524",
|
||||
"source_tokens": 65524,
|
||||
"compressed_tokens": 3252,
|
||||
"compress_wall_s": 4.3655310838948935,
|
||||
"compress_tok_s": 15009.399484458598,
|
||||
"compression_ratio": 0.04963066967828582,
|
||||
"retained_key": true,
|
||||
"retained_answer": true
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx131068",
|
||||
"source_tokens": 131068,
|
||||
"compressed_tokens": 6524,
|
||||
"compress_wall_s": 10.801249213982373,
|
||||
"compress_tok_s": 12134.522350463923,
|
||||
"compression_ratio": 0.04977568895535142,
|
||||
"retained_key": true,
|
||||
"retained_answer": true
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx199996",
|
||||
"source_tokens": 199996,
|
||||
"compressed_tokens": 0,
|
||||
"compress_wall_s": 0.10222281608730555,
|
||||
"compress_tok_s": 1956471.2424786773,
|
||||
"compression_ratio": 0.0,
|
||||
"retained_key": false,
|
||||
"retained_answer": false
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx259996",
|
||||
"source_tokens": 259996,
|
||||
"compressed_tokens": 0,
|
||||
"compress_wall_s": 0.09130622097291052,
|
||||
"compress_tok_s": 2847516.820098575,
|
||||
"compression_ratio": 0.0,
|
||||
"retained_key": false,
|
||||
"retained_answer": false
|
||||
}
|
||||
],
|
||||
"resource_summary": {
|
||||
"samples": 20,
|
||||
"mem_max_mib": 21024.0,
|
||||
"temp_max_c": 61.0,
|
||||
"fan_max_pct": 67.0,
|
||||
"power_avg_w": 198.707,
|
||||
"power_max_w": 228.89,
|
||||
"util_avg_pct": 88.7,
|
||||
"util_max_pct": 100.0
|
||||
},
|
||||
"logs": {
|
||||
"pflash": "/opt/ai/github/club-3090/results/lucebox-pflash-niah-20260504-150321/pflash_daemon.log",
|
||||
"monitor": "/opt/ai/github/club-3090/results/lucebox-pflash-niah-20260504-150321/gpu_monitor.csv"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 24126 MiB):
|
||||
Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, VRAM: 24126 MiB
|
||||
[drafter] loaded Qwen3-0.6B BF16: n_layer=28 n_head=16 n_kv=8 n_embd=1024 n_ff=3072 head_dim=128 vocab=151936
|
||||
[pflash-daemon] ready load=0.553s vocab=151936
|
||||
[pflash-daemon] compress start tokens=16372 keep=0.050 lookahead=2 chunk=32 pool=13 path=/opt/ai/github/club-3090/results/lucebox-pflash-niah-20260504-150321/niah_ctx16372/prompt_counted.bin
|
||||
[qwen3-0.6b-fp] layer 1/28 done (A=0.142s FP=0.011s B=0.017s)
|
||||
[qwen3-0.6b-fp] layer 28/28 done (A=0.314s FP=0.204s B=0.301s)
|
||||
[qwen3-0.6b-fp] forward 0.83s (S=16372, A=0.31s FP=0.20s B=0.30s) tail-score 0.18s total 1.00s
|
||||
[drafter] forward+score in 1.08s S=16372
|
||||
[drafter] score_and_compress total 1.08s S=16372 kept=788 (25/512 chunks)
|
||||
[pflash-daemon] compress done 1.081s in=16372 out=788 ratio=0.0481
|
||||
[pflash-daemon] compress start tokens=32764 keep=0.050 lookahead=2 chunk=32 pool=13 path=/opt/ai/github/club-3090/results/lucebox-pflash-niah-20260504-150321/niah_ctx32764/prompt_counted.bin
|
||||
[qwen3-0.6b-fp] layer 1/28 done (A=0.010s FP=0.015s B=0.023s)
|
||||
[qwen3-0.6b-fp] layer 28/28 done (A=0.346s FP=0.539s B=0.602s)
|
||||
[qwen3-0.6b-fp] forward 1.49s (S=32764, A=0.35s FP=0.54s B=0.60s) tail-score 0.20s total 1.69s
|
||||
[drafter] forward+score in 1.80s S=32764
|
||||
[drafter] score_and_compress total 1.80s S=32764 kept=1628 (51/1024 chunks)
|
||||
[pflash-daemon] compress done 1.797s in=32764 out=1628 ratio=0.0497
|
||||
[pflash-daemon] compress start tokens=65524 keep=0.050 lookahead=2 chunk=32 pool=13 path=/opt/ai/github/club-3090/results/lucebox-pflash-niah-20260504-150321/niah_ctx65524/prompt_counted.bin
|
||||
[qwen3-0.6b-fp] layer 1/28 done (A=0.021s FP=0.070s B=0.041s)
|
||||
[qwen3-0.6b-fp] layer 28/28 done (A=0.731s FP=1.480s B=1.149s)
|
||||
[qwen3-0.6b-fp] forward 3.37s (S=65524, A=0.73s FP=1.48s B=1.15s) tail-score 0.80s total 4.17s
|
||||
[drafter] forward+score in 4.36s S=65524
|
||||
[drafter] score_and_compress total 4.36s S=65524 kept=3252 (102/2048 chunks)
|
||||
[pflash-daemon] compress done 4.360s in=65524 out=3252 ratio=0.0496
|
||||
[pflash-daemon] compress start tokens=131068 keep=0.050 lookahead=2 chunk=32 pool=13 path=/opt/ai/github/club-3090/results/lucebox-pflash-niah-20260504-150321/niah_ctx131068/prompt_counted.bin
|
||||
[qwen3-0.6b-fp] layer 1/28 done (A=0.050s FP=0.159s B=0.084s)
|
||||
[qwen3-0.6b-fp] layer 28/28 done (A=1.466s FP=4.331s B=2.406s)
|
||||
[qwen3-0.6b-fp] forward 8.22s (S=131068, A=1.47s FP=4.33s B=2.41s) tail-score 2.39s total 10.60s
|
||||
[drafter] forward+score in 10.79s S=131068
|
||||
[drafter] score_and_compress total 10.79s S=131068 kept=6524 (204/4096 chunks)
|
||||
[pflash-daemon] compress done 10.792s in=131068 out=6524 ratio=0.0498
|
||||
[pflash-daemon] compress start tokens=199996 keep=0.050 lookahead=2 chunk=32 pool=13 path=/opt/ai/github/club-3090/results/lucebox-pflash-niah-20260504-150321/niah_ctx199996/prompt_counted.bin
|
||||
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 390.62 MiB on device 0: cudaMalloc failed: out of memory
|
||||
alloc_tensor_range: failed to allocate CUDA0 buffer of size 409591808
|
||||
[pflash-daemon] compress failed: forward_qwen3_0p6b: K_curr/V_curr/Q_last alloc failed at layer 25
|
||||
[pflash-daemon] compress start tokens=259996 keep=0.050 lookahead=2 chunk=32 pool=13 path=/opt/ai/github/club-3090/results/lucebox-pflash-niah-20260504-150321/niah_ctx259996/prompt_counted.bin
|
||||
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 507.80 MiB on device 0: cudaMalloc failed: out of memory
|
||||
alloc_tensor_range: failed to allocate CUDA0 buffer of size 532471808
|
||||
[pflash-daemon] compress failed: forward_qwen3_0p6b: K_curr/V_curr/Q_last alloc failed at layer 18
|
||||
[pflash-daemon] stopped
|
||||
89
results/lucebox-pflash-niah-20260504-150321/summary.json
Normal file
89
results/lucebox-pflash-niah-20260504-150321/summary.json
Normal file
@@ -0,0 +1,89 @@
|
||||
{
|
||||
"date": "2026-05-04",
|
||||
"mode": "dual_gpu_pflash_phase_split",
|
||||
"pflash_gpu": 0,
|
||||
"pflash_daemon_ready_s": 0.6323865110753104,
|
||||
"pflash_drafter": "/mnt/models/gguf/qwen3-draft-0.6b/Qwen3-0.6B-BF16.gguf",
|
||||
"tokenizer": "Qwen/Qwen3-0.6B",
|
||||
"keep_ratio": 0.05,
|
||||
"lookahead": 2,
|
||||
"chunk_size": 32,
|
||||
"pool_kernel": 13,
|
||||
"pflash_k_type": "compute",
|
||||
"cases": [
|
||||
{
|
||||
"name": "niah_ctx16372",
|
||||
"source_tokens": 16372,
|
||||
"compressed_tokens": 788,
|
||||
"compress_wall_s": 1.0830025799805298,
|
||||
"compress_tok_s": 15117.230838262947,
|
||||
"compression_ratio": 0.04813095528951869,
|
||||
"retained_key": true,
|
||||
"retained_answer": true
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx32764",
|
||||
"source_tokens": 32764,
|
||||
"compressed_tokens": 1628,
|
||||
"compress_wall_s": 1.799773955019191,
|
||||
"compress_tok_s": 18204.50835430088,
|
||||
"compression_ratio": 0.04968868270052497,
|
||||
"retained_key": true,
|
||||
"retained_answer": true
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx65524",
|
||||
"source_tokens": 65524,
|
||||
"compressed_tokens": 3252,
|
||||
"compress_wall_s": 4.3655310838948935,
|
||||
"compress_tok_s": 15009.399484458598,
|
||||
"compression_ratio": 0.04963066967828582,
|
||||
"retained_key": true,
|
||||
"retained_answer": true
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx131068",
|
||||
"source_tokens": 131068,
|
||||
"compressed_tokens": 6524,
|
||||
"compress_wall_s": 10.801249213982373,
|
||||
"compress_tok_s": 12134.522350463923,
|
||||
"compression_ratio": 0.04977568895535142,
|
||||
"retained_key": true,
|
||||
"retained_answer": true
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx199996",
|
||||
"source_tokens": 199996,
|
||||
"compressed_tokens": 0,
|
||||
"compress_wall_s": 0.10222281608730555,
|
||||
"compress_tok_s": 1956471.2424786773,
|
||||
"compression_ratio": 0.0,
|
||||
"retained_key": false,
|
||||
"retained_answer": false
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx259996",
|
||||
"source_tokens": 259996,
|
||||
"compressed_tokens": 0,
|
||||
"compress_wall_s": 0.09130622097291052,
|
||||
"compress_tok_s": 2847516.820098575,
|
||||
"compression_ratio": 0.0,
|
||||
"retained_key": false,
|
||||
"retained_answer": false
|
||||
}
|
||||
],
|
||||
"resource_summary": {
|
||||
"samples": 20,
|
||||
"mem_max_mib": 21024.0,
|
||||
"temp_max_c": 61.0,
|
||||
"fan_max_pct": 67.0,
|
||||
"power_avg_w": 198.707,
|
||||
"power_max_w": 228.89,
|
||||
"util_avg_pct": 88.7,
|
||||
"util_max_pct": 100.0
|
||||
},
|
||||
"logs": {
|
||||
"pflash": "/opt/ai/github/club-3090/results/lucebox-pflash-niah-20260504-150321/pflash_daemon.log",
|
||||
"monitor": "/opt/ai/github/club-3090/results/lucebox-pflash-niah-20260504-150321/gpu_monitor.csv"
|
||||
}
|
||||
}
|
||||
28
results/lucebox-pflash-niah-20260504-150321/summary.md
Normal file
28
results/lucebox-pflash-niah-20260504-150321/summary.md
Normal file
@@ -0,0 +1,28 @@
|
||||
# Dual-GPU PFlash Phase-Split Report
|
||||
|
||||
- PFlash GPU: `0`
|
||||
- PFlash daemon ready: `0.63 s`
|
||||
- keep ratio: `0.05`
|
||||
- lookahead: `2`
|
||||
- PFlash K cache: `compute`
|
||||
|
||||
## Resource Peak
|
||||
|
||||
| gpu | samples | peak mem MiB | peak temp C | avg power W | peak power W | avg util % | peak util % |
|
||||
|---:|---:|---:|---:|---:|---:|---:|---:|
|
||||
| 0 | 20 | 21024.00 | 61.00 | 198.71 | 228.89 | 88.70 | 100.00 |
|
||||
|
||||
## Cases
|
||||
|
||||
| case | source tokens | compressed tokens | ratio | PFlash s | PFlash tok/s | key retained | answer retained |
|
||||
|---|---:|---:|---:|---:|---:|:---:|:---:|
|
||||
| niah_ctx16372 | 16372 | 788 | 0.0481 | 1.08 | 15117.23 | yes | yes |
|
||||
| niah_ctx32764 | 32764 | 1628 | 0.0497 | 1.80 | 18204.51 | yes | yes |
|
||||
| niah_ctx65524 | 65524 | 3252 | 0.0496 | 4.37 | 15009.40 | yes | yes |
|
||||
| niah_ctx131068 | 131068 | 6524 | 0.0498 | 10.80 | 12134.52 | yes | yes |
|
||||
| niah_ctx199996 | 199996 | 0 | 0.0000 | 0.10 | 1956471.24 | no | no |
|
||||
| niah_ctx259996 | 259996 | 0 | 0.0000 | 0.09 | 2847516.82 | no | no |
|
||||
|
||||
Files:
|
||||
- pflash: `/opt/ai/github/club-3090/results/lucebox-pflash-niah-20260504-150321/pflash_daemon.log`
|
||||
- monitor: `/opt/ai/github/club-3090/results/lucebox-pflash-niah-20260504-150321/gpu_monitor.csv`
|
||||
@@ -0,0 +1,25 @@
|
||||
ts,phase,index,temp_c,fan_pct,power_w,power_limit_w,mem_used_mib,mem_total_mib,util_pct
|
||||
1777907191.039,pflash_load,0,43,0,27.92,230.00,1,24576,0
|
||||
1777907191.039,pflash_load,1,49,0,22.60,230.00,1,24576,0
|
||||
1777907192.089,niah_ctx131068,0,47,0,83.13,230.00,18690,24576,17
|
||||
1777907192.089,niah_ctx131068,1,50,0,25.62,230.00,1,24576,0
|
||||
1777907193.135,niah_ctx131068,0,51,0,214.50,230.00,18944,24576,99
|
||||
1777907193.135,niah_ctx131068,1,50,0,26.32,230.00,1,24576,0
|
||||
1777907194.195,niah_ctx131068,0,53,0,227.52,230.00,19138,24576,100
|
||||
1777907194.195,niah_ctx131068,1,50,0,23.25,230.00,1,24576,0
|
||||
1777907195.247,niah_ctx131068,0,53,0,229.01,230.00,19138,24576,99
|
||||
1777907195.247,niah_ctx131068,1,49,0,22.55,230.00,1,24576,0
|
||||
1777907196.291,niah_ctx131068,0,54,0,227.81,230.00,18944,24576,100
|
||||
1777907196.291,niah_ctx131068,1,49,0,22.45,230.00,1,24576,0
|
||||
1777907197.337,niah_ctx131068,0,53,56,229.25,230.00,19138,24576,99
|
||||
1777907197.337,niah_ctx131068,1,49,0,22.45,230.00,1,24576,0
|
||||
1777907198.384,niah_ctx131068,0,54,52,227.66,230.00,19138,24576,100
|
||||
1777907198.384,niah_ctx131068,1,49,0,22.55,230.00,1,24576,0
|
||||
1777907199.430,niah_ctx131068,0,54,54,229.18,230.00,18944,24576,99
|
||||
1777907199.430,niah_ctx131068,1,49,0,22.46,230.00,1,24576,0
|
||||
1777907200.474,niah_ctx131068,0,53,56,211.57,230.00,21024,24576,94
|
||||
1777907200.474,niah_ctx131068,1,49,0,22.56,230.00,1,24576,0
|
||||
1777907201.614,niah_ctx131068,0,53,58,186.76,230.00,21024,24576,93
|
||||
1777907201.614,niah_ctx131068,1,49,0,22.60,230.00,1,24576,0
|
||||
1777907202.805,niah_ctx131068,0,53,58,186.52,230.00,2048,24576,84
|
||||
1777907202.805,niah_ctx131068,1,49,0,22.61,230.00,1,24576,0
|
||||
|
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,62 @@
|
||||
[transformers] PyTorch was not found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
|
||||
[transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (251956 > 131072). Running this sequence through the model will result in indexing errors
|
||||
[transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (199996 > 131072). Running this sequence through the model will result in indexing errors
|
||||
{
|
||||
"date": "2026-05-04",
|
||||
"mode": "dual_gpu_pflash_phase_split",
|
||||
"pflash_gpu": 0,
|
||||
"pflash_daemon_ready_s": 0.6027854409767315,
|
||||
"pflash_drafter": "/mnt/models/gguf/qwen3-draft-0.6b/Qwen3-0.6B-BF16.gguf",
|
||||
"tokenizer": "Qwen/Qwen3-0.6B",
|
||||
"keep_ratio": 0.05,
|
||||
"lookahead": 2,
|
||||
"chunk_size": 32,
|
||||
"pool_kernel": 13,
|
||||
"pflash_k_type": "q8_0",
|
||||
"cases": [
|
||||
{
|
||||
"name": "niah_ctx131068",
|
||||
"source_tokens": 131068,
|
||||
"compressed_tokens": 6524,
|
||||
"compress_wall_s": 10.65377599094063,
|
||||
"compress_tok_s": 12302.492572722838,
|
||||
"compression_ratio": 0.04977568895535142,
|
||||
"retained_key": true,
|
||||
"retained_answer": true
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx199996",
|
||||
"source_tokens": 199996,
|
||||
"compressed_tokens": 0,
|
||||
"compress_wall_s": 0.10241466399747878,
|
||||
"compress_tok_s": 1952806.2895848923,
|
||||
"compression_ratio": 0.0,
|
||||
"retained_key": false,
|
||||
"retained_answer": false
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx259996",
|
||||
"source_tokens": 259996,
|
||||
"compressed_tokens": 0,
|
||||
"compress_wall_s": 0.10354933398775756,
|
||||
"compress_tok_s": 2510841.837290222,
|
||||
"compression_ratio": 0.0,
|
||||
"retained_key": false,
|
||||
"retained_answer": false
|
||||
}
|
||||
],
|
||||
"resource_summary": {
|
||||
"samples": 12,
|
||||
"mem_max_mib": 21024.0,
|
||||
"temp_max_c": 54.0,
|
||||
"fan_max_pct": 58.0,
|
||||
"power_avg_w": 190.06916666666666,
|
||||
"power_max_w": 229.25,
|
||||
"util_avg_pct": 82.0,
|
||||
"util_max_pct": 100.0
|
||||
},
|
||||
"logs": {
|
||||
"pflash": "/opt/ai/github/club-3090/results/lucebox-pflash-niah-q8k-20260504-150600/pflash_daemon.log",
|
||||
"monitor": "/opt/ai/github/club-3090/results/lucebox-pflash-niah-q8k-20260504-150600/gpu_monitor.csv"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 24126 MiB):
|
||||
Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, VRAM: 24126 MiB
|
||||
[drafter] loaded Qwen3-0.6B BF16: n_layer=28 n_head=16 n_kv=8 n_embd=1024 n_ff=3072 head_dim=128 vocab=151936
|
||||
[pflash-daemon] ready load=0.521s vocab=151936
|
||||
[pflash-daemon] compress start tokens=131068 keep=0.050 lookahead=2 chunk=32 pool=13 path=/opt/ai/github/club-3090/results/lucebox-pflash-niah-q8k-20260504-150600/niah_ctx131068/prompt_counted.bin
|
||||
[qwen3-0.6b-fp] layer 1/28 done (A=0.193s FP=0.152s B=0.086s)
|
||||
[qwen3-0.6b-fp] layer 28/28 done (A=1.584s FP=4.234s B=2.342s)
|
||||
[qwen3-0.6b-fp] forward 8.17s (S=131068, A=1.58s FP=4.23s B=2.34s) tail-score 2.34s total 10.52s
|
||||
[drafter] forward+score in 10.64s S=131068
|
||||
[drafter] score_and_compress total 10.64s S=131068 kept=6524 (204/4096 chunks)
|
||||
[pflash-daemon] compress done 10.645s in=131068 out=6524 ratio=0.0498
|
||||
[pflash-daemon] compress start tokens=199996 keep=0.050 lookahead=2 chunk=32 pool=13 path=/opt/ai/github/club-3090/results/lucebox-pflash-niah-q8k-20260504-150600/niah_ctx199996/prompt_counted.bin
|
||||
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 390.62 MiB on device 0: cudaMalloc failed: out of memory
|
||||
alloc_tensor_range: failed to allocate CUDA0 buffer of size 409591808
|
||||
[pflash-daemon] compress failed: forward_qwen3_0p6b: K_curr/V_curr/Q_last alloc failed at layer 25
|
||||
[pflash-daemon] compress start tokens=259996 keep=0.050 lookahead=2 chunk=32 pool=13 path=/opt/ai/github/club-3090/results/lucebox-pflash-niah-q8k-20260504-150600/niah_ctx259996/prompt_counted.bin
|
||||
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 507.80 MiB on device 0: cudaMalloc failed: out of memory
|
||||
alloc_tensor_range: failed to allocate CUDA0 buffer of size 532471808
|
||||
[pflash-daemon] compress failed: forward_qwen3_0p6b: K_curr/V_curr/Q_last alloc failed at layer 18
|
||||
[pflash-daemon] stopped
|
||||
59
results/lucebox-pflash-niah-q8k-20260504-150600/summary.json
Normal file
59
results/lucebox-pflash-niah-q8k-20260504-150600/summary.json
Normal file
@@ -0,0 +1,59 @@
|
||||
{
|
||||
"date": "2026-05-04",
|
||||
"mode": "dual_gpu_pflash_phase_split",
|
||||
"pflash_gpu": 0,
|
||||
"pflash_daemon_ready_s": 0.6027854409767315,
|
||||
"pflash_drafter": "/mnt/models/gguf/qwen3-draft-0.6b/Qwen3-0.6B-BF16.gguf",
|
||||
"tokenizer": "Qwen/Qwen3-0.6B",
|
||||
"keep_ratio": 0.05,
|
||||
"lookahead": 2,
|
||||
"chunk_size": 32,
|
||||
"pool_kernel": 13,
|
||||
"pflash_k_type": "q8_0",
|
||||
"cases": [
|
||||
{
|
||||
"name": "niah_ctx131068",
|
||||
"source_tokens": 131068,
|
||||
"compressed_tokens": 6524,
|
||||
"compress_wall_s": 10.65377599094063,
|
||||
"compress_tok_s": 12302.492572722838,
|
||||
"compression_ratio": 0.04977568895535142,
|
||||
"retained_key": true,
|
||||
"retained_answer": true
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx199996",
|
||||
"source_tokens": 199996,
|
||||
"compressed_tokens": 0,
|
||||
"compress_wall_s": 0.10241466399747878,
|
||||
"compress_tok_s": 1952806.2895848923,
|
||||
"compression_ratio": 0.0,
|
||||
"retained_key": false,
|
||||
"retained_answer": false
|
||||
},
|
||||
{
|
||||
"name": "niah_ctx259996",
|
||||
"source_tokens": 259996,
|
||||
"compressed_tokens": 0,
|
||||
"compress_wall_s": 0.10354933398775756,
|
||||
"compress_tok_s": 2510841.837290222,
|
||||
"compression_ratio": 0.0,
|
||||
"retained_key": false,
|
||||
"retained_answer": false
|
||||
}
|
||||
],
|
||||
"resource_summary": {
|
||||
"samples": 12,
|
||||
"mem_max_mib": 21024.0,
|
||||
"temp_max_c": 54.0,
|
||||
"fan_max_pct": 58.0,
|
||||
"power_avg_w": 190.06916666666666,
|
||||
"power_max_w": 229.25,
|
||||
"util_avg_pct": 82.0,
|
||||
"util_max_pct": 100.0
|
||||
},
|
||||
"logs": {
|
||||
"pflash": "/opt/ai/github/club-3090/results/lucebox-pflash-niah-q8k-20260504-150600/pflash_daemon.log",
|
||||
"monitor": "/opt/ai/github/club-3090/results/lucebox-pflash-niah-q8k-20260504-150600/gpu_monitor.csv"
|
||||
}
|
||||
}
|
||||
25
results/lucebox-pflash-niah-q8k-20260504-150600/summary.md
Normal file
25
results/lucebox-pflash-niah-q8k-20260504-150600/summary.md
Normal file
@@ -0,0 +1,25 @@
|
||||
# Dual-GPU PFlash Phase-Split Report
|
||||
|
||||
- PFlash GPU: `0`
|
||||
- PFlash daemon ready: `0.60 s`
|
||||
- keep ratio: `0.05`
|
||||
- lookahead: `2`
|
||||
- PFlash K cache: `q8_0`
|
||||
|
||||
## Resource Peak
|
||||
|
||||
| gpu | samples | peak mem MiB | peak temp C | avg power W | peak power W | avg util % | peak util % |
|
||||
|---:|---:|---:|---:|---:|---:|---:|---:|
|
||||
| 0 | 12 | 21024.00 | 54.00 | 190.07 | 229.25 | 82.00 | 100.00 |
|
||||
|
||||
## Cases
|
||||
|
||||
| case | source tokens | compressed tokens | ratio | PFlash s | PFlash tok/s | key retained | answer retained |
|
||||
|---|---:|---:|---:|---:|---:|:---:|:---:|
|
||||
| niah_ctx131068 | 131068 | 6524 | 0.0498 | 10.65 | 12302.49 | yes | yes |
|
||||
| niah_ctx199996 | 199996 | 0 | 0.0000 | 0.10 | 1952806.29 | no | no |
|
||||
| niah_ctx259996 | 259996 | 0 | 0.0000 | 0.10 | 2510841.84 | no | no |
|
||||
|
||||
Files:
|
||||
- pflash: `/opt/ai/github/club-3090/results/lucebox-pflash-niah-q8k-20260504-150600/pflash_daemon.log`
|
||||
- monitor: `/opt/ai/github/club-3090/results/lucebox-pflash-niah-q8k-20260504-150600/gpu_monitor.csv`
|
||||
Reference in New Issue
Block a user