docs(hardware): @apnar prefill-heavy 5090 sweep — proves per-workload power ceiling

Headline finding from @apnar's 4th + 5th sweeps in one day (disc #86):
- Decode N=8 (Gemma 4 + MTP): tops at 551W actual draw (vs 547W at N=4)
  → confirms decode is memory-bandwidth bound, not concurrency-limited
- Prefill-heavy (Qwen3.6 long-text): hits 599.98W actual draw at 600W cap
  → 99.997% cap-respect, full TDP saturation

Per-workload-class power ceiling on the 5090:
- Decode workloads: ~547-551W max (memory bandwidth limit, regardless of cap)
- Prefill workloads: ~600W (compute-bound, scales with TDP)

Both workload classes share efficiency knee at 400W cap (67% of stock TDP) —
that "60-85% of stock TDP" cross-rig pattern holds across workloads. But
absolute throughput behavior differs: decode at 600W gives only 4% more TPS
than at 400W (bandwidth ceiling); prefill at 600W gives 19% more TPS (compute
actually uses the watts).

Practical implication for 5090 deployments:
- Chat/IDE-agent: cap at 400W (huge efficiency win, ~5% TPS loss)
- RAG/long-context: leave at stock 600W (compute-bound, watts buy throughput)
- Mixed: 400W if chat-dominant; pure prefill loads suffer

New chart: docs/img/power-cap-5090-qwen36-prefill.png shows the curve with
the "599.98W actual at 600W cap" callout.

HARDWARE.md additions:
- 2 new BENCHMARKS rows for prefill-heavy 400W (sweet spot) + 600W (saturation)
- Inline embed of the prefill chart after the decode (Gemma 4 + MTP) chart
- New "Per-workload-class power ceilings" subsection with the cross-workload
  bottleneck table

Note on apnar's contribution: he ran 5 sweeps total today (50W resolution
v1, 50W after calibration fix v2, 10W canonical anchor v3, decode N=8
headroom probe v4, prefill-heavy compute-saturation test v5) — all on the
old token-bounded bench architecture before today's time-bounded redesign
shipped (his sweeps at 13-14 UTC, my time-bounded redesign at 18:15 UTC).
The cross-validation depth is rare — most cross-rig contributions stop at
one sweep.
This commit is contained in:
noonghunna
2026-05-07 18:33:56 +00:00
parent 1f94478179
commit d5ef8c89d0
3 changed files with 159 additions and 0 deletions

View File

@@ -126,6 +126,8 @@ How `decode-single` is timed (the new default since 2026-05-07):
| 5090 | air | vLLM `gemma-mtp` (TP=1) | Gemma 4 31B + MTP | **400W** ⭐ | 571.45 | 700.92 | **1.429** | [@apnar #86](https://github.com/noonghunna/club-3090/discussions/86#discussioncomment-16840610) |
| 5090 | air | vLLM `gemma-mtp` (TP=1) | Gemma 4 31B + MTP | 510W (peak narr) | 619.45 | 723.82 | 1.215 | same |
| 5090 | air | vLLM `gemma-mtp` (TP=1) | Gemma 4 31B + MTP | 600W (stock) | 600.65 | 756.67 | 1.103 | same |
| 5090 | air | vLLM `long-text` (Qwen3.6 27B) | **prefill-heavy** | **400W** ⭐ | 247.33 | (n/a) | **0.618** | [@apnar #86](https://github.com/noonghunna/club-3090/discussions/86#discussioncomment-16844473) |
| 5090 | air | vLLM `long-text` (Qwen3.6 27B) | **prefill-heavy** | 600W (stock) | 294.63 | (n/a) | 0.491 | same — **599.98W actual draw, full TDP saturation** |
⭐ = peak TPS/W efficiency on that rig.
@@ -137,6 +139,19 @@ For rigs where we have full 10W-resolution sweeps, the curves below show TPS + T
*5090 air-cooled + Gemma 4 31B + MTP, 21-cap sweep at 10W resolution. Yellow callout: 400W sweet spot (1.43 TPS/W). Red-shaded: 530-600W = workload-limited, ~547W max actual draw regardless of cap. Source data: [disc #86](https://github.com/noonghunna/club-3090/discussions/86#discussioncomment-16840610) (@apnar). Source script: [`img/power-cap-5090-gemma4.py`](img/power-cap-5090-gemma4.py).*
![5090 + Qwen3.6 + vLLM prefill-heavy power-cap efficiency curve (apnar)](img/power-cap-5090-qwen36-prefill.png)
*5090 air-cooled + Qwen3.6-27B AutoRound INT4 + vLLM long-text compose, 21-cap sweep, **prefill-heavy** workload (~50K-token prompt + max_tokens=10). **At 600W cap, actual draw = 599.98W (99.997% cap-respect)** — proving the decode-bound ~547W ceiling on this card is a memory-bandwidth limit, not a hardware/firmware cap. Prefill is compute-bound and saturates the full 600W TDP cleanly. Source data: [disc #86](https://github.com/noonghunna/club-3090/discussions/86#discussioncomment-16844473) (@apnar). Source script: [`img/power-cap-5090-qwen36-prefill.py`](img/power-cap-5090-qwen36-prefill.py).*
**Per-workload-class power ceilings on the 5090 (validated cross-workload by @apnar)**:
| Workload class | Bottleneck | Max sustainable draw | Best efficiency cap |
|---|---|---:|---:|
| **Decode** (chat / generation, decode-concurrent N=4 or N=8) | Memory bandwidth | ~547-551W | 400W (1.43 TPS/W) |
| **Prefill** (RAG, long-context, batch) | Compute (matmul) | **~600W (full TDP)** | 400W (0.618 TPS/W) |
The cross-workload pattern: **both workload classes have efficiency knee at 400W (67% of stock TDP)**, but prefill needs the full 600W envelope to maximize absolute throughput while decode never uses more than ~550W regardless of cap. **Practical implication**: cap your 5090 at 400W for max efficiency on chat workloads (you lose <5% TPS); for prefill-heavy long-context workloads, leave at stock 600W if you want max throughput, accept ~30% efficiency cost.
![4090 + Qwen3.6-27B + llama.cpp power-cap efficiency curve (laurimyllari)](img/power-cap-4090-qwen36.png)
*4090 air-cooled + Qwen3.6-27B Q3_K_XL + llama.cpp default, 15-cap sweep at 10W resolution (260-400W). Yellow callout: 260W sweet spot (0.186 TPS/W) at 33% below the 4090's 450W stock TDP. **+8% TPS for +54% wattage** going from 260W to 400W the 4090 is heavily workload-saturated on this single-stream decode path. Source data: [disc #62](https://github.com/noonghunna/club-3090/discussions/62#discussioncomment-16832066) (@laurimyllari). Source script: [`img/power-cap-4090-qwen36.py`](img/power-cap-4090-qwen36.py).*

Binary file not shown.

After

Width:  |  Height:  |  Size: 252 KiB

View File

@@ -0,0 +1,144 @@
"""Generate 5090 prefill-heavy power-cap efficiency chart from @apnar's sweep.
Source data: 2026-05-07 disc #86, 1× RTX 5090 air-cooled, vLLM long-text compose
running Qwen3.6-27B AutoRound INT4. Load mode: prefill-heavy (single ~50K-token
prompt with max_tokens=10). Each row: prefill TPS = response.usage.prompt_tokens
/ request wall time, median of 3 runs.
This is the companion chart to power-cap-5090-gemma4.png — that's the DECODE
efficiency curve, this is the PREFILL efficiency curve. Together they show
the per-workload-class power ceiling on the 5090.
"""
import matplotlib.pyplot as plt
# (cap_W, prefill_TPS, actual_W, eff_TPS_per_W) — full 21-cap sweep
data = [
(400, 247.33, 399.99, 0.618),
(410, 252.34, 409.99, 0.615),
(420, 255.28, 419.99, 0.608),
(430, 258.63, 429.99, 0.601),
(440, 262.26, 439.99, 0.596),
(450, 265.71, 449.99, 0.590),
(460, 268.92, 459.99, 0.585),
(470, 272.97, 469.99, 0.581),
(480, 275.81, 479.99, 0.575),
(490, 277.50, 489.99, 0.566),
(500, 278.90, 499.99, 0.558),
(510, 281.14, 509.99, 0.551),
(520, 282.44, 519.99, 0.543),
(530, 283.89, 529.99, 0.536),
(540, 285.49, 539.99, 0.529),
(550, 287.28, 549.99, 0.522),
(560, 288.39, 559.98, 0.515),
(570, 289.69, 569.99, 0.508),
(580, 291.66, 579.98, 0.503),
(590, 293.24, 589.99, 0.497),
(600, 294.63, 599.98, 0.491),
]
caps = [d[0] for d in data]
tps = [d[1] for d in data]
draw = [d[2] for d in data]
eff = [d[3] for d in data]
plt.rcParams.update({
"font.family": "sans-serif",
"font.size": 12,
"axes.titlesize": 16,
"axes.titleweight": "bold",
"axes.labelsize": 13,
"figure.facecolor": "white",
"axes.facecolor": "white",
})
fig, ax1 = plt.subplots(figsize=(11, 6.4), dpi=150)
# Left axis: prefill TPS
color_prefill = "#7b3fa0"
ax1.plot(caps, tps, "o-", color=color_prefill, linewidth=2.2, markersize=6,
label="Prefill TPS (compute-bound)", zorder=3)
ax1.set_xlabel("Power cap (W)", fontsize=13)
ax1.set_ylabel("Prefill TPS (~50K-token prompt + max_tokens=10)", fontsize=13)
ax1.set_xlim(395, 605)
ax1.set_ylim(240, 305)
ax1.grid(True, alpha=0.3, zorder=0)
ax1.tick_params(axis="both", labelsize=11)
# Right axis: actual draw + efficiency on twin axis
ax2 = ax1.twinx()
color_draw = "#b35900"
color_eff = "#d62728"
ax2.plot(caps, draw, "s-", color=color_draw, linewidth=1.5, markersize=5,
alpha=0.7, label="Actual draw (W)", zorder=2)
ax2.set_ylabel("Actual draw (W)", color=color_draw, fontsize=12)
ax2.tick_params(axis="y", labelcolor=color_draw, labelsize=11)
ax2.set_ylim(395, 605)
ax3 = ax1.twinx()
ax3.spines["right"].set_position(("outward", 60))
ax3.plot(caps, eff, "^--", color=color_eff, linewidth=1.5, markersize=4,
alpha=0.85, label="Efficiency (TPS/W)", zorder=1)
ax3.set_ylabel("Efficiency: prefill TPS/W", color=color_eff, fontsize=12)
ax3.tick_params(axis="y", labelcolor=color_eff, labelsize=11)
ax3.set_ylim(0.47, 0.65)
# Annotate cap-respect: prefill HITS the cap (compute-bound)
ax1.annotate(
"★ At 600W cap: 599.98W actual draw\n(prefill is compute-bound — 5090 uses\nthe full 600W TDP)",
xy=(600, 294.63),
xytext=(465, 248),
fontsize=10.5,
fontweight="bold",
bbox=dict(boxstyle="round,pad=0.4", facecolor="#fff3cd", edgecolor="goldenrod", linewidth=1.2),
arrowprops=dict(arrowstyle="->", color="goldenrod", lw=1.5),
zorder=4,
)
# Sweet spot annotation: 400W (best efficiency, like decode)
ax1.axvline(400, color="goldenrod", linestyle=":", alpha=0.5, linewidth=1.5)
ax1.annotate(
"★ 400W cap\n0.618 TPS/W (best efficiency)\n247 prefill TPS",
xy=(400, 247.33),
xytext=(420, 268),
fontsize=10.5,
fontweight="bold",
bbox=dict(boxstyle="round,pad=0.4", facecolor="#fff3cd", edgecolor="goldenrod", linewidth=1.2),
arrowprops=dict(arrowstyle="->", color="goldenrod", lw=1.5),
zorder=4,
)
# Title
ax1.set_title(
"RTX 5090 + Qwen3.6-27B + vLLM — prefill-heavy power-cap curve",
pad=14,
)
# Subtitle — explicitly call out the cross-workload finding
fig.text(
0.5, 0.92,
"1× 5090 air-cooled, vLLM long-text compose, ~50K-token prompt, max_tokens=10 | "
"Compare to decode (gemma-4-mtp) chart: decode tops at ~550W, prefill hits 600W cleanly | data: @apnar",
ha="center", fontsize=9.5, color="#666",
style="italic",
)
# Combined legend
lines1, labels1 = ax1.get_legend_handles_labels()
lines2, labels2 = ax2.get_legend_handles_labels()
lines3, labels3 = ax3.get_legend_handles_labels()
ax1.legend(lines1 + lines2 + lines3, labels1 + labels2 + labels3,
loc="upper left", fontsize=10, framealpha=0.95,
edgecolor="#ccc")
# Footer
fig.text(
0.99, 0.01,
"github.com/noonghunna/club-3090",
ha="right", fontsize=9, color="#888", style="italic",
)
plt.tight_layout(rect=(0, 0.02, 1, 0.92))
out = "/tmp/power_cap_sweep_5090_prefill.png"
plt.savefig(out, dpi=150, bbox_inches="tight", facecolor="white")
print(f"Saved: {out}")