docs(hardware): add Qwen3.6-35B-A3B (MoE) 3090 power-cap charts + comparison
Same 3090 (GPU 0, air-cooled), same engine (mainline llama.cpp), same Q4-class quant (Q4_K_XL). Only the model changes: dense Qwen3.6-27B vs A3B 35B (3B active per token). Findings: 1. MoE shifts decode sweet spot 80W lower: - Dense decode: 290W → 0.111 TPS/W, SM 1380 MHz at sweet spot - A3B decode: 210W → 0.546 TPS/W, SM 1290 MHz at sweet spot Each token only activates 3B of 35B params on MoE → less compute per token → bandwidth-bound knee fires at lower power. 2. Prefill sweet spot is workload-determined, NOT model-determined: - Dense prefill: 250W → 3.633 TPS/W - A3B prefill: 250W → 9.865 TPS/W Both converge to same cap because prefill is compute-bound regardless of MoE routing. 3. Boost-clock plateau depends on workload AND model: - Dense decode: PLATEAU at 340-370W (SM 1560 MHz lock) - A3B decode: NO PLATEAU (SM climbs smoothly 1875→1890→1890→1905) - Dense prefill: PLATEAU at 330-370W (SM 1605-1620) - A3B prefill: PLATEAU at 340-370W (SM 1680-1710) Plateau auto-detection correctly flagged dense decode but not A3B decode — confirming firmware operating-point selection responds to compute pressure, not just to cap value. Adds: - docs/img/power-cap-3090-a3b-decode.py + .png - docs/img/power-cap-3090-a3b-prefill.py + .png - HARDWARE.md cross-rig table rows for both A3B sweeps - HARDWARE.md "Same hardware, MoE workload" subsection with comparison table + practical recommendation (A3B users → 210W cap, vs 290W for dense Qwen) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
noonghunna
@@ -151,6 +151,10 @@ How `decode-single` is timed (the new default since 2026-05-07):
|
||||
| 3090 | air | llama.cpp `prefill-heavy` (Qwen3.6-27B) | **prefill-heavy** | **250W** ⭐ | 906.79 | (n/a) | **3.633** | @noonghunna (this rig, 21-cap adaptive sweep, ~6m, SM 1350 MHz at sweet spot) |
|
||||
| 3090 | air | llama.cpp `prefill-heavy` (Qwen3.6-27B) | **prefill-heavy** | 370W (stock) | 1051.07 | (n/a) | 3.211 | same — SM locks at 1605-1620 MHz across 330-370W (boost-clock plateau, 327W draw) |
|
||||
| 3090 | air | llama.cpp `prefill-heavy` (Qwen3.6-27B) | **prefill-heavy** | 390W (max) | 1104.81 | (n/a) | 2.898 | same — SM 1710 MHz at 381W draw |
|
||||
| 3090 | air | llama.cpp default | **Qwen3.6 35B-A3B (MoE)** Q4_K_XL | **210W** ⭐ | 114.59 | 113.79 | **0.546** | @noonghunna (MoE shifts decode sweet spot 80W lower vs dense, SM 1290 MHz, no plateau) |
|
||||
| 3090 | air | llama.cpp default | Qwen3.6 35B-A3B (MoE) Q4_K_XL | 370W (stock) | 136.84 | 136.65 | 0.386 | same — SM climbs smoothly 1875→1905 across 340-370W (NO plateau, unlike dense) |
|
||||
| 3090 | air | llama.cpp `prefill-heavy` (35B-A3B MoE) | **prefill-heavy** | **250W** ⭐ | 2461.22 | (n/a) | **9.865** | @noonghunna (MoE prefill knee at SAME 250W as dense — workload-class converges, SM 1380 MHz) |
|
||||
| 3090 | air | llama.cpp `prefill-heavy` (35B-A3B MoE) | **prefill-heavy** | 370W (stock) | 2794.36 | (n/a) | 8.363 | same — SM locks at 1680-1710 MHz across 340-370W (boost-clock plateau detected) |
|
||||
| 4090 | air | llama.cpp default | Qwen3.6 27B Q3_K_XL | **260W** ⭐ | 48.41 | 48.43 | 0.186 | [@laurimyllari #62](https://github.com/noonghunna/club-3090/discussions/62#discussioncomment-16832066) |
|
||||
| 4090 | air | llama.cpp default | Qwen3.6 27B Q3_K_XL | 280W | 49.54 | 49.10 | 0.177 | [@laurimyllari #62](https://github.com/noonghunna/club-3090/discussions/62#discussioncomment-16832066) |
|
||||
| 4090 | air | llama.cpp default | Qwen3.6 27B Q3_K_XL | 300W | 50.26 | 50.02 | 0.168 | [@laurimyllari #62](https://github.com/noonghunna/club-3090/discussions/62#discussioncomment-16832066) |
|
||||
@@ -198,7 +202,32 @@ The cross-workload pattern: **both workload classes have efficiency knee at 400W
|
||||
|
||||
*3090 air-cooled + Qwen3.6-27B Q3_K_XL + mainline llama.cpp, 21-cap **prefill-heavy** sweep 190-390W via adaptive prompt calibration (probe TPS at 390W → size prompt for 10s prefill at high cap → 11K-token prompt used across all caps). **Total wall: ~6m.** Yellow callout: **250W sweet spot (3.633 prefill TPS/W, SM 1350 MHz)** at **68% of stock TDP** — different sweet spot than decode-single's 290W on the same rig because prefill is more compute-bound and reaches diminishing returns earlier. Boost-clock plateau visible at 330-370W: SM clock locks at 1605-1620 MHz across all five caps with identical 327W draw + 1050 prefill TPS. Plateau escapes at 380W → SM 1665 MHz, draw 355W, TPS 1080. Companion to the decode chart above; together they show **same card has different power-knee for different workload class** — and both workloads share the same firmware boost-clock plateau pattern, just with slightly different clock setpoints.*
|
||||
|
||||
**Cross-rig pattern**: efficiency knee falls at **~60-85% of stock TDP** across consumer Ampere/Ada — start there for a new card class and zoom in. Ada (4090) is proportionally more aggressive than Ampere (3090) — 4090 cuts 33% of stock TDP for ~7% TPS loss; 3090 cuts 15% of stock for ~5% loss.
|
||||
#### Same hardware, MoE workload — sweet spot shifts 80W lower for decode
|
||||
|
||||
Running the same sweep on **Qwen3.6-35B-A3B (MoE, 3B active params per token)** on the same 3090 GPU 0 reveals that **model architecture moves the sweet spot meaningfully**:
|
||||
|
||||
|
||||
|
||||
*3090 air-cooled + Qwen3.6-35B-A3B Q4_K_XL + mainline llama.cpp, 21-cap decode-single sweep, time-bounded bench. **Total wall: ~8m.** Yellow callout: **210W sweet spot (0.546 TPS/W, SM 1290 MHz)** at **57% of stock 370W TDP** — that's **80W lower than the dense Qwen3.6-27B sweet spot at 290W** on the same hardware. Purple-shaded zone 340-370W: **NO boost-clock plateau** — SM clock climbs smoothly 1875→1890→1890→1905 across that cap range (vs the dense Qwen which locks at exactly 1560 MHz). Plateau auto-detection correctly flagged dense Qwen but did NOT flag A3B. Source script: [`img/power-cap-3090-a3b-decode.py`](img/power-cap-3090-a3b-decode.py).*
|
||||
|
||||
|
||||
|
||||
*3090 air-cooled + A3B Q4_K_XL + mainline llama.cpp at -c 65536, 21-cap **prefill-heavy** sweep with adaptive prompt calibration (~31K-token prompt sized for 10s prefill at 390W cap). **Total wall: ~6m.** Yellow callout: **250W sweet spot (9.865 prefill TPS/W, SM 1380 MHz)** — **same cap as dense Qwen3.6-27B prefill** (also 250W). Boost-clock plateau auto-detected at 340-370W: SM 1680-1710 MHz, 334W draw, 2802 TPS. So both dense and MoE share the prefill plateau pattern, just at different SM clock setpoints. Source script: [`img/power-cap-3090-a3b-prefill.py`](img/power-cap-3090-a3b-prefill.py).*
|
||||
|
||||
**Two findings from this comparison** (same hardware, same engine, same Q4-class quant, only model changes):
|
||||
|
||||
| Workload | Dense (27B) sweet spot | MoE (A3B) sweet spot | Plateau on dense? | Plateau on MoE? |
|
||||
|---|---:|---:|:---:|:---:|
|
||||
| **Decode-single** | 290W | **210W** (–80W) | ✅ SM 1560 MHz | ❌ no plateau |
|
||||
| **Prefill-heavy** | 250W | **250W** (same) | ✅ SM 1605-1620 | ✅ SM 1680-1710 |
|
||||
|
||||
1. **MoE shifts the decode sweet spot 80W lower** because each token only activates 3B of the 35B params — much less per-token compute than dense, so the bandwidth-bound knee fires at lower power. The prefill sweet spot stays at 250W on both because prefill is compute-bound regardless (the full per-layer matmul still happens, MoE routing only affects which experts compute).
|
||||
|
||||
2. **Boost-clock plateau is workload-AND-model dependent**: dense decode has it (SM 1560 MHz lock 340-370W), A3B decode does not (SM climbs smoothly). The firmware's choice of operating point responds to the instruction-mix profile, not just to the cap value. Both models exhibit the plateau on prefill (where compute pressure is high), neither model exhibits it for short-prompt decode on smaller compute pressure.
|
||||
|
||||
**Practical implication**: cap recommendations should be model-class aware. A user running A3B as their primary chat model on 3090 should cap at **210W** (49 W less than dense Qwen's 290W recommendation) and gain the same efficiency win plus a 5°C cooler operating temperature.
|
||||
|
||||
**Cross-rig pattern**: efficiency knee falls at **~60-85% of stock TDP** across consumer Ampere/Ada — start there for a new card class and zoom in. Ada (4090) is proportionally more aggressive than Ampere (3090) — 4090 cuts 33% of stock TDP for ~7% TPS loss; 3090 cuts 15% of stock for ~5% loss. **MoE-class models lower this further**: A3B sweet spot at 57% of stock TDP for decode (vs 78% for dense on the same card).
|
||||
|
||||
**5090 compute-saturation note**: @apnar's data shows the 5090 caps at ~430W actual draw on Qwen3.6-27B even when allowed up to 575W — the workload is compute-saturated, not power-saturated. So 400W cap delivers ~equal TPS to 575W. **Confirmed cross-workload on Gemma 4 31B + MTP**: 21-cap sweep at 10W resolution shows actual draw plateaus at ~547W beyond 530W cap (no thermal throttle, GPU temp peaked 66°C — compute / memory bandwidth limit, not thermal). **Same 400W sweet spot** despite ~5× different absolute TPS class. Pattern: the 5090 + consumer-air-cooled platform appears to have a workload-independent ~400W efficiency knee on this rig class.
|
||||
|
||||
|
||||
BIN
docs/img/power-cap-3090-a3b-decode.png
Normal file
BIN
docs/img/power-cap-3090-a3b-decode.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 240 KiB |
156
docs/img/power-cap-3090-a3b-decode.py
Normal file
156
docs/img/power-cap-3090-a3b-decode.py
Normal file
@@ -0,0 +1,156 @@
|
||||
"""Generate 3090 A3B MoE decode-single power-cap chart.
|
||||
|
||||
Source data: 2026-05-08 sweep, dual-3090 rig (GPU 0 used), air-cooled.
|
||||
Engine: mainline llama.cpp (ghcr.io/ggml-org/llama.cpp:server-cuda) +
|
||||
Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf (MoE, 3B active per token), single-stream.
|
||||
|
||||
Companion to power-cap-3090-qwen36.png (dense Qwen3.6-27B on the same rig).
|
||||
The headline finding: same hardware, MoE workload shifts sweet spot
|
||||
from 290W (dense) → 210W (MoE) — an 80W gap driven by the lower
|
||||
compute-per-token of the MoE path (bandwidth becomes the binding
|
||||
constraint earlier).
|
||||
|
||||
Notable difference from the dense decode chart: NO firmware boost-clock
|
||||
plateau in the 340-370W range. SM clock climbs smoothly across that
|
||||
range (1875 → 1890 → 1890 → 1905) instead of locking at a single value.
|
||||
Plateau auto-detection correctly flagged dense Qwen but did NOT flag
|
||||
A3B — confirming that firmware operating-point selection responds to
|
||||
the model's compute profile, not just to the cap value.
|
||||
"""
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
# (cap_W, narr_TPS, code_TPS, actual_W, sm_clk_MHz, eff_TPS_per_W) — 21-cap clean sweep
|
||||
data = [
|
||||
(190, 92.20, 91.60, 189.72, 900, 0.486),
|
||||
(200, 104.09, 104.08, 199.73, 1095, 0.521),
|
||||
(210, 114.59, 113.79, 209.71, 1290, 0.546), # ⭐ sweet spot
|
||||
(220, 119.46, 119.07, 219.70, 1425, 0.544),
|
||||
(230, 122.57, 122.68, 229.69, 1530, 0.534),
|
||||
(240, 124.78, 124.95, 239.63, 1590, 0.521),
|
||||
(250, 127.16, 126.99, 249.55, 1635, 0.510),
|
||||
(260, 128.45, 128.56, 259.55, 1680, 0.495),
|
||||
(270, 130.26, 129.97, 269.37, 1710, 0.484),
|
||||
(280, 131.46, 131.14, 279.26, 1740, 0.471),
|
||||
(290, 132.23, 132.25, 289.27, 1755, 0.457),
|
||||
(300, 133.27, 133.23, 298.96, 1785, 0.446),
|
||||
(310, 134.07, 134.07, 309.19, 1815, 0.434),
|
||||
(320, 134.94, 135.06, 318.88, 1845, 0.423),
|
||||
(330, 135.54, 135.65, 328.96, 1860, 0.412),
|
||||
(340, 136.25, 135.84, 338.71, 1875, 0.402),
|
||||
(350, 136.55, 136.35, 348.73, 1890, 0.392),
|
||||
(360, 136.66, 136.65, 354.04, 1890, 0.386),
|
||||
(370, 136.84, 136.65, 354.23, 1905, 0.386),
|
||||
(380, 137.26, 137.46, 377.33, 1920, 0.364),
|
||||
(390, 137.73, 137.85, 385.80, 1935, 0.357),
|
||||
]
|
||||
|
||||
caps = [d[0] for d in data]
|
||||
narr = [d[1] for d in data]
|
||||
code = [d[2] for d in data]
|
||||
draw = [d[3] for d in data]
|
||||
sm_clk = [d[4] for d in data]
|
||||
eff = [d[5] for d in data]
|
||||
|
||||
plt.rcParams.update({
|
||||
"font.family": "sans-serif",
|
||||
"font.size": 12,
|
||||
"axes.titlesize": 16,
|
||||
"axes.titleweight": "bold",
|
||||
"axes.labelsize": 13,
|
||||
"figure.facecolor": "white",
|
||||
"axes.facecolor": "white",
|
||||
})
|
||||
|
||||
fig, ax1 = plt.subplots(figsize=(11, 6.4), dpi=150)
|
||||
|
||||
# Left axis: TPS
|
||||
color_narr = "#7b3fa0"
|
||||
color_code = "#1f77b4"
|
||||
ax1.plot(caps, narr, "o-", color=color_narr, linewidth=2.2, markersize=6,
|
||||
label="Narrative TPS (A3B MoE)", zorder=3)
|
||||
ax1.plot(caps, code, "s-", color=color_code, linewidth=2.2, markersize=6,
|
||||
label="Code TPS (A3B MoE)", zorder=3)
|
||||
ax1.set_xlabel("Power cap (W)", fontsize=13)
|
||||
ax1.set_ylabel("Wall TPS (single-stream, llama.cpp mainline)", fontsize=13)
|
||||
ax1.set_xlim(185, 395)
|
||||
ax1.set_ylim(85, 145)
|
||||
ax1.grid(True, alpha=0.3, zorder=0)
|
||||
ax1.tick_params(axis="both", labelsize=11)
|
||||
|
||||
# Right axis: TPS/W efficiency
|
||||
ax2 = ax1.twinx()
|
||||
color_eff = "#d62728"
|
||||
ax2.plot(caps, eff, "^--", color=color_eff, linewidth=1.8, markersize=5,
|
||||
alpha=0.9, label="Efficiency (narr TPS/W)", zorder=2)
|
||||
ax2.set_ylabel("Efficiency: TPS/W (narrative)", color=color_eff, fontsize=13)
|
||||
ax2.tick_params(axis="y", labelcolor=color_eff, labelsize=11)
|
||||
ax2.set_ylim(0.34, 0.58)
|
||||
|
||||
# Sweet spot annotation: 210W
|
||||
ax1.axvline(210, color="goldenrod", linestyle=":", alpha=0.5, linewidth=1.5)
|
||||
ax1.annotate(
|
||||
"★ 210W cap\n0.546 TPS/W (best efficiency)\n114.6 narr / 113.8 code\nSM 1290 MHz, 57% of stock TDP\n→ 80W lower than dense Qwen3.6-27B",
|
||||
xy=(210, 114.59),
|
||||
xytext=(225, 88),
|
||||
fontsize=10.5,
|
||||
fontweight="bold",
|
||||
bbox=dict(boxstyle="round,pad=0.4", facecolor="#fff3cd", edgecolor="goldenrod", linewidth=1.2),
|
||||
arrowprops=dict(arrowstyle="->", color="goldenrod", lw=1.5),
|
||||
zorder=4,
|
||||
)
|
||||
|
||||
# No-plateau annotation
|
||||
ax1.axvspan(335, 375, alpha=0.10, color="#9b59b6", zorder=0)
|
||||
ax1.text(355, 142.5, "no boost-clock plateau\n(SM climbs 1875→1905,\nunlike dense Qwen at this range)",
|
||||
fontsize=9.5, ha="center", color="#5b3578", fontstyle="italic")
|
||||
|
||||
# Stock TDP marker at 370W
|
||||
ax1.axvline(370, color="#888", linestyle="--", alpha=0.6, linewidth=1.2)
|
||||
ax1.annotate(
|
||||
"stock TDP\n370W (GPU 0)",
|
||||
xy=(370, 137),
|
||||
xytext=(372, 137.5),
|
||||
fontsize=10,
|
||||
ha="left",
|
||||
color="#555",
|
||||
fontstyle="italic",
|
||||
)
|
||||
|
||||
# Compare with dense Qwen sweet spot
|
||||
ax1.text(295, 87, "(compare: dense Qwen3.6-27B sweet spot at 290W on same rig)",
|
||||
fontsize=9, ha="center", color="#666", fontstyle="italic")
|
||||
|
||||
# Title
|
||||
ax1.set_title(
|
||||
"RTX 3090 + Qwen3.6-35B-A3B (MoE) + llama.cpp — power-cap efficiency curve",
|
||||
pad=14,
|
||||
)
|
||||
|
||||
# Subtitle
|
||||
fig.text(
|
||||
0.5, 0.92,
|
||||
"1× 3090 air-cooled (GPU 0 of dual-3090 rig), mainline llama.cpp + A3B Q4_K_XL GGUF, "
|
||||
"time-bounded single-stream | data: @noonghunna",
|
||||
ha="center", fontsize=10, color="#666",
|
||||
style="italic",
|
||||
)
|
||||
|
||||
# Combined legend
|
||||
lines1, labels1 = ax1.get_legend_handles_labels()
|
||||
lines2, labels2 = ax2.get_legend_handles_labels()
|
||||
ax1.legend(lines1 + lines2, labels1 + labels2,
|
||||
loc="lower right", fontsize=11, framealpha=0.95,
|
||||
edgecolor="#ccc")
|
||||
|
||||
# Footer
|
||||
fig.text(
|
||||
0.99, 0.01,
|
||||
"github.com/noonghunna/club-3090",
|
||||
ha="right", fontsize=9, color="#888", style="italic",
|
||||
)
|
||||
|
||||
plt.tight_layout(rect=(0, 0.02, 1, 0.92))
|
||||
|
||||
out = "/tmp/power_cap_sweep_3090_a3b_decode.png"
|
||||
plt.savefig(out, dpi=150, bbox_inches="tight", facecolor="white")
|
||||
print(f"Saved: {out}")
|
||||
BIN
docs/img/power-cap-3090-a3b-prefill.png
Normal file
BIN
docs/img/power-cap-3090-a3b-prefill.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 254 KiB |
160
docs/img/power-cap-3090-a3b-prefill.py
Normal file
160
docs/img/power-cap-3090-a3b-prefill.py
Normal file
@@ -0,0 +1,160 @@
|
||||
"""Generate 3090 A3B MoE prefill-heavy power-cap chart.
|
||||
|
||||
Source data: 2026-05-08 sweep, 1× 3090 air-cooled (GPU 0).
|
||||
Engine: mainline llama.cpp + Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf at -c 65536.
|
||||
Methodology: power-cap-sweep --load-mode prefill-heavy with adaptive prompt
|
||||
calibration (probe at 390W, ~31K-token prompt sized for 10s prefill at high
|
||||
cap). Calibration probe: 10042 tok in 3.216s = 3122.51 TPS at 390W.
|
||||
|
||||
Companion to power-cap-3090-qwen36-prefill.png (dense Qwen3.6-27B prefill
|
||||
on the same rig). Headline: prefill sweet spot is 250W on BOTH dense and
|
||||
A3B MoE — same hardware, same workload class, same firmware optimum.
|
||||
This contrasts with decode-single, where MoE shifts the sweet spot from
|
||||
290W (dense) to 210W (MoE). Prefill is more compute-bound, so the knee
|
||||
location is determined by hardware compute regime rather than model
|
||||
architecture.
|
||||
|
||||
Boost-clock plateau detected at 340-370W: SM 1680-1710 MHz, 334W draw,
|
||||
2802 TPS (auto-detected by power-cap-sweep.sh).
|
||||
"""
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
# (cap_W, prefill_TPS, actual_W, sm_clk_MHz, eff_TPS_per_W) — 21-cap clean sweep
|
||||
data = [
|
||||
(190, 1542.94, 189.74, 720, 8.132),
|
||||
(200, 1690.30, 199.69, 855, 8.465),
|
||||
(210, 1860.96, 209.80, 945, 8.870),
|
||||
(220, 2039.30, 219.69, 1095, 9.283),
|
||||
(230, 2210.92, 229.54, 1215, 9.632),
|
||||
(240, 2356.70, 239.73, 1320, 9.831),
|
||||
(250, 2461.22, 249.49, 1380, 9.865), # ⭐ sweet spot
|
||||
(260, 2523.67, 259.46, 1440, 9.727),
|
||||
(270, 2558.30, 269.56, 1485, 9.491),
|
||||
(280, 2611.96, 279.34, 1530, 9.350),
|
||||
(290, 2657.45, 288.95, 1560, 9.197),
|
||||
(300, 2699.15, 299.10, 1605, 9.024),
|
||||
(310, 2726.33, 308.69, 1635, 8.832),
|
||||
(320, 2763.36, 318.62, 1650, 8.673),
|
||||
(330, 2786.61, 328.53, 1665, 8.482),
|
||||
(340, 2802.41, 334.05, 1680, 8.389), # ←┐
|
||||
(350, 2801.40, 334.89, 1710, 8.365), # │ boost-clock plateau (auto-detected)
|
||||
(360, 2797.63, 333.89, 1695, 8.379), # │ SM 1680-1710 MHz lock,
|
||||
(370, 2794.36, 334.15, 1695, 8.363), # ←┘ 334W draw, 2802 TPS
|
||||
(380, 2866.93, 361.98, 1740, 7.920),
|
||||
(390, 2900.05, 387.58, 1785, 7.482),
|
||||
]
|
||||
|
||||
caps = [d[0] for d in data]
|
||||
tps = [d[1] for d in data]
|
||||
draw = [d[2] for d in data]
|
||||
sm_clk = [d[3] for d in data]
|
||||
eff = [d[4] for d in data]
|
||||
|
||||
plt.rcParams.update({
|
||||
"font.family": "sans-serif",
|
||||
"font.size": 12,
|
||||
"axes.titlesize": 16,
|
||||
"axes.titleweight": "bold",
|
||||
"axes.labelsize": 13,
|
||||
"figure.facecolor": "white",
|
||||
"axes.facecolor": "white",
|
||||
})
|
||||
|
||||
fig, ax1 = plt.subplots(figsize=(11, 6.4), dpi=150)
|
||||
|
||||
# Left axis: prefill TPS
|
||||
color_tps = "#7b3fa0"
|
||||
ax1.plot(caps, tps, "o-", color=color_tps, linewidth=2.2, markersize=6,
|
||||
label="Prefill TPS (A3B MoE, compute-bound)", zorder=3)
|
||||
ax1.set_xlabel("Power cap (W)", fontsize=13)
|
||||
ax1.set_ylabel("Prefill TPS (~31K-token prompt + max_tokens=10)", fontsize=13)
|
||||
ax1.set_xlim(185, 395)
|
||||
ax1.set_ylim(1400, 3000)
|
||||
ax1.grid(True, alpha=0.3, zorder=0)
|
||||
ax1.tick_params(axis="both", labelsize=11)
|
||||
|
||||
# Right axis: efficiency
|
||||
ax2 = ax1.twinx()
|
||||
color_eff = "#d62728"
|
||||
ax2.plot(caps, eff, "^--", color=color_eff, linewidth=1.8, markersize=5,
|
||||
alpha=0.9, label="Efficiency (prefill TPS/W)", zorder=2)
|
||||
ax2.set_ylabel("Efficiency: prefill TPS/W", color=color_eff, fontsize=13)
|
||||
ax2.tick_params(axis="y", labelcolor=color_eff, labelsize=11)
|
||||
ax2.set_ylim(7.3, 10.0)
|
||||
|
||||
# Sweet spot annotation: 250W
|
||||
ax1.axvline(250, color="goldenrod", linestyle=":", alpha=0.5, linewidth=1.5)
|
||||
ax1.annotate(
|
||||
"★ 250W cap\n9.865 TPS/W (best efficiency)\n2461 prefill TPS\nSM 1380 MHz, 68% of stock TDP\n→ same as dense Qwen3.6-27B prefill sweet spot",
|
||||
xy=(250, 2461.22),
|
||||
xytext=(265, 1700),
|
||||
fontsize=10.5,
|
||||
fontweight="bold",
|
||||
bbox=dict(boxstyle="round,pad=0.4", facecolor="#fff3cd", edgecolor="goldenrod", linewidth=1.2),
|
||||
arrowprops=dict(arrowstyle="->", color="goldenrod", lw=1.5),
|
||||
zorder=4,
|
||||
)
|
||||
|
||||
# Boost-clock plateau region (340-370W → SM 1680-1710, 334W draw, 2802 TPS)
|
||||
ax1.axvspan(335, 375, alpha=0.10, color="orange", zorder=0)
|
||||
ax1.text(355, 1430, "boost-clock plateau\n(caps 340-370W → SM locked at 1680-1710 MHz,\n334W draw, 2802 prefill TPS)",
|
||||
fontsize=9.5, ha="center", color="#aa5500", fontstyle="italic")
|
||||
|
||||
# Stock TDP marker
|
||||
ax1.axvline(370, color="#888", linestyle="--", alpha=0.6, linewidth=1.2)
|
||||
ax1.annotate(
|
||||
"stock TDP\n370W (GPU 0)",
|
||||
xy=(370, 2900),
|
||||
xytext=(372, 2920),
|
||||
fontsize=10,
|
||||
ha="left",
|
||||
color="#555",
|
||||
fontstyle="italic",
|
||||
)
|
||||
|
||||
# Plateau-escape annotation at 380W
|
||||
ax1.annotate(
|
||||
"plateau escape:\nSM jumps 1710→1740 MHz",
|
||||
xy=(380, 2866.93),
|
||||
xytext=(330, 2950),
|
||||
fontsize=9,
|
||||
color="#aa5500",
|
||||
fontstyle="italic",
|
||||
arrowprops=dict(arrowstyle="->", color="#aa5500", lw=0.9, alpha=0.7),
|
||||
zorder=4,
|
||||
)
|
||||
|
||||
# Title
|
||||
ax1.set_title(
|
||||
"RTX 3090 + Qwen3.6-35B-A3B (MoE) + llama.cpp — prefill-heavy power-cap curve",
|
||||
pad=14,
|
||||
)
|
||||
|
||||
# Subtitle
|
||||
fig.text(
|
||||
0.5, 0.92,
|
||||
"1× 3090 air-cooled, mainline llama.cpp + A3B Q4_K_XL GGUF (-c 65536), adaptive prompt sizing "
|
||||
"(31K tokens calibrated at 390W cap) | data: @noonghunna",
|
||||
ha="center", fontsize=10, color="#666",
|
||||
style="italic",
|
||||
)
|
||||
|
||||
# Combined legend
|
||||
lines1, labels1 = ax1.get_legend_handles_labels()
|
||||
lines2, labels2 = ax2.get_legend_handles_labels()
|
||||
ax1.legend(lines1 + lines2, labels1 + labels2,
|
||||
loc="lower right", fontsize=11, framealpha=0.95,
|
||||
edgecolor="#ccc")
|
||||
|
||||
# Footer
|
||||
fig.text(
|
||||
0.99, 0.01,
|
||||
"github.com/noonghunna/club-3090",
|
||||
ha="right", fontsize=9, color="#888", style="italic",
|
||||
)
|
||||
|
||||
plt.tight_layout(rect=(0, 0.02, 1, 0.92))
|
||||
|
||||
out = "/tmp/power_cap_sweep_3090_a3b_prefill.png"
|
||||
plt.savefig(out, dpi=150, bbox_inches="tight", facecolor="white")
|
||||
print(f"Saved: {out}")
|
||||
Reference in New Issue
Block a user