alex/club-3090
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(README): correct 35B-A3B llama.cpp ❌→✅ (mainline serves the MoE; #390)

Browse Source 
deucebucket (#390) ran qwen3.6-35b-a3b on mainline llama.cpp single-3090
(141/144 TPS, 15.1GB @131K). The README support table marked llama.cpp ❌
for this MoE, conflating 'no shipped catalog compose' with 'unsupported' —
contradicted by our own docs/HARDWARE.md mainline power-cap curves on this
exact model and the llama-cpp-mainline engine profile (qwen3-next-moe).
Note added that ik_llama is the shipped single-card path.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
noonghunna
2026-06-12 22:59:37 +00:00
co-authored by Claude Opus 4.8
parent 520f5b909b
commit 559a3febf7
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -112,7 +112,7 @@ Each hardware page lists every supported model with the working composes for tha
|---|---|---|---|---|
| **[Qwen3.6-27B](models/qwen3.6-27b/)** | Production-ready ⭐ | 1× / 2× 3090 | vLLM ✅ · llama.cpp ✅ · ik_llama ✅ | Vision · tools · MTP n=3 · up to 262K ctx · vLLM dual = 89/127 TPS · llama.cpp single = 200K max-safe, no prefill cliffs · ik_llama IQ4_KS = ~60/69 TPS (fastest single-card) |
| **[Gemma 4 31B](models/gemma-4-31b/)** | Production-ready | 1× ¹ / 2× 3090 | vLLM ✅ (dual) · llama.cpp ⚠️ (community fork; mainline blocked on FA hdim=512) | Vision · tools · MTP n=3 (Google official drafter) **OR** DFlash n=7 (z-lab drafter) · up to 262K ctx via INT8 PTH KV (PR [#40391](https://github.com/vllm-project/vllm/pull/40391) vendored) · MTP dual = 106/141 TPS at 32K, 95/126 at 262K · DFlash dual = 105/177 TPS at 32K (code-optimal) · **beellama.cpp = single-card default** ⭐ 47/88 TPS, 131–200K ctx, 109–114/150 8-pack ([discussion #239](https://github.com/noonghunna/club-3090/discussions/239)) |
| **[Qwen3.6 35B-A3B](models/qwen3.6-35b-a3b/)** ⭐ NEW v0.7.3 | Production-ready (ik-llama single-card) · Preview (vLLM dual) | 1× / 2× 3090 | vLLM ✅ (preview) · ik_llama ✅ · llama.cpp ❌ | **MoE (256 experts × 8 active, ~3 B active params)** · vision · tools · **ik_llama `fit-mtp.yml` single-card (Mudler APEX I-Compact)** = 103/149 TPS at 196K, hermes 11/20 + aider 12/30 + cli 12/40 ([PR #243](https://github.com/noonghunna/club-3090/pull/243)) · vLLM preview dual = 182/177 TPS at 16K (no MTP, no TQ3, no Genesis) |
| **[Qwen3.6 35B-A3B](models/qwen3.6-35b-a3b/)** ⭐ NEW v0.7.3 | Production-ready (ik-llama single-card) · Preview (vLLM dual) | 1× / 2× 3090 | vLLM ✅ (preview) · ik_llama ✅ · llama.cpp ✅ (mainline runs it — see `docs/HARDWARE.md`; ik_llama is the shipped single-card path) | **MoE (256 experts × 8 active, ~3 B active params)** · vision · tools · **ik_llama `fit-mtp.yml` single-card (Mudler APEX I-Compact)** = 103/149 TPS at 196K, hermes 11/20 + aider 12/30 + cli 12/40 ([PR #243](https://github.com/noonghunna/club-3090/pull/243)) · vLLM preview dual = 182/177 TPS at 16K (no MTP, no TQ3, no Genesis) |
| **[Gemma 4 26B-A4B](models/gemma-4-26b-a4b/)** ⭐ NEW v0.7.3 | Production via AWQ (Intel AutoRound INT4 blocked on Ampere) | 2× 3090 ² | vLLM ✅ (AWQ overlay) · llama.cpp ❌ | **MoE (128 experts × 8 active, ~4 B active params)** · vision · tools · AWQ dual = **139/139 TPS at 32K**, CV 0.2% / 0.0% |
¹ Single-card Gemma 4 on Ampere 24 GB: vLLM + mainline llama.cpp are blocked (head_dim=512 FA wall, no Ampere FA kernel yet). The community **beellama.cpp** fork builds with `FA_ALL_QUANTS=ON` and runs cleanly on a single 3090 — 47/88 TPS, 100–150K ctx, 109/114 8-pack (validated 2026-05-27). 32 GB+ GPUs run the standard vLLM path (validated on RTX 5090 32 GB by [@apnar](https://github.com/noonghunna/club-3090/discussions/67#discussioncomment-16832042)).