From e1137d6889bbb6c7d7ddb584c942488f654c3b86 Mon Sep 17 00:00:00 2001 From: noonghunna <10742901+noonghunna@users.noreply.github.com> Date: Sat, 9 May 2026 12:19:44 +0000 Subject: [PATCH] composes: formalize Status enum + Caveats field (100% coverage) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous "Status optional, only when not production" convention left readers guessing whether absence-of-status meant "validated production" or "author forgot to fill it in." Making Status required + enumerated removes that ambiguity. Schema enhancement (codified in AGENTS.md): - Status: now REQUIRED, exactly one of: ✅ Production — verify-full + stress + bench + soak PASS ⚠️ Production w/ caveats — works under documented constraints 🧪 Experimental — under active validation 👁️ Preview — known quality issues, not production ⏸️ Upstream-gated — blocked by external action (PR/driver) 🗑️ Deprecated — kept for historical reference - Caveats: REQUIRED if Status is ⚠️/👁️/⏸️/🗑️, OMITTED for ✅/🧪. Single-line summary or short bullet list with issue/PR links. Status applied to all 27 composes (vLLM + llama-cpp, all topologies): ✅ Production: 20 (canonical Qwen + Gemma + llama.cpp + gemma-awq) ⚠️ Production w/ caveats: 4 (long-text*, long-vision, carnice-bf16mtp) ⏸️ Upstream-gated: 2 (gemma single boot-OOM, gemma dflash-int8 needs #42102) 👁️ Preview: 1 (qwopus-bf16mtp) 🧪 Experimental: 0 🗑️ Deprecated: 0 llama-cpp composes also gained `Genesis: N/A — llama.cpp engine` for parity with the vLLM schema (Genesis is Qwen3-Next-specific and vLLM-only; explicit N/A prevents future readers from looking for patches that don't exist). Co-Authored-By: Claude Opus 4.7 (1M context) --- AGENTS.md | 18 +++++++++++++++++- .../gemma-4-31b/vllm/compose/dual/dflash.yml | 1 + .../vllm/compose/dual/docker-compose.yml | 1 + models/gemma-4-31b/vllm/compose/dual/int8.yml | 1 + .../vllm/compose/single/docker-compose.yml | 2 ++ .../llama-cpp/compose/single/concurrent.yml | 2 ++ .../compose/single/docker-compose.yml | 2 ++ .../vllm/compose/dual/carnice-bf16mtp.yml | 3 ++- .../vllm/compose/dual/dflash-noviz.yml | 1 + .../qwen3.6-27b/vllm/compose/dual/dflash.yml | 1 + .../vllm/compose/dual/docker-compose.yml | 1 + .../vllm/compose/dual/nvlink-dflash-noviz.yml | 1 + .../vllm/compose/dual/nvlink-dflash.yml | 1 + .../vllm/compose/dual/nvlink-turbo.yml | 1 + .../qwen3.6-27b/vllm/compose/dual/nvlink.yml | 1 + models/qwen3.6-27b/vllm/compose/dual/turbo.yml | 1 + .../qwen3.6-27b/vllm/compose/multi4/dflash.yml | 1 + .../vllm/compose/multi4/docker-compose.yml | 1 + .../vllm/compose/single/bounded-thinking.yml | 1 + .../vllm/compose/single/docker-compose.yml | 1 + .../vllm/compose/single/long-text-no-mtp.yml | 3 ++- .../vllm/compose/single/long-text.yml | 3 ++- .../vllm/compose/single/long-vision.yml | 3 ++- .../vllm/compose/single/minimal.yml | 1 + .../vllm/compose/single/tools-text.yml | 1 + 25 files changed, 48 insertions(+), 5 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index df96cda9..a90fdfd7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -113,12 +113,28 @@ Every compose starts with a `Profile (at-a-glance)` block declaring the (Model, # Vision: # Max ctx: # Genesis: -# Status: +# Status: +# Caveats: # Best for: # --------------------------------------------------------------------------- # (existing free-form description continues below) ``` +**Status enum** — pick exactly one: + +| Value | Meaning | Validation gate | +|---|---|---| +| `✅ Production` | Recommended for users. | verify-full 8/8 + verify-stress 7/7 + bench (BENCHMARKS row) + soak-continuous PASS. | +| `⚠️ Production w/ caveats` | Works under documented constraints; not the same as broken. | Same gates as Production, but a known-and-disclosed limitation exists (e.g., Cliff 2b at >50K). Caveats line MUST list the constraint. | +| `🧪 Experimental` | Under active validation; may not boot or pass all tests. | Typically untracked in git. No production guarantee. | +| `👁️ Preview` | Known quality issues; tracked but not for production. | E.g., quality regressions in soak / NIAH. Caveats line MUST list specific issues. | +| `⏸️ Upstream-gated` | Exists but blocked by external action (PR merge, driver fix, hardware ceiling). | Boots only with vendored override OR doesn't boot until external dep lands. Caveats line MUST point at the external dep. | +| `🗑️ Deprecated` | Kept for historical reference; will be removed. | N/A — flagged for cleanup. | + +**Why this enum exists**: the previous "Status optional, only when not production" convention left readers guessing whether absence-of-status meant "validated production" or "author forgot to fill it in." Making Status required + enumerated removes that ambiguity. Users picking a config can scan to one field and know the lifecycle stage instantly; new contributors must consciously declare it when authoring. + +The `Caveats:` line is REQUIRED whenever Status is ⚠️ / 👁️ / ⏸️ / 🗑️, OMITTED for ✅ / 🧪. Format: a single-line summary or a short bullet list, with links to issues / discussions / upstream PRs where relevant. + This rule applies to **shipped composes AND local-only test composes** — apply the convention even before deciding whether to ship; it avoids a rename later if the experiment graduates. When testing a new model, create the directory hierarchy from the start: `models///compose//docker-compose.yml`. The hierarchy enforces the convention; filenames encode only the feature stack within that topology. When the model isn't Qwen3-Next, write `Genesis: N/A — Genesis is Qwen3-Next-specific` in the profile schema so readers don't expect Genesis-style perf folds where they don't apply. diff --git a/models/gemma-4-31b/vllm/compose/dual/dflash.yml b/models/gemma-4-31b/vllm/compose/dual/dflash.yml index 02bc74ae..3e65a733 100644 --- a/models/gemma-4-31b/vllm/compose/dual/dflash.yml +++ b/models/gemma-4-31b/vllm/compose/dual/dflash.yml @@ -7,6 +7,7 @@ # Vision: yes # Max ctx: 32K (BF16 ceiling — for 262K code-optimal use dual-dflash-int8.yml) # Genesis: N/A — Genesis is Qwen3-Next-specific +# Status: ✅ Production # Best for: Peak code TPS on Gemma 4 (105/177 — +18% code over MTP) ⭐ # --------------------------------------------------------------------------- # Gemma-4-31B-it (Intel AutoRound INT4) + z-lab Gemma 4 DFlash drafter diff --git a/models/gemma-4-31b/vllm/compose/dual/docker-compose.yml b/models/gemma-4-31b/vllm/compose/dual/docker-compose.yml index c380752c..72e4323b 100644 --- a/models/gemma-4-31b/vllm/compose/dual/docker-compose.yml +++ b/models/gemma-4-31b/vllm/compose/dual/docker-compose.yml @@ -7,6 +7,7 @@ # Vision: yes # Max ctx: 32K (BF16 ceiling — for 262K use dual-int8.yml) # Genesis: N/A — Genesis is Qwen3-Next-specific +# Status: ✅ Production # Best for: Gemma 4 dual-card default — vision + tools + balanced TPS ⭐ # --------------------------------------------------------------------------- # Gemma-4-31B-it (Intel AutoRound INT4) + Google MTP "assistant" drafter diff --git a/models/gemma-4-31b/vllm/compose/dual/int8.yml b/models/gemma-4-31b/vllm/compose/dual/int8.yml index 0d170576..733725ca 100644 --- a/models/gemma-4-31b/vllm/compose/dual/int8.yml +++ b/models/gemma-4-31b/vllm/compose/dual/int8.yml @@ -7,6 +7,7 @@ # Vision: yes # Max ctx: 262K (8.2× over bf16 ceiling) — override MAX_NUM_SEQS=4 for 98K + 3.6× concurrency # Genesis: N/A — Genesis is Qwen3-Next-specific +# Status: ✅ Production # Best for: Gemma 4 long-context default — full 262K ctx ⭐ # --------------------------------------------------------------------------- # Gemma-4-31B-it (Intel AutoRound INT4) + Google MTP "assistant" drafter diff --git a/models/gemma-4-31b/vllm/compose/single/docker-compose.yml b/models/gemma-4-31b/vllm/compose/single/docker-compose.yml index a117bfd3..7d42da34 100644 --- a/models/gemma-4-31b/vllm/compose/single/docker-compose.yml +++ b/models/gemma-4-31b/vllm/compose/single/docker-compose.yml @@ -7,6 +7,8 @@ # Vision: yes # Max ctx: ~16K (theoretical) — boot fails before serving on 24 GB # Genesis: N/A — Genesis is Qwen3-Next-specific +# Status: ⏸️ Upstream-gated +# Caveats: Boot-OOMs on Ampere 24 GB regardless of KV format (weights+drafter+profiling at 8K ctx leaves no KV pool). Compose preserved for 32 GB+ GPUs (RTX 5090 32 GB validated by @apnar disc #67 — 160/215 TPS at 32K MTP). Re-evaluate when (a) vLLM adds Ampere-aware fp8 dispatch OR (b) PR #41745-style relaxation lands. # Best for: Single-card 32 GB+ GPUs (RTX 5090 32 GB validated by @apnar disc #67) # For 24 GB Ampere, use dual.yml (dual-card) instead. # --------------------------------------------------------------------------- diff --git a/models/qwen3.6-27b/llama-cpp/compose/single/concurrent.yml b/models/qwen3.6-27b/llama-cpp/compose/single/concurrent.yml index a0d418ba..472e79f6 100644 --- a/models/qwen3.6-27b/llama-cpp/compose/single/concurrent.yml +++ b/models/qwen3.6-27b/llama-cpp/compose/single/concurrent.yml @@ -7,6 +7,8 @@ # KV: q4_0 (4-bit packed) # Vision: yes (mmproj F16) # Max ctx: 192K pool / 4 parallel slots +# Genesis: N/A — llama.cpp engine; Genesis is vLLM/Qwen3-Next-specific +# Status: ✅ Production # Best for: Single-card multi-tenant llama.cpp — 4 concurrent agents # at smaller per-stream ctx; trade max-ctx for parallelism # --------------------------------------------------------------------------- diff --git a/models/qwen3.6-27b/llama-cpp/compose/single/docker-compose.yml b/models/qwen3.6-27b/llama-cpp/compose/single/docker-compose.yml index 91f80318..5c99d3ed 100644 --- a/models/qwen3.6-27b/llama-cpp/compose/single/docker-compose.yml +++ b/models/qwen3.6-27b/llama-cpp/compose/single/docker-compose.yml @@ -7,6 +7,8 @@ # KV: q4_0 (4-bit packed) # Vision: yes (mmproj F16) # Max ctx: 262K (full model native — no Cliff 1 / Cliff 2) +# Genesis: N/A — llama.cpp engine; Genesis is vLLM/Qwen3-Next-specific +# Status: ✅ Production # Best for: Bulletproof single-card path — slow decode (~21 TPS) but # cliff-immune; recommended fallback when vLLM hits OOM at long ctx # --------------------------------------------------------------------------- diff --git a/models/qwen3.6-27b/vllm/compose/dual/carnice-bf16mtp.yml b/models/qwen3.6-27b/vllm/compose/dual/carnice-bf16mtp.yml index 4e0bdc0d..aade6f39 100644 --- a/models/qwen3.6-27b/vllm/compose/dual/carnice-bf16mtp.yml +++ b/models/qwen3.6-27b/vllm/compose/dual/carnice-bf16mtp.yml @@ -8,7 +8,8 @@ # Vision: yes # Max ctx: 262K # Genesis: v7.72.2 (full PROD env stack) -# Status: AL=2.0 (vs Lorbus's 3.4-3.8) — TPS 72/80, working but suboptimal +# Status: ⚠️ Production w/ caveats +# Caveats: MTP AL=2.0 vs Lorbus's 3.4-3.8 (TPS 72/80, working but suboptimal). Carnice was uploaded with stripped MTP keys; we delta-merged the head. Three diagnostic steps queued in /opt/ai/club-3090-todo.md (Tier 2 Carnice plan). # Best for: Hermes-style agentic tool-calling specialization # --------------------------------------------------------------------------- # Carnice-V2-27B + BF16 MTP overlay — dual RTX 3090 variant. diff --git a/models/qwen3.6-27b/vllm/compose/dual/dflash-noviz.yml b/models/qwen3.6-27b/vllm/compose/dual/dflash-noviz.yml index eb4cf8fa..7307a765 100644 --- a/models/qwen3.6-27b/vllm/compose/dual/dflash-noviz.yml +++ b/models/qwen3.6-27b/vllm/compose/dual/dflash-noviz.yml @@ -7,6 +7,7 @@ # Vision: no (vision tower dropped — frees ~0.5 GB/card for ctx) # Max ctx: 200K (+15K vs dual-dflash.yml) # Genesis: none — DFlash drafter handles spec-decode independently +# Status: ✅ Production # Best for: Peak code TPS, no-vision, with extra context headroom ⭐ # --------------------------------------------------------------------------- # Dual-card DFlash text-only — TP=2 + DFlash N=5 + 200K ctx + NO vision. diff --git a/models/qwen3.6-27b/vllm/compose/dual/dflash.yml b/models/qwen3.6-27b/vllm/compose/dual/dflash.yml index 462c5797..0929cc2a 100644 --- a/models/qwen3.6-27b/vllm/compose/dual/dflash.yml +++ b/models/qwen3.6-27b/vllm/compose/dual/dflash.yml @@ -7,6 +7,7 @@ # Vision: yes # Max ctx: 185K # Genesis: none — DFlash drafter handles spec-decode independently +# Status: ✅ Production # Best for: Peak code TPS with vision (AL ~4.4 vs MTP's 3.4) ⭐ # --------------------------------------------------------------------------- # Dual-card DFlash — TP=2 + DFlash N=5 spec-decode + 185K ctx + vision. diff --git a/models/qwen3.6-27b/vllm/compose/dual/docker-compose.yml b/models/qwen3.6-27b/vllm/compose/dual/docker-compose.yml index 2782f507..96069247 100644 --- a/models/qwen3.6-27b/vllm/compose/dual/docker-compose.yml +++ b/models/qwen3.6-27b/vllm/compose/dual/docker-compose.yml @@ -7,6 +7,7 @@ # Vision: yes # Max ctx: 262K (237K single-prompt verified) # Genesis: none — intentionally Genesis-free (isolation control + fallback) +# Status: ✅ Production # Best for: General-purpose dual-card vision + tools + long-ctx default ⭐ # --------------------------------------------------------------------------- # Dual RTX 3090 — DEFAULT for 2× cards. Qwen3.6-27B + MTP n=3 + fp8 KV + vision. diff --git a/models/qwen3.6-27b/vllm/compose/dual/nvlink-dflash-noviz.yml b/models/qwen3.6-27b/vllm/compose/dual/nvlink-dflash-noviz.yml index 3fbc62f9..d5484bb3 100644 --- a/models/qwen3.6-27b/vllm/compose/dual/nvlink-dflash-noviz.yml +++ b/models/qwen3.6-27b/vllm/compose/dual/nvlink-dflash-noviz.yml @@ -7,6 +7,7 @@ # Vision: no (frees ~0.5 GB/card for ctx) # Max ctx: 200K (+15K vs dual-nvlink-dflash.yml; NVLink lifts max to 188K stable) # Genesis: none +# Status: ✅ Production # Best for: NVLink + peak code TPS, no-vision (max ctx headroom) # --------------------------------------------------------------------------- # Dual RTX 3090 with NVLink + DFlash text-only — TP=2 + DFlash N=5 + 200K ctx diff --git a/models/qwen3.6-27b/vllm/compose/dual/nvlink-dflash.yml b/models/qwen3.6-27b/vllm/compose/dual/nvlink-dflash.yml index b0dce805..e3e831f7 100644 --- a/models/qwen3.6-27b/vllm/compose/dual/nvlink-dflash.yml +++ b/models/qwen3.6-27b/vllm/compose/dual/nvlink-dflash.yml @@ -7,6 +7,7 @@ # Vision: yes # Max ctx: 185K # Genesis: none — DFlash drafter handles spec-decode independently +# Status: ✅ Production # Best for: NVLink + peak code TPS — +17% over PCIe-only dual-dflash.yml # --------------------------------------------------------------------------- # Dual RTX 3090 with NVLink + DFlash — TP=2 + DFlash N=5 spec-decode + 185K ctx diff --git a/models/qwen3.6-27b/vllm/compose/dual/nvlink-turbo.yml b/models/qwen3.6-27b/vllm/compose/dual/nvlink-turbo.yml index ef5d5eff..2b2531c9 100644 --- a/models/qwen3.6-27b/vllm/compose/dual/nvlink-turbo.yml +++ b/models/qwen3.6-27b/vllm/compose/dual/nvlink-turbo.yml @@ -7,6 +7,7 @@ # Vision: yes # Max ctx: 262K # Genesis: v7.72.2 (full PROD env stack) +# Status: ✅ Production # Best for: NVLink + multi-tenant — +11% narr / +12% code over PCIe-only dual-turbo.yml # --------------------------------------------------------------------------- # Dual-card Turbo with NVLink — TP=2 + TurboQuant KV (turboquant_3bit_nc) + MTP n=3 diff --git a/models/qwen3.6-27b/vllm/compose/dual/nvlink.yml b/models/qwen3.6-27b/vllm/compose/dual/nvlink.yml index 445a1159..77ae5c29 100644 --- a/models/qwen3.6-27b/vllm/compose/dual/nvlink.yml +++ b/models/qwen3.6-27b/vllm/compose/dual/nvlink.yml @@ -7,6 +7,7 @@ # Vision: yes # Max ctx: 262K # Genesis: v7.72.2 +# Status: ✅ Production # Best for: Users with the NVLink bridge — +15% narr / +15% code over PCIe-only dual.yml # --------------------------------------------------------------------------- # Dual RTX 3090 with NVLink — opt-in variant for users with the bridge installed. diff --git a/models/qwen3.6-27b/vllm/compose/dual/turbo.yml b/models/qwen3.6-27b/vllm/compose/dual/turbo.yml index ba12a6cc..d595533a 100644 --- a/models/qwen3.6-27b/vllm/compose/dual/turbo.yml +++ b/models/qwen3.6-27b/vllm/compose/dual/turbo.yml @@ -7,6 +7,7 @@ # Vision: yes # Max ctx: 262K # Genesis: v7.72.2 (PN59 streaming-GDN + PN34 workspace-lock-relax + PN12) +# Status: ✅ Production # Best for: Multi-tenant agent fleet — 4 streams at full ctx (4.67× concurrency vs dual.yml) ⭐ # --------------------------------------------------------------------------- # Dual-card Turbo — TP=2 + TurboQuant KV (turboquant_3bit_nc) + MTP n=3 + Genesis v7.69 dev. diff --git a/models/qwen3.6-27b/vllm/compose/multi4/dflash.yml b/models/qwen3.6-27b/vllm/compose/multi4/dflash.yml index 1872b1f1..1aed4691 100644 --- a/models/qwen3.6-27b/vllm/compose/multi4/dflash.yml +++ b/models/qwen3.6-27b/vllm/compose/multi4/dflash.yml @@ -7,6 +7,7 @@ # Vision: yes # Max ctx: 262K # Genesis: none — DFlash drafter handles spec-decode independently +# Status: ✅ Production # Best for: 4-card peak code TPS path # --------------------------------------------------------------------------- # Quad RTX 3090 DFlash — TP=4 + DFlash N=5 + 262K ctx + vision. diff --git a/models/qwen3.6-27b/vllm/compose/multi4/docker-compose.yml b/models/qwen3.6-27b/vllm/compose/multi4/docker-compose.yml index 8e51756b..f15c0f13 100644 --- a/models/qwen3.6-27b/vllm/compose/multi4/docker-compose.yml +++ b/models/qwen3.6-27b/vllm/compose/multi4/docker-compose.yml @@ -7,6 +7,7 @@ # Vision: yes # Max ctx: 262K # Genesis: v7.72.2 (full PROD env stack) +# Status: ✅ Production # Best for: 4-card TP=4 baseline — high concurrency, max KV pool size # --------------------------------------------------------------------------- # Quad RTX 3090 — TP=4 baseline. Qwen3.6-27B + MTP n=3 + fp8 KV + vision. diff --git a/models/qwen3.6-27b/vllm/compose/single/bounded-thinking.yml b/models/qwen3.6-27b/vllm/compose/single/bounded-thinking.yml index c5a08a8d..944f1b33 100644 --- a/models/qwen3.6-27b/vllm/compose/single/bounded-thinking.yml +++ b/models/qwen3.6-27b/vllm/compose/single/bounded-thinking.yml @@ -7,6 +7,7 @@ # Vision: yes # Max ctx: 180K # Genesis: v7.72.2 (full PROD env stack) +# Status: ✅ Production # Best for: Reasoning/coding workloads with bounded thinking budget # via FSM-constrained scratchpad grammar — caps think budget, # improves HE+/LCB scores, +24pp accuracy on max_tokens=4096 diff --git a/models/qwen3.6-27b/vllm/compose/single/docker-compose.yml b/models/qwen3.6-27b/vllm/compose/single/docker-compose.yml index f95dc62e..aab1a0f5 100644 --- a/models/qwen3.6-27b/vllm/compose/single/docker-compose.yml +++ b/models/qwen3.6-27b/vllm/compose/single/docker-compose.yml @@ -7,6 +7,7 @@ # Vision: yes # Max ctx: 48K (mem-util 0.92 — safe single-card default) # Genesis: v7.72.2 (full PROD env stack) +# Status: ✅ Production # Best for: Single-card default — vision + tools + MTP + Cliff-safe at # <48K ctx; recommended starting point for 1× 3090 # --------------------------------------------------------------------------- diff --git a/models/qwen3.6-27b/vllm/compose/single/long-text-no-mtp.yml b/models/qwen3.6-27b/vllm/compose/single/long-text-no-mtp.yml index efa0fa65..156cda6a 100644 --- a/models/qwen3.6-27b/vllm/compose/single/long-text-no-mtp.yml +++ b/models/qwen3.6-27b/vllm/compose/single/long-text-no-mtp.yml @@ -16,7 +16,8 @@ # Vision: no (text-only) # Max ctx: 200K (max-context single-card path) # Genesis: v7.72.2 -# Status: ⚠️ Cliff 2b applies at >50K single-prompt — see warning above +# Status: ⚠️ Production w/ caveats +# Caveats: Cliff 2b at >50K single-prompt context. Sandermage/genesis-vllm-patches#22. Workaround: dual.yml or llamacpp/default. # Best for: Max single-card context window for one-shot summarization / # long-document Q&A; slower decode (no spec-decode) is the trade # --------------------------------------------------------------------------- diff --git a/models/qwen3.6-27b/vllm/compose/single/long-text.yml b/models/qwen3.6-27b/vllm/compose/single/long-text.yml index 720543e0..ae3f0b11 100644 --- a/models/qwen3.6-27b/vllm/compose/single/long-text.yml +++ b/models/qwen3.6-27b/vllm/compose/single/long-text.yml @@ -16,7 +16,8 @@ # Vision: no (text-only — frees VRAM for ctx) # Max ctx: 180K (was 214K pre-Cliff-2b; reduced for safety) # Genesis: v7.72.2 -# Status: ⚠️ NOT IDE-agent safe (Cliff 2b at >50K — see warning above) +# Status: ⚠️ Production w/ caveats +# Caveats: Cliff 2b at >50K single-prompt context (Genesis PN59 doesn't engage on chunked-prefill). NOT IDE-agent safe. Filed at Sandermage/genesis-vllm-patches#22. Workaround: dual.yml or llamacpp/default. # Best for: RAG / single-shot summarization at <25K accumulated ctx; # for IDE agents use tools-text.yml or dual.yml instead # --------------------------------------------------------------------------- diff --git a/models/qwen3.6-27b/vllm/compose/single/long-vision.yml b/models/qwen3.6-27b/vllm/compose/single/long-vision.yml index 1628a7b6..26772e7f 100644 --- a/models/qwen3.6-27b/vllm/compose/single/long-vision.yml +++ b/models/qwen3.6-27b/vllm/compose/single/long-vision.yml @@ -16,7 +16,8 @@ # Vision: yes (vision tower active) # Max ctx: 145K (vision tower costs ~500 MiB activation budget vs long-text) # Genesis: v7.72.2 -# Status: ⚠️ Cliff 2b applies at >50K single-prompt — see warning above +# Status: ⚠️ Production w/ caveats +# Caveats: Cliff 2b at >50K single-prompt context. Sandermage/genesis-vllm-patches#22. Workaround: dual.yml or llamacpp/default. # Best for: Single-card vision workloads (image Q&A, multimodal RAG) # --------------------------------------------------------------------------- # Long-vision variant — 198K context with vision on a single 3090. diff --git a/models/qwen3.6-27b/vllm/compose/single/minimal.yml b/models/qwen3.6-27b/vllm/compose/single/minimal.yml index 38d49ae4..653eef2c 100644 --- a/models/qwen3.6-27b/vllm/compose/single/minimal.yml +++ b/models/qwen3.6-27b/vllm/compose/single/minimal.yml @@ -7,6 +7,7 @@ # Vision: no # Max ctx: 65K (mem-util 0.95) # Genesis: none — bare vLLM, isolation control +# Status: ✅ Production # Best for: Debugging baseline / 20 GB Ampere fallback when TQ3 paths # don't fit / first-time setup verification # --------------------------------------------------------------------------- diff --git a/models/qwen3.6-27b/vllm/compose/single/tools-text.yml b/models/qwen3.6-27b/vllm/compose/single/tools-text.yml index 971c1da4..5a0bcfc8 100644 --- a/models/qwen3.6-27b/vllm/compose/single/tools-text.yml +++ b/models/qwen3.6-27b/vllm/compose/single/tools-text.yml @@ -7,6 +7,7 @@ # Vision: no # Max ctx: 75K (Cliff 1 mech B safe via Genesis P8 + fp8 KV path) # Genesis: v7.72.2 (P8 active for IDE-agent prefill safety) +# Status: ✅ Production # Best for: IDE-agent workloads (Cline / OpenCode / Roo / Claude Code / # Cursor) — fp8 KV escape valve from the long-text TQ3 leak (see #16) # ---------------------------------------------------------------------------