alex/club-3090
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(qwen3.6-27b): route non-TQ3 composes to vllm-nightly-clean

Browse Source 
Per the TQ3-only Genesis policy (Genesis is strictly required only for
turboquant_3bit_nc KV — Cliff 2 mitigations are recommended but not
required to boot), the Qwen 27B composes that don't use TQ3 KV no longer
need to be anchored to the Genesis-locked SHA. They can ride the latest
unconstrained nightly (vllm-nightly-clean → bf610c2f).

Composes moved from vllm-nightly-mtp to vllm-nightly-clean (8 entries):
- vllm/tools-text         single/tools-text.yml      fp8_e5m2
- vllm/minimal            single/minimal.yml         fp8_e5m2
- vllm/dual               dual/docker-compose.yml    fp8_e5m2
- vllm/dual-bf16          dual/bf16.yml              bf16
- vllm/dual-carnice-bf16mtp dual/carnice-bf16mtp.yml fp8_e5m2
- vllm/dual-qwopus-bf16mtp  dual/qwopus-bf16mtp.yml  fp8_e5m2
- vllm/dual-nvlink        dual/nvlink.yml (extends)  fp8_e5m2
- vllm/dual4              multi4/docker-compose.yml  fp8_e5m2

TQ3-using composes (vllm/default, vllm/long-text, vllm/long-text-no-mtp,
vllm/long-vision, vllm/bounded-thinking, vllm/dual-turbo,
vllm/dual-tq3-mtp, vllm/dual-tq3-mtp-genesis, vllm/dual-tq3-nomtp,
vllm/dual-nvlink-turbo) stay on vllm-nightly-mtp.

Model profile update:
- qwen3.6-27b.requires_genesis flipped true → false.
  Strictly bootable on any qwen3-next-hybrid-capable vLLM nightly.
  Genesis is required only for TQ3 KV format; that's enforced at the
  compose level via Engine-profile selection.

Engine profile update:
- vllm-nightly-clean.supported_model_families adds qwen3-next-hybrid.
- Notes corrected to reflect the TQ3-only policy and broader family
  coverage.

Test updates:
- to_compose_name strict match: updated to expect vllm-nightly-clean
  for fp8/tp=2 long-ctx Qwen.
- C6 test reframed: under the TQ3-only policy no model declares
  requires_genesis=true, so the Genesis enforcement happens at C15
  (engine feature) level, not C6. Test now asserts positive (Qwen +
  fp8 on non-Genesis engine is valid) AND negative (Qwen + TQ3 on
  non-Genesis engine fails C15).

All compat tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
noonghunna
2026-05-15 13:01:55 +00:00
committed by noonghunna
parent cf0451a7ad
commit 3b2d940d26
11 changed files with 38 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
models/qwen3.6-27b/vllm/compose/dual/bf16.yml
View File

@@ -22,7 +22,7 @@
# ===========================================================================
# Hardware metadata (parsed by scripts/preflight.sh):
# Requires-min-vram-gb: 24
# Engine-profile: vllm-nightly-mtp
# Engine-profile: vllm-nightly-clean
# Requires-min-gpu-count: 2
# Tensor-parallel: 2
services:

2
models/qwen3.6-27b/vllm/compose/dual/carnice-bf16mtp.yml
View File

@@ -50,7 +50,7 @@
# ===========================================================================
# Hardware metadata (parsed by scripts/preflight.sh):
# Requires-min-vram-gb: 24
# Engine-profile: vllm-nightly-mtp
# Engine-profile: vllm-nightly-clean
# Requires-min-gpu-count: 2
# Tensor-parallel: 2
services:

2
models/qwen3.6-27b/vllm/compose/dual/docker-compose.yml
View File

@@ -51,7 +51,7 @@
# ===========================================================================
# Hardware metadata (parsed by scripts/preflight.sh):
# Requires-min-vram-gb: 24
# Engine-profile: vllm-nightly-mtp
# Engine-profile: vllm-nightly-clean
# Requires-min-gpu-count: 2
# Tensor-parallel: 2
services:

2
models/qwen3.6-27b/vllm/compose/dual/qwopus-bf16mtp.yml
View File

@@ -35,7 +35,7 @@
# ===========================================================================
# Hardware metadata (parsed by scripts/preflight.sh):
# Requires-min-vram-gb: 24
# Engine-profile: vllm-nightly-mtp
# Engine-profile: vllm-nightly-clean
# Requires-min-gpu-count: 2
# Tensor-parallel: 2
services:

2
models/qwen3.6-27b/vllm/compose/multi4/docker-compose.yml
View File

@@ -59,7 +59,7 @@
# ===========================================================================
# Hardware metadata (parsed by scripts/preflight.sh):
# Requires-min-vram-gb: 24
# Engine-profile: vllm-nightly-mtp
# Engine-profile: vllm-nightly-clean
# Requires-min-gpu-count: 4
# Tensor-parallel: 4
services:

2
models/qwen3.6-27b/vllm/compose/single/minimal.yml
View File

@@ -33,7 +33,7 @@
# ===========================================================================
# Hardware metadata (parsed by scripts/preflight.sh):
# Requires-min-vram-gb: 20
# Engine-profile: vllm-nightly-mtp
# Engine-profile: vllm-nightly-clean
# Requires-min-gpu-count: 1
# Tensor-parallel: 1
services:

2
models/qwen3.6-27b/vllm/compose/single/tools-text.yml
View File

@@ -35,7 +35,7 @@
# ===========================================================================
# Hardware metadata (parsed by scripts/preflight.sh):
# Requires-min-vram-gb: 24
# Engine-profile: vllm-nightly-mtp
# Engine-profile: vllm-nightly-clean
# Requires-min-gpu-count: 1
# Tensor-parallel: 1
services:

16
scripts/lib/profiles/compose_registry.py
View File

@@ -88,14 +88,14 @@ COMPOSE_REGISTRY = {
),
"vllm/tools-text": _entry(
model="qwen3.6-27b", weights_variant="autoround_int4", workload="tool-heavy",
engine="vllm-nightly-mtp", drafter="qwen-mtp-builtin", kv_format="fp8_e5m2",
engine="vllm-nightly-clean", drafter="qwen-mtp-builtin", kv_format="fp8_e5m2",
tp=1, max_ctx=75000, max_num_seqs=1, mem_util=0.97,
compose_path="models/qwen3.6-27b/vllm/compose/single/tools-text.yml",
default_port=8020,
),
"vllm/minimal": _entry(
model="qwen3.6-27b", weights_variant="autoround_int4", workload="fast-chat",
engine="vllm-nightly-mtp", drafter=None, kv_format="fp8_e5m2",
engine="vllm-nightly-clean", drafter=None, kv_format="fp8_e5m2",
tp=1, max_ctx=32768, max_num_seqs=1, mem_util=0.92,
compose_path="models/qwen3.6-27b/vllm/compose/single/minimal.yml",
default_port=8020,
@@ -104,7 +104,7 @@ COMPOSE_REGISTRY = {
# Qwen 3.6 27B, vLLM dual/multi-card.
"vllm/dual": _entry(
model="qwen3.6-27b", weights_variant="autoround_int4", workload="long-ctx-single",
engine="vllm-nightly-mtp", drafter="qwen-mtp-builtin", kv_format="fp8_e5m2",
engine="vllm-nightly-clean", drafter="qwen-mtp-builtin", kv_format="fp8_e5m2",
tp=2, max_ctx=262144, max_num_seqs=2, mem_util=0.92,
compose_path="models/qwen3.6-27b/vllm/compose/dual/docker-compose.yml",
default_port=8010, recommended_engine_features=["marlin_pad_sub_tile_n"],
@@ -133,7 +133,7 @@ COMPOSE_REGISTRY = {
),
"vllm/dual-bf16": _entry(
model="qwen3.6-27b", weights_variant="autoround_int4", workload="long-ctx-single",
engine="vllm-nightly-mtp", drafter="qwen-mtp-builtin", kv_format="bf16",
engine="vllm-nightly-clean", drafter="qwen-mtp-builtin", kv_format="bf16",
tp=2, max_ctx=200000, max_num_seqs=1, mem_util=0.92,
compose_path="models/qwen3.6-27b/vllm/compose/dual/bf16.yml",
default_port=8012,
@@ -168,21 +168,21 @@ COMPOSE_REGISTRY = {
),
"vllm/dual-carnice-bf16mtp": _entry(
model="qwen3.6-27b", weights_variant="carnice_bf16mtp", workload="long-ctx-single",
engine="vllm-nightly-mtp", drafter="qwen-mtp-builtin", kv_format="fp8_e5m2",
engine="vllm-nightly-clean", drafter="qwen-mtp-builtin", kv_format="fp8_e5m2",
tp=2, max_ctx=262144, max_num_seqs=2, mem_util=0.92,
compose_path="models/qwen3.6-27b/vllm/compose/dual/carnice-bf16mtp.yml",
default_port=8070,
),
"vllm/dual-qwopus-bf16mtp": _entry(
model="qwen3.6-27b", weights_variant="qwopus_bf16mtp", workload="long-ctx-single",
engine="vllm-nightly-mtp", drafter="qwen-mtp-builtin", kv_format="fp8_e5m2",
engine="vllm-nightly-clean", drafter="qwen-mtp-builtin", kv_format="fp8_e5m2",
tp=2, max_ctx=262144, max_num_seqs=2, mem_util=0.92,
compose_path="models/qwen3.6-27b/vllm/compose/dual/qwopus-bf16mtp.yml",
default_port=8071,
),
"vllm/dual-nvlink": _entry(
model="qwen3.6-27b", weights_variant="autoround_int4", workload="long-ctx-single",
engine="vllm-nightly-mtp", drafter="qwen-mtp-builtin", kv_format="fp8_e5m2",
engine="vllm-nightly-clean", drafter="qwen-mtp-builtin", kv_format="fp8_e5m2",
tp=2, max_ctx=262144, max_num_seqs=2, mem_util=0.92,
compose_path="models/qwen3.6-27b/vllm/compose/dual/nvlink.yml",
default_port=8014, requires_nvlink=True, recommended_engine_features=["marlin_pad_sub_tile_n"],
@@ -211,7 +211,7 @@ COMPOSE_REGISTRY = {
),
"vllm/dual4": _entry(
model="qwen3.6-27b", weights_variant="autoround_int4", workload="multi-stream-tenant",
engine="vllm-nightly-mtp", drafter="qwen-mtp-builtin", kv_format="fp8_e5m2",
engine="vllm-nightly-clean", drafter="qwen-mtp-builtin", kv_format="fp8_e5m2",
tp=4, max_ctx=262144, max_num_seqs=4, mem_util=0.92,
compose_path="models/qwen3.6-27b/vllm/compose/multi4/docker-compose.yml",
default_port=8015,

3
scripts/lib/profiles/engines/vllm-nightly-clean.yml
View File

@@ -11,6 +11,7 @@ supported_model_families:
- dense
- gemma4-swa-dense
- gemma4-swa-moe
- qwen3-next-hybrid
- qwen3-next-moe
features:
int8_per_token_head: false
@@ -37,4 +38,4 @@ supported_weight_formats:
required_overlays: []
vendored_overlays: []
required_genesis: false
notes: "Unconstrained-nightly path: free to bump to whatever's current on Docker Hub, since no Genesis patches are anchored here. Primary use: families that don't need DeltaNet stabilization (dense, gemma4-swa-dense, gemma4-swa-moe). Also accepts qwen3-next-moe as a *preview/smoke* route — model loads + runs but Cliff 2 mitigations and TQ3 KV are unavailable; production Qwen MoE should target vllm-nightly-mtp once Genesis v7.73.x re-anchors on a post-#42521 SHA. Excludes qwen3-next-hybrid (dense Qwen3-Next: prod 27B path is Genesis-locked, no preview value here). Excludes turboquant_3bit_nc KV (Genesis-only feature). Launch exports VLLM_NIGHTLY_SHA from this spec; VLLM_IMAGE override still works."
notes: "Unconstrained-nightly path: free to bump to whatever's current on Docker Hub, since no Genesis patches are anchored here. Per the TQ3-only Genesis policy (Genesis is strictly required only for turboquant_3bit_nc KV), this engine accepts every supported model family — non-TQ3 composes for Qwen 3.6-27B (qwen3-next-hybrid) and Gemma 4-31B (gemma4-swa-dense) route here, alongside the new MoE additions (qwen3-next-moe, gemma4-swa-moe). For Qwen3-Next workloads at long context (>21-26K), Genesis-anchored vllm-nightly-mtp is *recommended* for Cliff 2 mitigations but not strictly required to boot. Excludes turboquant_3bit_nc KV (Genesis-only feature) — TQ3 composes must route to vllm-nightly-mtp. Launch exports VLLM_NIGHTLY_SHA from this spec; VLLM_IMAGE override still works."

9
scripts/lib/profiles/models/qwen3.6-27b.yml
View File

@@ -51,5 +51,12 @@ valid_tp:
- 1
- 2
- 4
requires_genesis: true
# Strictly bootable on any vLLM nightly that supports the qwen3-next-hybrid
# family. Genesis is REQUIRED for TQ3 KV (turboquant_3bit_nc — Genesis-only
# feature). For non-TQ3 KV formats (fp8, bf16, q4_0, k8v4) Genesis is
# recommended but not strictly needed: it adds Cliff 2 mitigations
# (PN12/PN25/PN34) for long-context stability past ~21-26K. Composes encode
# the engine choice via their `Engine-profile:` header — non-TQ3 composes
# route to vllm-nightly-clean, TQ3 composes route to vllm-nightly-mtp.
requires_genesis: false

16
scripts/tests/test-profiles-compat.sh
View File

@@ -163,12 +163,22 @@ assert not r.valid
assert any(reason.startswith("C5:") for reason in r.reasons), r.reasons
PY
run_test "C6 Genesis one-way implication: Qwen on non-Genesis vLLM rejected" <<'PY'
run_test "C6 Genesis one-way: TQ3 KV on non-Genesis engine rejected (via C15)" <<'PY'
# Under the TQ3-only Genesis policy, no model declares requires_genesis=true
# (so C6 has no current model-level trigger). Genesis is enforced at the
# *feature* level via C15: requesting turboquant_3bit_nc on an engine that
# doesn't expose it (e.g. vllm-stable-next) fails C15. The previous
# C6 assertion (Qwen on non-Genesis vLLM rejected) no longer holds —
# Qwen 27B with fp8 is valid on non-Genesis engines.
from scripts.lib.profiles.compat import load_profiles, fits
p = load_profiles()
# Positive: Qwen 27B + fp8 on non-Genesis engine is now valid.
r = fits([p.hardware["rtx-3090"]], p.models["qwen3.6-27b"], p.workloads["long-ctx-single"], p.engines["vllm-stable-next"], kv_format="fp8_e5m2", tp=1, project_vram=False)
assert r.valid, r.reasons
# Negative: Qwen 27B + TQ3 on non-Genesis engine fails C15.
r = fits([p.hardware["rtx-3090"]], p.models["qwen3.6-27b"], p.workloads["long-ctx-single"], p.engines["vllm-stable-next"], kv_format="turboquant_3bit_nc", tp=1, project_vram=False, required_engine_features=["turboquant_3bit_nc"])
assert not r.valid
assert any(reason.startswith("C6:") for reason in r.reasons), r.reasons
assert any(reason.startswith("C15:") for reason in r.reasons), r.reasons
PY
run_test "C7 drafter method: DFlash on MTP-only engine rejected" <<'PY'
@@ -289,7 +299,7 @@ from scripts.lib.profiles.compat import load_profiles, to_compose_name
p = load_profiles()
name = to_compose_name(
p.models["qwen3.6-27b"],
p.engines["vllm-nightly-mtp"],
p.engines["vllm-nightly-clean"],
p.drafters["qwen-mtp-builtin"],
"fp8_e5m2",
2,