Three related [C0]/eligibility pull-gate fixes for the curated-swap surface —
an uncurated derive (abliterated / fine-tune) of a model we already serve.
1. Wrapper-arch alias. `pull.sh --profile-like` was false-aborting at [C0] with
"no arch_patches matrix row for 'Qwen3_5ForConditionalGeneration'". That is
the OUTER multimodal wrapper class the weights report; the patch matrix is
keyed on the inner canonical `Qwen3NextForCausalLM`. arch_patches.yml is a
closed key-set, so the alias lives in the editable arch_model_xref.
- profile_runtime.yml: `config_architectures: [Qwen3_5ForConditionalGeneration]`
on the Qwen3NextForCausalLM xref entry.
- generate_compose.py: `resolve_arch_from_config()` maps a config.json
architectures[0] string -> (canonical_arch, arch_row) via that alias.
- gates.py [C0]: resolve the wrapper arch via the alias before declaring
NO_ARCH_ROW. The hybrid now reports ENGINE_SUPPORTED.
2. GGUF axis. `supported_weight_formats` was declared on every engine but never
enforced (only `kv_format` was). The deriver blocks GGUF on the derive path,
but the curated registry / curated-swap path had no such guard. gates.py [C0]
now rejects a `gguf` weight_format on an engine whose supported_weight_formats
lacks `gguf` (structural axis; matches the `gguf` token only, so a derive's
raw dtype spelling bf16/float16 is never false-rejected).
3. Won't-fit size advisory. The eligibility no-fit-model abort for a hybrid/MoE
derive now appends (a) an actionable NOTE pointing at the curated-swap path +
docs/BRING_YOUR_OWN.md, and (b) a coarse weights-only VRAM verdict: when the
raw weights exceed the detected topology's total VRAM they won't fit at ANY
KV, so say so concretely (the huihui abliterated bf16 ~54 GB vs 2×24 GB case)
instead of a generic stop. `_weights_oversize_advisory()` is pure/total —
empty when it fits / size unknown / headless.
Docs + tests:
- BRING_YOUR_OWN.md: new section C — "Swap a curated model for a fine-tune /
abliterated variant -> reuse its compose" (artifact↔engine + quant + MTP
caveats, worked example).
- test-pullgate-gates.sh: wrapper-arch ALIAS [C0] case + resolve_arch_from_config()
unit + GGUF-on-vLLM runtime-incompatible + no-false-positive control +
_weights_oversize_advisory() unit (oversize / fits / headless / malformed).
Co-authored-by: noonghunna <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
Replace the purged `vllm-nightly-clean` pin (nightly-bf610c2f, now 404 on
Docker Hub — so the launcher injected a dead image for every slug on it)
with a v0.22.0 two-engine design:
- `vllm-stable`: OVERLAY-FREE v0.22.0 — the broad successor to
vllm-nightly-clean. Serves Qwen3-Next (hybrid 27b + MoE 35B-A3B), generic
`dense` transformers, and uncurated derived-emission bases. Migrates
vllm/qwen-35b-a3b-dual, vllm/dual, vllm/minimal (live-validated on stock
v0.22.0: TP=2 + MTP + tool-call clean; marlin-pad confirmed unnecessary).
Added as the first loads:true arch pin on the qwen + dense rows.
- `vllm-gemma-stable`: KEPT as the OVERLAY-CARRYING v0.22.0 engine (#40391
per-head INT8 KV + #42006 tool-parser) — the Gemma 4 path. Unchanged.
- `vllm-pip-baseline`: the renamed pip `dense` lower-bound (frees the
`vllm-stable` name for the docker engine above).
Why two engines and not one: `vendored_overlays` is LOAD-BEARING, not
documentation. `derived_emittable` (CONTRACT-5) refuses any engine whose
`vendored_overlays != []` as a derived-emission base, and `diagnose-profile`
expects the Gemma overlay declared on its engine. A single engine cannot be
both overlay-free (for derived/Qwen bases) and overlay-carrying (for Gemma
provenance) — so the two stay split. Compose owns patch APPLICATION; engine
owns provenance/compatibility. (An attempt to fold both into one engine broke
4 tests via exactly these two consumers; the split is the validated shape.)
Also adds scripts/tests/test-compose-image-drift.sh: asserts every fixed
`${VLLM_IMAGE:-<tag>}` compose default equals its engine's `install.spec`
(catches bump-the-engine-forget-the-literal drift), and a docs/ADDING_MODELS.md
"Coherence rules" section codifying the compose/engine/patch separation.
42/42 gate green; resolver emits v0.22.0 for all migrated Qwen + Gemma slugs.
The remaining experimental/preview slugs still on vllm-nightly-clean migrate
in follow-up as each is validated, then that engine is deprecated.
Refs #254.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
CONTRACT-2 (§10-R4) arch-family registry expansion: +13 safetensors arch
rows in arch_patches.yml (PhiForCausalLM — the microsoft/phi-2 STEP V1
on-rig no-arch-row anchor — Phi3Small, Gemma/Gemma3/Gemma3-CG, Starcoder2,
Cohere, InternLM2, Mixtral/Qwen2Moe/Qwen3Moe MoE, Qwen2-VL). Additive data
only, zero [C0]/decision-logic change. Zero false-pass by construction:
each follows the established estimated-lower-bound/unverified-TRC precedent
so [C0] still resolves needs-trust-remote-code-ack (fail-closed, bypassable
ONLY by --trust-remote-code) — the expansion drops only the
--experimental-arch requirement, never auto-passes; an arch still absent
still hard-blocks no-arch-row. test-pullgate-gates.sh proves both, plus the
#146-shape worked acceptance case (a hand-added awq_bf16_int4 weights
variant the expanded flag schema/parity machinery absorbs cleanly).
CONTRACT-2b-i chat-template attribution + behavioral drift_guard: new
`chat_template` delivery class (VALID_DELIVERY_MECHANISM); froggeric (22
composes — 18 direct + 4 nvlink* via REAL Docker Compose extends: merge)
and carnice (mount-only) brought under load_bearing_when + a behavioral
drift_guard whose check encodes the self-contained symmetric restart+settle
protocol (identical docker restart both arms, /v1/models healthy, 60s
settle, >=3 bench runs/arm, grand-mean same-segment compare, flag only a
3/3 deterministic regression). Effective coverage uses REAL merge
semantics: docker compose config (preferred) or a deterministic offline
extends: merge applying the same rules (additive sequence merge; `!reset`
removal) — never the unsound single-base text concat. .jinja artifact
discovery catches an orphan vendored template. test-patch-attribution.sh
adds the class checks + an H4 fixture asserting a `!reset` child AND a
stopped-extending child both lose coverage (the false-negative is the
dangerous direction). Generator emit kept in lock-step with reaches().
Documented as PATCH_POLICY.md §3.1. Rig-independent leak assertions added
(str(abs_dir) not in shared; repo-relative-only — never a /opt|/home
substring allowlist).
RED-LINE: gates.py/pull.py/deriver.py/capture.py/loop_input.py/
classifier.py/dedup.py/submit_pull.py/kv-calc.py/failure_fingerprints.yml
byte-unchanged; no shipped compose changed; patch_attribution.py c0_state/
is_artifact/compose_text/service_body byte-identical (additive only). Full
test-*.sh suite green in the CI condition; kv-calc --calibration N/N.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
STEP 3: scripts/generate-compose.sh (+ scripts/lib/generate_compose.py).
Implements the brief's steps 1-10: scope gates first (type!=vllm /
genesis_equipped -> clean refuse), engine-pin loads:true validation
(image NEVER rewritten), arch via model_slugs/arch_model_xref, tp/kv
validation, trc {true,unverified} security refusal, compose-keyed patch
selection, delivery-gap-before-drift-guard, graded drift-guard
(capability-scoped -> OMIT+DEGRADED+--accept-degraded, foundational ->
hard-refuse, never repair, never wire a failed patch). Emits from the
captured compose_service_template: param-slots/constants verbatim, image
expression passed through verbatim, --trust-remote-code never emitted
in-scope (governed slot, locked §88), wiring re-derived only at the two
named insertion points, synthesizes nothing else. 3-category provenance
header above services: so STEP-2 service_body() discards it.
STEP 4: scripts/tests/test-generate-compose.sh. 5 golden triples
(vllm/minimal, vllm/dual, vllm/gemma-mtp, vllm/gemma-int8 [full,
multi-file overlay], vllm/gemma-dflash [dflash]) — all verified
genesis_equipped:false. Per triple: semantic diff vs shipped confined to
the two insertion points (image + constants verbatim), selected+wired
subset-of-shipped, wired pass reaches() on the GENERATED compose,
selected-but-undelivered NOT reachable, 3-category header, no
--trust-remote-code emitted. Plus the refusal/degraded matrix and
kv_arg() unit table. Imports (does not reimplement) patch_attribution.
test-patch-attribution.sh stays byte-identical (61 patch / 11 arch / 18
calibration, same 15 known-gap lines); all other test-*.sh remain RC=0.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
