bump Genesis pin 753344b → fc89395 (v7.66 dev tip)

v7.66 ships 3 new patches relevant to our config:
- PN33 (default ON): spec-decode warmup K-aware sizing, vllm#37521 backport
  EXTENDED beyond EAGLE to cover MTP/ngram. Sander claimed it closes both
  ampersandru's mid-stream OOM AND our workspace_lock AssertionError.
- PN25 v7.66: refactored from `@torch.library.custom_op` to
  `direct_register_custom_op` + `Library("genesis", "FRAGMENT")` at module
  level. Schema introspection at import time eliminates the
  `infer_schema skipped frame` Dynamo crash class.
- PN32 (default OFF): GDN chunked-prefill for Cliff 2 single-24GB-GPU OOM.

Cross-rig validation findings on 1×3090 TP=1
--------------------------------------------

**PN33 partial — narrows but does not close workspace_lock on TP=1.**

Sander's claim was that PN33 closes both ampersandru's mid-stream OOM
AND our workspace_lock AssertionError. Tested both:

| Test                                       | PN33 result      |
|--------------------------------------------|------------------|
| Engine boot (profile_run workspace lock)   |  closed       |
| Runtime decode (`turboquant_attn.py:1350`) |  still fires  |

Engine boots cleanly without `patch_workspace_lock_disable.py` sidecar
when PN33 is on, BUT the first decode request crashes with the same
`AssertionError: Workspace is locked but allocation from
turboquant_attn.py:1350:_decode_attention requires 0.76 MB`.

Net: keep `patch_workspace_lock_disable.py` sidecar mounted. PN33
narrows the bug surface but doesn't close it for our config.

**PN25 v7.66 still doesn't work on TP=1.**

Sander's `direct_register_custom_op` + `Library("genesis", "FRAGMENT")`
approach replaces v7.65's `@torch.library.custom_op`, eliminating the
`infer_schema` skipped-frame issue. But on TP=1 the new failure mode is
`Library("genesis", "FRAGMENT")` itself failing inside dynamo trace at
`instantiate_user_defined_class_object` (different mechanism, same root
cause: Library construction inside trace context disallowed on TP=1).

Net: keep `patch_pn25_genesis_register_fix.py` v3 (import-time approach).
Our patch text-patches activation.py to register the op at module-import
time as a cached global, BEFORE any trace context exists. Survives both
the v7.65 `@custom_op` and v7.66 `Library` failure modes because we
register outside the trace entirely.

**PN30 dst-shaped temp fix carries forward cleanly.**

Our `patch_pn30_dst_shaped_temp_fix.py` anchor still matches v7.66's
PN30 wiring file. All 4 TQ3 composes still pass probes 4 + 5 (multi-turn
agent, LCB-coding) which would otherwise crash with Sander's upstream
PN30 a9977d8 (compact `.contiguous()` row-stride corruption — see
genesis-vllm-patches#17 reply for the diagnosis).

**PN31 still doesn't fit on 24 GB.** Same memory pressure as v7.65 round.

Validation matrix on v7.66
--------------------------

| Compose            | Probes (verify-stress.sh)                |
|--------------------|-------------------------------------------|
| long-text          | 6/7  (Cliff 2 only fail)               |
| long-vision        | 6/7  (Cliff 2 only fail)               |
| bounded-thinking   | 6/7  (Cliff 2 only fail)               |
| dual-turbo (TP=2)  | 6/7  (Cliff 2 only fail)               |

Same coverage as v7.65 + our patches. No new regressions on v7.66.

Net effect of pin bump
----------------------

- Get Sander's v7.66 + PN33 (validated improvement, even if partial)
- Get PN32 available for opt-in (Cliff 2 mitigation, untested by us)
- Same 3 local sidecars retained (PN25 v3, PN30 fix, workspace_lock)
- No simplification possible yet

Per-config + cross-rig summary in
results/v0.20-migration/v766-pin-results.summary.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
noonghunna
2026-05-02 03:40:06 +00:00
parent b875624f2d
commit 7a7efbea0d
2 changed files with 55 additions and 3 deletions

View File

@@ -0,0 +1,52 @@
date: 2026-05-02
genesis_pin_bump: 753344b → fc89395 (v7.65 dev tip → v7.66 dev tip)
new_patches_active:
- PN33 spec-decode warmup K-aware (vllm#37521 backport, default ON)
- PN25 v7.66 (direct_register_custom_op refactor, default ON)
- PN30 still default OFF — opt-in
- PN31 still default OFF — opt-in
- PN32 NEW (GDN chunked-prefill for Cliff 2 single-24GB-GPU OOM, default OFF)
cross_rig_findings:
PN33_workspace_lock_test: |
Sander claimed PN33 closes both ampersandru's mid-stream OOM AND our
workspace_lock AssertionError. Cross-rig validation 2026-05-02 AM:
PN33 closes BOOT-time workspace_lock issue (engine starts cleanly without
our patch_workspace_lock_disable.py sidecar applied), BUT the runtime
workspace_lock AssertionError at `turboquant_attn.py:1350:_decode_attention`
still fires on the first decode request. Two distinct paths:
- Boot-time profile_run workspace lock: closed by PN33 ✅
- Runtime decode workspace lock: NOT closed by PN33, still needs our
patch_workspace_lock_disable.py sidecar ❌
Net: keep our sidecar, but the bug surface narrowed.
PN25_v766_test: |
Sander's v7.66 PN25 (direct_register_custom_op + Library FRAGMENT) does
NOT work on TP=1 without our local v3 import-time patch. New failure
mode is `Library("genesis", "FRAGMENT")` itself fails inside dynamo
trace at `instantiate_user_defined_class_object` (different mechanism
from v7.65's @custom_op + infer_schema, but same root cause: Library
construction inside trace context not allowed on TP=1).
Net: keep our local v3 import-time patch.
PN30_with_dst_shaped_temp_fix: |
Our local PN30 dst-shaped temp fix still applies cleanly on v7.66
Genesis tree (anchor matches). All 4 TQ3 composes pass probes 4 + 5
(multi-turn agent, LCB-coding) which would otherwise crash with
Sander's upstream PN30 a9977d8 (compact .contiguous() row-stride bug).
verify_stress_results_v766:
long-text (180K + 0.95): 6/7 (Cliff 2 only fail)
long-vision (145K + 0.95): 6/7 (Cliff 2 only fail)
bounded-thinking (180K + 0.95): 6/7 (Cliff 2 only fail)
dual-turbo (262K + 0.85, TP=2): 6/7 (Cliff 2 only fail)
local_sidecars_kept:
- patch_pn25_genesis_register_fix.py (v3 import-time approach for TP=1)
- patch_pn30_dst_shaped_temp_fix.py (replaces Sander's compact .contiguous())
- patch_workspace_lock_disable.py (PN33 closes boot-time, this still needed for runtime decode path)
- patch_tolist_cudagraph.py (cudagraph capture fix)
local_sidecars_dropped: |
None this round — PN33 narrowed but didn't close the workspace_lock
surface, all 3 sidecars still required.

View File

@@ -93,7 +93,7 @@ echo "Model dir: ${MODEL_DIR}"
# vllm/_genesis package + per-patch env opts). Newer composes mount the package;
# the legacy compose still references the v7.13 shim.
# Pin Genesis to the exact commit our published numbers were measured against.
# Currently pointing at v7.65 dev tip (commit d89a089, 2026-05-01 PM). Bumped
# Currently pointing at v7.66 dev tip (commit fc89395, 2026-05-02 AM). Bumped
# from v7.64 (64dd18b) for the v7.65 patch set:
# - P38B / P15B — close the Cliff 1 mech B cascade (issues #14 + #15) via
# compile-safe in-source hook + FA varlen workspace clamp.
@@ -105,11 +105,11 @@ echo "Model dir: ${MODEL_DIR}"
# Sandermage's 27B-specific tuning).
# - PN28 — merge_attn_states NaN guard backport (vllm#39148).
# - Cliff 8 hardening (partial_apply_warnings counter in boot summary).
# Pinned to dev SHA d89a089 because v7.65 is feature-complete on dev but not
# Pinned to dev SHA fc89395 because v7.66 is feature-complete on dev but not
# yet tagged; SHA pin is immutable.
# Bumping GENESIS_PIN requires re-running verify-full.sh against your composes
# to confirm the new commit works on your config.
GENESIS_PIN="${GENESIS_PIN:-753344b}"
GENESIS_PIN="${GENESIS_PIN:-fc89395}"
if [[ "${SKIP_GENESIS:-0}" != "1" ]]; then
if [[ -d "${GENESIS_DIR}/.git" ]]; then