From 7a7efbea0dfd6694abe0bcfcbdb570d85fb9a884 Mon Sep 17 00:00:00 2001 From: noonghunna <10742901+noonghunna@users.noreply.github.com> Date: Sat, 2 May 2026 03:40:06 +0000 Subject: [PATCH] =?UTF-8?q?bump=20Genesis=20pin=20753344b=20=E2=86=92=20fc?= =?UTF-8?q?89395=20(v7.66=20dev=20tip)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit v7.66 ships 3 new patches relevant to our config: - PN33 (default ON): spec-decode warmup K-aware sizing, vllm#37521 backport EXTENDED beyond EAGLE to cover MTP/ngram. Sander claimed it closes both ampersandru's mid-stream OOM AND our workspace_lock AssertionError. - PN25 v7.66: refactored from `@torch.library.custom_op` to `direct_register_custom_op` + `Library("genesis", "FRAGMENT")` at module level. Schema introspection at import time eliminates the `infer_schema skipped frame` Dynamo crash class. - PN32 (default OFF): GDN chunked-prefill for Cliff 2 single-24GB-GPU OOM. Cross-rig validation findings on 1×3090 TP=1 -------------------------------------------- **PN33 partial — narrows but does not close workspace_lock on TP=1.** Sander's claim was that PN33 closes both ampersandru's mid-stream OOM AND our workspace_lock AssertionError. Tested both: | Test | PN33 result | |--------------------------------------------|------------------| | Engine boot (profile_run workspace lock) | ✅ closed | | Runtime decode (`turboquant_attn.py:1350`) | ❌ still fires | Engine boots cleanly without `patch_workspace_lock_disable.py` sidecar when PN33 is on, BUT the first decode request crashes with the same `AssertionError: Workspace is locked but allocation from turboquant_attn.py:1350:_decode_attention requires 0.76 MB`. Net: keep `patch_workspace_lock_disable.py` sidecar mounted. PN33 narrows the bug surface but doesn't close it for our config. **PN25 v7.66 still doesn't work on TP=1.** Sander's `direct_register_custom_op` + `Library("genesis", "FRAGMENT")` approach replaces v7.65's `@torch.library.custom_op`, eliminating the `infer_schema` skipped-frame issue. But on TP=1 the new failure mode is `Library("genesis", "FRAGMENT")` itself failing inside dynamo trace at `instantiate_user_defined_class_object` (different mechanism, same root cause: Library construction inside trace context disallowed on TP=1). Net: keep `patch_pn25_genesis_register_fix.py` v3 (import-time approach). Our patch text-patches activation.py to register the op at module-import time as a cached global, BEFORE any trace context exists. Survives both the v7.65 `@custom_op` and v7.66 `Library` failure modes because we register outside the trace entirely. **PN30 dst-shaped temp fix carries forward cleanly.** Our `patch_pn30_dst_shaped_temp_fix.py` anchor still matches v7.66's PN30 wiring file. All 4 TQ3 composes still pass probes 4 + 5 (multi-turn agent, LCB-coding) which would otherwise crash with Sander's upstream PN30 a9977d8 (compact `.contiguous()` row-stride corruption — see genesis-vllm-patches#17 reply for the diagnosis). **PN31 still doesn't fit on 24 GB.** Same memory pressure as v7.65 round. Validation matrix on v7.66 -------------------------- | Compose | Probes (verify-stress.sh) | |--------------------|-------------------------------------------| | long-text | 6/7 ✅ (Cliff 2 only fail) | | long-vision | 6/7 ✅ (Cliff 2 only fail) | | bounded-thinking | 6/7 ✅ (Cliff 2 only fail) | | dual-turbo (TP=2) | 6/7 ✅ (Cliff 2 only fail) | Same coverage as v7.65 + our patches. No new regressions on v7.66. Net effect of pin bump ---------------------- - Get Sander's v7.66 + PN33 (validated improvement, even if partial) - Get PN32 available for opt-in (Cliff 2 mitigation, untested by us) - Same 3 local sidecars retained (PN25 v3, PN30 fix, workspace_lock) - No simplification possible yet Per-config + cross-rig summary in results/v0.20-migration/v766-pin-results.summary. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../v0.20-migration/v766-pin-results.summary | 52 +++++++++++++++++++ scripts/setup.sh | 6 +-- 2 files changed, 55 insertions(+), 3 deletions(-) create mode 100644 results/v0.20-migration/v766-pin-results.summary diff --git a/results/v0.20-migration/v766-pin-results.summary b/results/v0.20-migration/v766-pin-results.summary new file mode 100644 index 00000000..6aa02f1d --- /dev/null +++ b/results/v0.20-migration/v766-pin-results.summary @@ -0,0 +1,52 @@ +date: 2026-05-02 +genesis_pin_bump: 753344b → fc89395 (v7.65 dev tip → v7.66 dev tip) +new_patches_active: + - PN33 spec-decode warmup K-aware (vllm#37521 backport, default ON) + - PN25 v7.66 (direct_register_custom_op refactor, default ON) + - PN30 still default OFF — opt-in + - PN31 still default OFF — opt-in + - PN32 NEW (GDN chunked-prefill for Cliff 2 single-24GB-GPU OOM, default OFF) + +cross_rig_findings: + PN33_workspace_lock_test: | + Sander claimed PN33 closes both ampersandru's mid-stream OOM AND our + workspace_lock AssertionError. Cross-rig validation 2026-05-02 AM: + PN33 closes BOOT-time workspace_lock issue (engine starts cleanly without + our patch_workspace_lock_disable.py sidecar applied), BUT the runtime + workspace_lock AssertionError at `turboquant_attn.py:1350:_decode_attention` + still fires on the first decode request. Two distinct paths: + - Boot-time profile_run workspace lock: closed by PN33 ✅ + - Runtime decode workspace lock: NOT closed by PN33, still needs our + patch_workspace_lock_disable.py sidecar ❌ + Net: keep our sidecar, but the bug surface narrowed. + + PN25_v766_test: | + Sander's v7.66 PN25 (direct_register_custom_op + Library FRAGMENT) does + NOT work on TP=1 without our local v3 import-time patch. New failure + mode is `Library("genesis", "FRAGMENT")` itself fails inside dynamo + trace at `instantiate_user_defined_class_object` (different mechanism + from v7.65's @custom_op + infer_schema, but same root cause: Library + construction inside trace context not allowed on TP=1). + Net: keep our local v3 import-time patch. + + PN30_with_dst_shaped_temp_fix: | + Our local PN30 dst-shaped temp fix still applies cleanly on v7.66 + Genesis tree (anchor matches). All 4 TQ3 composes pass probes 4 + 5 + (multi-turn agent, LCB-coding) which would otherwise crash with + Sander's upstream PN30 a9977d8 (compact .contiguous() row-stride bug). + +verify_stress_results_v766: + long-text (180K + 0.95): 6/7 (Cliff 2 only fail) + long-vision (145K + 0.95): 6/7 (Cliff 2 only fail) + bounded-thinking (180K + 0.95): 6/7 (Cliff 2 only fail) + dual-turbo (262K + 0.85, TP=2): 6/7 (Cliff 2 only fail) + +local_sidecars_kept: + - patch_pn25_genesis_register_fix.py (v3 import-time approach for TP=1) + - patch_pn30_dst_shaped_temp_fix.py (replaces Sander's compact .contiguous()) + - patch_workspace_lock_disable.py (PN33 closes boot-time, this still needed for runtime decode path) + - patch_tolist_cudagraph.py (cudagraph capture fix) + +local_sidecars_dropped: | + None this round — PN33 narrowed but didn't close the workspace_lock + surface, all 3 sidecars still required. diff --git a/scripts/setup.sh b/scripts/setup.sh index 79b01fbb..562aa9c1 100755 --- a/scripts/setup.sh +++ b/scripts/setup.sh @@ -93,7 +93,7 @@ echo "Model dir: ${MODEL_DIR}" # vllm/_genesis package + per-patch env opts). Newer composes mount the package; # the legacy compose still references the v7.13 shim. # Pin Genesis to the exact commit our published numbers were measured against. -# Currently pointing at v7.65 dev tip (commit d89a089, 2026-05-01 PM). Bumped +# Currently pointing at v7.66 dev tip (commit fc89395, 2026-05-02 AM). Bumped # from v7.64 (64dd18b) for the v7.65 patch set: # - P38B / P15B — close the Cliff 1 mech B cascade (issues #14 + #15) via # compile-safe in-source hook + FA varlen workspace clamp. @@ -105,11 +105,11 @@ echo "Model dir: ${MODEL_DIR}" # Sandermage's 27B-specific tuning). # - PN28 — merge_attn_states NaN guard backport (vllm#39148). # - Cliff 8 hardening (partial_apply_warnings counter in boot summary). -# Pinned to dev SHA d89a089 because v7.65 is feature-complete on dev but not +# Pinned to dev SHA fc89395 because v7.66 is feature-complete on dev but not # yet tagged; SHA pin is immutable. # Bumping GENESIS_PIN requires re-running verify-full.sh against your composes # to confirm the new commit works on your config. -GENESIS_PIN="${GENESIS_PIN:-753344b}" +GENESIS_PIN="${GENESIS_PIN:-fc89395}" if [[ "${SKIP_GENESIS:-0}" != "1" ]]; then if [[ -d "${GENESIS_DIR}/.git" ]]; then