7a7efbea0d
v7.66 ships 3 new patches relevant to our config:
- PN33 (default ON): spec-decode warmup K-aware sizing, vllm#37521 backport
EXTENDED beyond EAGLE to cover MTP/ngram. Sander claimed it closes both
ampersandru's mid-stream OOM AND our workspace_lock AssertionError.
- PN25 v7.66: refactored from `@torch.library.custom_op` to
`direct_register_custom_op` + `Library("genesis", "FRAGMENT")` at module
level. Schema introspection at import time eliminates the
`infer_schema skipped frame` Dynamo crash class.
- PN32 (default OFF): GDN chunked-prefill for Cliff 2 single-24GB-GPU OOM.
Cross-rig validation findings on 1×3090 TP=1
--------------------------------------------
**PN33 partial — narrows but does not close workspace_lock on TP=1.**
Sander's claim was that PN33 closes both ampersandru's mid-stream OOM
AND our workspace_lock AssertionError. Tested both:
| Test | PN33 result |
|--------------------------------------------|------------------|
| Engine boot (profile_run workspace lock) | ✅ closed |
| Runtime decode (`turboquant_attn.py:1350`) | ❌ still fires |
Engine boots cleanly without `patch_workspace_lock_disable.py` sidecar
when PN33 is on, BUT the first decode request crashes with the same
`AssertionError: Workspace is locked but allocation from
turboquant_attn.py:1350:_decode_attention requires 0.76 MB`.
Net: keep `patch_workspace_lock_disable.py` sidecar mounted. PN33
narrows the bug surface but doesn't close it for our config.
**PN25 v7.66 still doesn't work on TP=1.**
Sander's `direct_register_custom_op` + `Library("genesis", "FRAGMENT")`
approach replaces v7.65's `@torch.library.custom_op`, eliminating the
`infer_schema` skipped-frame issue. But on TP=1 the new failure mode is
`Library("genesis", "FRAGMENT")` itself failing inside dynamo trace at
`instantiate_user_defined_class_object` (different mechanism, same root
cause: Library construction inside trace context disallowed on TP=1).
Net: keep `patch_pn25_genesis_register_fix.py` v3 (import-time approach).
Our patch text-patches activation.py to register the op at module-import
time as a cached global, BEFORE any trace context exists. Survives both
the v7.65 `@custom_op` and v7.66 `Library` failure modes because we
register outside the trace entirely.
**PN30 dst-shaped temp fix carries forward cleanly.**
Our `patch_pn30_dst_shaped_temp_fix.py` anchor still matches v7.66's
PN30 wiring file. All 4 TQ3 composes still pass probes 4 + 5 (multi-turn
agent, LCB-coding) which would otherwise crash with Sander's upstream
PN30 a9977d8 (compact `.contiguous()` row-stride corruption — see
genesis-vllm-patches#17 reply for the diagnosis).
**PN31 still doesn't fit on 24 GB.** Same memory pressure as v7.65 round.
Validation matrix on v7.66
--------------------------
| Compose | Probes (verify-stress.sh) |
|--------------------|-------------------------------------------|
| long-text | 6/7 ✅ (Cliff 2 only fail) |
| long-vision | 6/7 ✅ (Cliff 2 only fail) |
| bounded-thinking | 6/7 ✅ (Cliff 2 only fail) |
| dual-turbo (TP=2) | 6/7 ✅ (Cliff 2 only fail) |
Same coverage as v7.65 + our patches. No new regressions on v7.66.
Net effect of pin bump
----------------------
- Get Sander's v7.66 + PN33 (validated improvement, even if partial)
- Get PN32 available for opt-in (Cliff 2 mitigation, untested by us)
- Same 3 local sidecars retained (PN25 v3, PN30 fix, workspace_lock)
- No simplification possible yet
Per-config + cross-rig summary in
results/v0.20-migration/v766-pin-results.summary.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
53 lines
2.6 KiB
Plaintext
53 lines
2.6 KiB
Plaintext
date: 2026-05-02
|
|
genesis_pin_bump: 753344b → fc89395 (v7.65 dev tip → v7.66 dev tip)
|
|
new_patches_active:
|
|
- PN33 spec-decode warmup K-aware (vllm#37521 backport, default ON)
|
|
- PN25 v7.66 (direct_register_custom_op refactor, default ON)
|
|
- PN30 still default OFF — opt-in
|
|
- PN31 still default OFF — opt-in
|
|
- PN32 NEW (GDN chunked-prefill for Cliff 2 single-24GB-GPU OOM, default OFF)
|
|
|
|
cross_rig_findings:
|
|
PN33_workspace_lock_test: |
|
|
Sander claimed PN33 closes both ampersandru's mid-stream OOM AND our
|
|
workspace_lock AssertionError. Cross-rig validation 2026-05-02 AM:
|
|
PN33 closes BOOT-time workspace_lock issue (engine starts cleanly without
|
|
our patch_workspace_lock_disable.py sidecar applied), BUT the runtime
|
|
workspace_lock AssertionError at `turboquant_attn.py:1350:_decode_attention`
|
|
still fires on the first decode request. Two distinct paths:
|
|
- Boot-time profile_run workspace lock: closed by PN33 ✅
|
|
- Runtime decode workspace lock: NOT closed by PN33, still needs our
|
|
patch_workspace_lock_disable.py sidecar ❌
|
|
Net: keep our sidecar, but the bug surface narrowed.
|
|
|
|
PN25_v766_test: |
|
|
Sander's v7.66 PN25 (direct_register_custom_op + Library FRAGMENT) does
|
|
NOT work on TP=1 without our local v3 import-time patch. New failure
|
|
mode is `Library("genesis", "FRAGMENT")` itself fails inside dynamo
|
|
trace at `instantiate_user_defined_class_object` (different mechanism
|
|
from v7.65's @custom_op + infer_schema, but same root cause: Library
|
|
construction inside trace context not allowed on TP=1).
|
|
Net: keep our local v3 import-time patch.
|
|
|
|
PN30_with_dst_shaped_temp_fix: |
|
|
Our local PN30 dst-shaped temp fix still applies cleanly on v7.66
|
|
Genesis tree (anchor matches). All 4 TQ3 composes pass probes 4 + 5
|
|
(multi-turn agent, LCB-coding) which would otherwise crash with
|
|
Sander's upstream PN30 a9977d8 (compact .contiguous() row-stride bug).
|
|
|
|
verify_stress_results_v766:
|
|
long-text (180K + 0.95): 6/7 (Cliff 2 only fail)
|
|
long-vision (145K + 0.95): 6/7 (Cliff 2 only fail)
|
|
bounded-thinking (180K + 0.95): 6/7 (Cliff 2 only fail)
|
|
dual-turbo (262K + 0.85, TP=2): 6/7 (Cliff 2 only fail)
|
|
|
|
local_sidecars_kept:
|
|
- patch_pn25_genesis_register_fix.py (v3 import-time approach for TP=1)
|
|
- patch_pn30_dst_shaped_temp_fix.py (replaces Sander's compact .contiguous())
|
|
- patch_workspace_lock_disable.py (PN33 closes boot-time, this still needed for runtime decode path)
|
|
- patch_tolist_cudagraph.py (cudagraph capture fix)
|
|
|
|
local_sidecars_dropped: |
|
|
None this round — PN33 narrowed but didn't close the workspace_lock
|
|
surface, all 3 sidecars still required.
|