v7.66 ships 3 new patches relevant to our config:
- PN33 (default ON): spec-decode warmup K-aware sizing, vllm#37521 backport
EXTENDED beyond EAGLE to cover MTP/ngram. Sander claimed it closes both
ampersandru's mid-stream OOM AND our workspace_lock AssertionError.
- PN25 v7.66: refactored from `@torch.library.custom_op` to
`direct_register_custom_op` + `Library("genesis", "FRAGMENT")` at module
level. Schema introspection at import time eliminates the
`infer_schema skipped frame` Dynamo crash class.
- PN32 (default OFF): GDN chunked-prefill for Cliff 2 single-24GB-GPU OOM.
Cross-rig validation findings on 1×3090 TP=1
--------------------------------------------
**PN33 partial — narrows but does not close workspace_lock on TP=1.**
Sander's claim was that PN33 closes both ampersandru's mid-stream OOM
AND our workspace_lock AssertionError. Tested both:
| Test | PN33 result |
|--------------------------------------------|------------------|
| Engine boot (profile_run workspace lock) | ✅ closed |
| Runtime decode (`turboquant_attn.py:1350`) | ❌ still fires |
Engine boots cleanly without `patch_workspace_lock_disable.py` sidecar
when PN33 is on, BUT the first decode request crashes with the same
`AssertionError: Workspace is locked but allocation from
turboquant_attn.py:1350:_decode_attention requires 0.76 MB`.
Net: keep `patch_workspace_lock_disable.py` sidecar mounted. PN33
narrows the bug surface but doesn't close it for our config.
**PN25 v7.66 still doesn't work on TP=1.**
Sander's `direct_register_custom_op` + `Library("genesis", "FRAGMENT")`
approach replaces v7.65's `@torch.library.custom_op`, eliminating the
`infer_schema` skipped-frame issue. But on TP=1 the new failure mode is
`Library("genesis", "FRAGMENT")` itself failing inside dynamo trace at
`instantiate_user_defined_class_object` (different mechanism, same root
cause: Library construction inside trace context disallowed on TP=1).
Net: keep `patch_pn25_genesis_register_fix.py` v3 (import-time approach).
Our patch text-patches activation.py to register the op at module-import
time as a cached global, BEFORE any trace context exists. Survives both
the v7.65 `@custom_op` and v7.66 `Library` failure modes because we
register outside the trace entirely.
**PN30 dst-shaped temp fix carries forward cleanly.**
Our `patch_pn30_dst_shaped_temp_fix.py` anchor still matches v7.66's
PN30 wiring file. All 4 TQ3 composes still pass probes 4 + 5 (multi-turn
agent, LCB-coding) which would otherwise crash with Sander's upstream
PN30 a9977d8 (compact `.contiguous()` row-stride corruption — see
genesis-vllm-patches#17 reply for the diagnosis).
**PN31 still doesn't fit on 24 GB.** Same memory pressure as v7.65 round.
Validation matrix on v7.66
--------------------------
| Compose | Probes (verify-stress.sh) |
|--------------------|-------------------------------------------|
| long-text | 6/7 ✅ (Cliff 2 only fail) |
| long-vision | 6/7 ✅ (Cliff 2 only fail) |
| bounded-thinking | 6/7 ✅ (Cliff 2 only fail) |
| dual-turbo (TP=2) | 6/7 ✅ (Cliff 2 only fail) |
Same coverage as v7.65 + our patches. No new regressions on v7.66.
Net effect of pin bump
----------------------
- Get Sander's v7.66 + PN33 (validated improvement, even if partial)
- Get PN32 available for opt-in (Cliff 2 mitigation, untested by us)
- Same 3 local sidecars retained (PN25 v3, PN30 fix, workspace_lock)
- No simplification possible yet
Per-config + cross-rig summary in
results/v0.20-migration/v766-pin-results.summary.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
7a7efbea0d