v7.69 + #35975 + Codex P103 gate fix — Cliff 2 closure recipes

Six rounds of cross-rig bisect with Codex/ChatGPT diagnosis closes Cliff 2 at 60K on TP=1 + 24GB. Two shippable variants: Balanced MTP (long-text.yml updated): - Genesis v7.69 dev tip (commit 2db18df) - Codex r1 P103 gate fix applied to nested Genesis tree (cu_seqlens=[0,T] treated as dense single-seq, not multi-seq varlen). Sent to Sander as v7.70 proposal — diff in /tmp/cliff2_v770_cu_seqlens_response.md. - vllm#35975 backport sidecar (patch_inputs_embeds_optional.py) - mem-util 0.93 (down from 0.95) - max_model_len 180000 (admission ceiling at this mem-util) - MTP K=3 retained - 60K probe: HTTP 200 in 623s, recall correct, MTP AL=4.00 Max-context safety (long-text-no-mtp.yml — NEW): - Same patches, but MTP off + mem-util 0.95 - max_model_len 200000 (admission unlocked by removing MTP residency) - 60K probe: HTTP 200 in 537s, recall correct - 90K probe: indeterminate within 25-min curl budget - For long single-shot RAG / codebase analysis Diagnostic chain: 1. Codex r1 identified P103 gate as too broad (cu_seqlens != None bypasses chunking even for single-seq [0,T] case). Applied fix. 2. T=4128 distribution showed chunked path never engages on real serving (vLLM's outer chunked-prefill caps T well below MAX_T). 3. Codex r2: real Cliff 2 source is residency, not gate logic. 4. PR #35975 backport (skip inputs_embeds for text-only) frees 444 MiB at boot — necessary but not sufficient at 0.95 mem-util. 5. mem-util sweep at 0.92/0.93 with MTP+#35975 closed Cliff 2 at 60K. 6. MTP-off + 0.95 + 200K admission validated max-context variant. Codex's P103 gate fix is semantically correct and worth shipping in Genesis v7.70 even though it's not what closes 60K Cliff 2 on this config (the FLA call sees T=4128 already, well below MAX_T). Full diagnostic trail: results/v0.20-migration/v769-codex-r1-test.summary Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 15:24:49 +00:00
parent e0e1752b3a
commit f6613c869a
6 changed files with 1202 additions and 71 deletions
--- a/models/qwen3.6-27b/vllm/compose/docker-compose.long-text-no-mtp.yml
+++ b/models/qwen3.6-27b/vllm/compose/docker-compose.long-text-no-mtp.yml
@@ -0,0 +1,344 @@
+# ===========================================================================
+# Long-text NO-MTP variant — max-context safety path (200K, MTP-off, no vision).
+#
+# This variant TRADES decode TPS for prefill stability. MTP K=3 spec-decode
+# is OFF, freeing the draft-model residency budget for activation headroom
+# at long single-shot prompts. Cliff 2 ceiling lifts past where it sits on
+# the MTP-on `long-text.yml`; max_model_len declared at 200K (vs 180K on
+# the MTP-on variant).
+#
+# Pick this variant for:
+#   - Long single-shot RAG queries (60K-100K context, 1 turn)
+#   - Whole-codebase analysis (TTFT-bounded workloads)
+#   - Documents Q&A where context >> response length
+#
+# DON'T pick this variant for:
+#   - Multi-turn chat, agentic coding loops — slower per-token decode
+#     hurts UX. Use `long-text.yml` (MTP-on, 180K) instead.
+#   - Tool calling — same Cliff 1 mech B caveats apply (use `tools-text.yml`)
+#
+# Stack: vLLM v0.20.1rc1.dev16 + Genesis v7.69 dev (commit 2db18df) +
+# TurboQuant 3-bit KV + MTP DISABLED + vision tower disabled.
+#
+# vllm#35975 backport sidecar (`patch_inputs_embeds_optional.py`) is
+# applied at boot — frees ~64 MiB GPU + 64 MiB pinned CPU on text-only
+# models. Drops when PR merges upstream.
+#
+# Workloads that ARE safe on this compose:
+#   - Pure long-form text generation (essay, doc Q&A) — no tool schemas in prompt
+#   - RAG without tool calling (just context + question)
+#   - Chat without tools
+# Workloads that should use `tools-text.yml` (75K + fp8 KV + PN8) instead:
+#   - Any IDE coding agent
+#   - Any flow that puts tool/function schemas in the system prompt
+#
+# Stack: vLLM v0.20.1rc1.dev16 + Genesis v7.65 dev (commit d89a089) +
+# TurboQuant 3-bit KV + MTP n=3 + vision tower disabled. Migrated from
+# vLLM dev205 + Genesis v7.64 on 2026-05-01 PM. v0.20's revised TQ FA paths
+# close the synthetic 50K-token tool-prefill stress (which fired on dev205);
+# real IDE-agent shapes still hit the inductor compile-path leak above.
+#
+# v7.65 key patches active:
+#   - P38B (Genesis #14 fix): compile-safe in-source hook for
+#     _continuation_prefill — replaces our local patch_pn12_compile_safe_
+#     custom_op.py
+#   - P15B (Genesis #15 fix): FA varlen max_seqlen_k clamp at TQ wrapper
+#     boundary — replaces our local patch_fa_max_seqlen_clamp.py
+#   - PN25: Inductor-safe silu_and_mul opaque op (covers the same surface
+#     as our former patch_pn12_compile_safe_custom_op.py)
+#   - PN26b: Genesis-original sparse-V Triton kernel for SM86 (Ampere
+#     consumer) — first SM86 sparse-V kernel in any public tree. 27B
+#     tuning: BLOCK_KV=8 num_warps=4 threshold=0.01.
+#   - Cliff 8 hardening: partial_apply_warnings counter in boot summary
+#     surfaces silent anchor-drift skips
+#
+# Recommended client max_tokens: 8192 (FREE thinking on this compose; budget
+# for hard reasoning runs). 16384 for competition-grade problems. See
+# docs/EXAMPLES.md "max_tokens defaults" section for the full table.
+#
+# When to pick this over long-vision:
+#   - You don't need image input
+#   - You want the maximum context on a single 3090
+#   - You're doing long-context text workloads (codebase analysis, RAG
+#     retrievals, multi-turn chat with steady-state accumulation)
+#
+# When NOT to pick this:
+#   - Anything that needs vision — use long-vision (198K + vision) instead
+#   - Single-shot prompts ≥50–60K tokens — Cliff 2 (DeltaNet GDN) still
+#     applies on single-card. Route those to dual.yml (TP=2 splits the
+#     state across cards) or llamacpp/default (262K, different engine).
+#
+# Local sidecar (single one remaining on this compose):
+#   - patch_workspace_lock_disable.py — relaxes vllm#39226 strict
+#     WorkspaceManager.lock() assertion to a one-shot WARNING. P98 in v7.65
+#     covers this functionally but its drift marker auto-skips on v0.20
+#     (`UNIFORM_SINGLE_TOKEN_DECODE` false-positive). Drop when Sandermage
+#     ships the marker fix.
+#
+# Variants in this dir:
+#   File                                  Ctx     KV       Vision   Notes
+#   docker-compose.yml (default)          48K     TQ3      ✅       Production-safe; below both cliffs
+#   docker-compose.long-vision.yml        198K    TQ3      ✅       Long ctx + vision (cliffs closed by v0.20 + v7.65)
+#   docker-compose.long-text.yml (this)   214K    TQ3      ❌       Engine-ceiling text-only on v0.20 + 0.985 mem-util
+#   docker-compose.bounded-thinking.yml   214K    TQ3      ❌       Long-text + structured-CoT grammar in <think>
+#   docker-compose.tools-text.yml         75K     fp8      ❌       IDE agents — fp8 KV alternative
+#
+# Run with:
+#   cd <repo>/models/qwen3.6-27b/vllm/compose
+#   docker compose -f docker-compose.long-text.yml up -d
+# ===========================================================================
+services:
+  vllm-qwen36-27b-long-text-no-mtp:
+    image: vllm/vllm-openai:nightly-7a1eb8ac2ec4ea69338c51dc7afd4b15010abfa8
+    container_name: vllm-qwen36-27b-long-text-no-mtp
+    restart: "no"
+    ports:
+      - "${PORT:-8021}:8000"
+    volumes:
+      - ${MODEL_DIR:-../../../../models-cache}:/root/.cache/huggingface
+      # torch.compile + Triton kernel caches — first boot warms (~60-90 sec
+      # for Dynamo trace + Inductor compile + Triton JIT); subsequent boots
+      # reuse cached graphs and skip recompile. Pattern from Sandermage's PROD
+      # launch (start_27b_int4_TQ_k8v4.sh:31-32). Closes club-3090 #22.
+      - ../cache/torch_compile:/root/.cache/vllm/torch_compile_cache
+      - ../cache/triton:/root/.triton/cache
+      - ../patches/genesis/vllm/_genesis:/usr/local/lib/python3.12/dist-packages/vllm/_genesis:ro
+      - ../patches/patch_tolist_cudagraph.py:/patches/patch_tolist_cudagraph.py:ro
+      # workspace_lock_disable DROPPED on v7.69 — Genesis PN34 covers the
+      # boot-time AND runtime _decode_attention paths. Opt-in via
+      # GENESIS_ENABLE_PN34_WORKSPACE_LOCK_RELAX=1 in environment block.
+      # patch_inputs_embeds_optional.py — backport of vllm#35975 (~64 MiB GPU +
+      # 64 MiB pinned CPU savings on text-only models). Drops when PR merges.
+      - ../patches/patch_inputs_embeds_optional.py:/patches/patch_inputs_embeds_optional.py:ro
+      # NOTE: patch_pn12_ffn_pool_anchor.py + patch_pn12_compile_safe_custom_op.py
+      # + patch_fa_max_seqlen_clamp.py all DROPPED — Genesis v7.65 covers their
+      # surface natively (PN12 native + PN25 + PN17 + P15B). See migration
+      # commit history if you need to resurrect.
+    environment:
+      # Uncomment the next line to pin to a specific GPU (e.g. GPU 0):
+      # - CUDA_VISIBLE_DEVICES=0
+      - HUGGING_FACE_HUB_TOKEN=${HF_TOKEN:-}
+      - VLLM_WORKER_MULTIPROC_METHOD=spawn
+      - NCCL_CUMEM_ENABLE=0
+      - NCCL_P2P_DISABLE=1
+      # v0.20 made cudagraph memory profiling default-ON (vllm#38284), eats
+      # ~0.0055 mem-util ≈ 120 MiB KV pool. Disabling restores ~4K of ctx.
+      - VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0
+      - VLLM_NO_USAGE_STATS=1
+      - PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True,max_split_size_mb:512
+      - VLLM_FLOAT32_MATMUL_PRECISION=high
+      - VLLM_USE_FLASHINFER_SAMPLER=1
+      - OMP_NUM_THREADS=1
+      - CUDA_DEVICE_MAX_CONNECTIONS=8
+      - VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
+      - VLLM_MARLIN_USE_ATOMIC_ADD=1
+      - TRITON_CACHE_DIR=/root/.triton/cache
+      # FULL Genesis v7.65 PROD env-var set per Sandermage's
+      # bare_metal_27b_int4_TQ_k8v4.sh. Validated 2026-05-01 PM dual-3090:
+      # +50% TPS over the prior subset (code 77 → 116 wall_TPS).
+      - GENESIS_ENABLE_P4=1
+      - GENESIS_ENABLE_P58_ASYNC_PLACEHOLDER_FIX=1
+      - GENESIS_ENABLE_P60_GDN_NGRAM_FIX=1
+      - GENESIS_ENABLE_P60B_TRITON_KERNEL=1
+      - GENESIS_ENABLE_P61_QWEN3_MULTI_TOOL=1
+      - GENESIS_ENABLE_P61B_STREAMING_OVERLAP=1
+      - GENESIS_ENABLE_P62_STRUCT_OUT_SPEC_TIMING=1
+      - GENESIS_ENABLE_P64_QWEN3CODER_MTP_STREAMING=1
+      - GENESIS_ENABLE_P65_TURBOQUANT_SPEC_CG_DOWNGRADE=1
+      - GENESIS_ENABLE_P66_CUDAGRAPH_SIZE_FILTER=1
+      - GENESIS_ENABLE_P67_TQ_MULTI_QUERY_KERNEL=1
+      # P68/P69 — v7.65 raised default threshold 8000 → 50000 chars (~12.5K
+      # tokens). Now safe with typical IDE-agent system+tool prompts; below
+      # the threshold both gates are no-op. Override via the env var if needed.
+      - GENESIS_ENABLE_P68_AUTO_FORCE_TOOL=1
+      - GENESIS_ENABLE_P69_LONG_CTX_TOOL_REMINDER=1
+      - GENESIS_P68_P69_LONG_CTX_THRESHOLD_CHARS=50000
+      - GENESIS_ENABLE_P72_PROFILE_RUN_CAP=1
+      - GENESIS_PROFILE_RUN_CAP_M=4128
+      - GENESIS_ENABLE_P74_CHUNK_CLAMP=1
+      - GENESIS_ENABLE_P83=1
+      - GENESIS_ENABLE_P85=1
+      - GENESIS_ENABLE_P87=1
+      - GENESIS_ENABLE_P91=1
+      - GENESIS_ENABLE_P94=1
+      - GENESIS_ENABLE_P98=1
+      - GENESIS_ENABLE_P99=1
+      - GENESIS_ENABLE_P100=1
+      - GENESIS_ENABLE_P101=1
+      # ─── Cliff 2 closure recipe (v7.69, single-24GB-GPU TP=1) ──────────
+      # P103 closes the FLA inner h tensor via chunk.py self-install hook
+      # (v7.69 F2 fix — survives `exec vllm serve` worker spawn).
+      # PN32 v2 closes the outer FLA call via _forward_core direct patch
+      # + initial_state threading (v7.69 F3 fix — composes with P103).
+      # Both required for Cliff 2 ceiling > 50K on TP=1.
+      - GENESIS_ENABLE_P103=1
+      - GENESIS_ENABLE_PN32_GDN_CHUNKED_PREFILL=1
+      - GENESIS_PN32_GDN_CHUNK_SIZE=8192
+      - GENESIS_PN32_GDN_CHUNK_THRESHOLD=16384
+      - GENESIS_FLA_FWD_H_MAX_T=16384
+      # PN34 (workspace_lock relaxation, opt-in default OFF). Replaces our
+      # patch_workspace_lock_disable.py sidecar.
+      - GENESIS_ENABLE_PN34_WORKSPACE_LOCK_RELAX=1
+      - GENESIS_ENABLE_PN8_MTP_DRAFT_ONLINE_QUANT=1
+      - GENESIS_ENABLE_PN9_INDEPENDENT_DRAFTER_ATTN=1
+      - GENESIS_ENABLE_PN11_GDN_AB_CONTIGUOUS=1
+      - GENESIS_ENABLE_PN12_FFN_INTERMEDIATE_POOL=1
+      - GENESIS_ENABLE_PN13_CUDA_GRAPH_LAMBDA_ARITY=1
+      - GENESIS_ENABLE_PN14_TQ_DECODE_OOB_CLAMP=1
+      - GENESIS_ENABLE_PN17_FA2_LSE_CLAMP=1
+      - GENESIS_ENABLE_PN25_SILU_INDUCTOR_SAFE=1
+      # PN30 — RE-ENABLED with our local dst-shaped temp fix
+      # (`patch_pn30_dst_shaped_temp_fix.py`, applied during setup.sh). The
+      # original Sander a9977d8 .contiguous() approach corrupted DS row strides
+      # by raw-memcpying a compact 10240×5 buffer into a 10240×6 destination
+      # block (row 1+ landed at the wrong destination offset, causing later
+      # TQ store CUDA assert). Our fix builds a destination-shaped temp inside
+      # `collect_mamba_copy_meta` and reuses PN30's temp-list lifetime handling.
+      # Diagnosis credit: ChatGPT/Codex CLI cross-check 2026-05-01 PM.
+      - GENESIS_ENABLE_PN30_DS_LAYOUT_SPEC_DECODE=1
+      # PN31 — DISABLED on 1×3090. Per-shape persistent VRAM grows as new
+      # shapes are seen during prefill; on TP=1 + 24GB, the residence cost
+      # outpaces the malloc-pressure relief. Sander explicitly warned in
+      # `753344b` he couldn't validate on 24GB; cross-rig validated 2026-05-01
+      # PM here that 0.95→0.93 mem-util can't carry both PN31 + DeltaNet
+      # `chunk_fwd_o` activation peak at 29K depth. Reported back on
+      # Sandermage/genesis-vllm-patches#15. Use tools-text.yml fp8 path for
+      # 25K+ tool-RETURN workloads in the meantime.
+      # - GENESIS_ENABLE_PN31_FA_VARLEN_PERSISTENT_OUT=1
+      - GENESIS_ENABLE_PN19_SCOPED_MAX_SPLIT=1
+      - GENESIS_ENABLE_PN22_LOCAL_ARGMAX_TP=1
+      # PN26b sparse-V Triton kernel — first SM86 (Ampere consumer) sparse-V
+      # kernel in any public tree. 27B-specific tuning per Sandermage's PROD.
+      - GENESIS_ENABLE_PN26_SPARSE_V=1
+      - GENESIS_PN26_SPARSE_V_BLOCK_KV=8
+      - GENESIS_PN26_SPARSE_V_NUM_WARPS=4
+      - GENESIS_PN26_SPARSE_V_THRESHOLD=0.01
+      # v7.65 #14 + #15 fixes (compile-safe in-source hook + FA varlen clamp).
+      # Replace our local patch_pn12_compile_safe_custom_op.py + P104 sidecars.
+      - GENESIS_ENABLE_P38B_COMPILE_SAFE=1
+      - GENESIS_ENABLE_P15B_FA_VARLEN_CLAMP=1
+      - GENESIS_PREALLOC_TOKEN_BUDGET=4128
+      - GENESIS_BUFFER_MODE=shared
+      # Explicit OFFs to match Sandermage's PROD env-var set verbatim:
+      # P78 (P78_TOLIST_CAPTURE_GUARD) — superseded by our patch_tolist_cudagraph.py
+      # P81 (FP8 block-scaled M<=8) — FP8-specific, no-op on our TQ3 path
+      # P82 — biased on small-batch single-stream Lorbus INT4 + MTP K=3 (Sander PROD)
+      - GENESIS_ENABLE_P78_TOLIST_CAPTURE_GUARD=0
+      - GENESIS_ENABLE_P81_FP8_BLOCK_SCALED_M_LE_8=0
+      - GENESIS_ENABLE_P82=0
+      - GENESIS_P82_THRESHOLD_SINGLE=0.3
+      # P82 stays OFF — biased on small-batch single-stream Lorbus INT4 + MTP K=3
+      # per Sandermage's PROD memory feedback_p82_*. P78 stays OFF (deprecated).
+      # vLLM env knobs from Sandermage's launch:
+      # VLLM_SSM_CONV_STATE_LAYOUT=DS — RE-ENABLED with our local PN30 fix
+      # (`patch_pn30_dst_shaped_temp_fix.py`, applied during setup.sh).
+      # Without our fix, Sander's PN30 a9977d8 corrupts DS row strides on
+      # spec-decode AL>1 paths. With our fix, PN30 builds a destination-shaped
+      # temp instead of a compact one, preserving DS layout. +6% TPS retained.
+      - VLLM_SSM_CONV_STATE_LAYOUT=DS
+      - VLLM_USE_FUSED_MOE_GROUPED_TOPK=1
+    shm_size: "16gb"
+    ipc: host
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              count: all
+              capabilities: [gpu]
+    entrypoint:
+      - /bin/bash
+      - -c
+      - |
+        set -e
+        pip install xxhash pandas scipy -q
+        # Pre-flight: Genesis patches must be populated. Empty volume mount
+        # = silent no-op apply_all = boot fails later with cryptic upstream
+        # error (e.g. "TurboQuant KV not supported for hybrid models", #13).
+        if [ ! -f /usr/local/lib/python3.12/dist-packages/vllm/_genesis/patches/apply_all.py ]; then
+          echo "ERROR: Genesis patches missing — host volume models/qwen3.6-27b/vllm/patches/genesis/ is empty." >&2
+          echo "       Run from repo root before 'docker compose up':" >&2
+          echo "         bash scripts/setup.sh qwen3.6-27b" >&2
+          exit 1
+        fi
+        python3 -m vllm._genesis.patches.apply_all
+        # patch_tolist_cudagraph.py — still needed pending upstream
+        # vllm#40807 .tolist() capture-stream guard fix.
+        python3 /patches/patch_tolist_cudagraph.py
+        # patch_inputs_embeds_optional.py — backport of vllm#35975
+        # (free ~64 MiB GPU + ~64 MiB pinned CPU on text-only models).
+        # Drops when PR merges upstream.
+        python3 /patches/patch_inputs_embeds_optional.py
+        exec vllm serve "$@"
+      - --
+    command:
+      - --model
+      - /root/.cache/huggingface/qwen3.6-27b-autoround-int4
+      - --served-model-name
+      - qwen3.6-27b-autoround
+      - --quantization
+      - auto_round
+      - --dtype
+      - float16
+      - --tensor-parallel-size
+      - "1"
+      # 180K + 0.97 — backed off from 214K + 0.985 on 2026-05-01 PM after
+      # verify-stress probe 1 (10K-token long-context needle) crashed with
+      # GDN forward OOM (`fla.ops.chunk.chunk_gated_delta_rule_fwd_h`
+      # `k.new_empty(B, NT, H, V, K)` requested 98 MiB, 46 MiB free). At
+      # 0.985 mem-util, activation budget was ~46 MiB at 10K prefill —
+      # tight enough that ANY single prompt over a few thousand tokens was
+      # a coin flip. The "Cliff 2 fires only at 50-60K" framing was
+      # config-specific to the dev205 + 0.92 default; at tighter mem-util
+      # the GDN forward buffer outpaces available activation much earlier.
+      #
+      # New target (validated 2026-05-01 PM via verify-stress.sh):
+      # 180K + 0.97 → ~6.4 GB after weights, ~5.9 GB KV pool, ~500 MiB
+      # activation safety. Probe 1 (10K needle) and probe 6 (math reasoning
+      # at max_tokens=8192) PASS. Safe single-prompt envelope ~30K; full
+      # 180K context for steady-state turn accumulation only.
+      #
+      # IDE-agent workloads (sys + tool schemas) STILL crash here per
+      # club-3090#16 — different bug (Cliff 1 mech B inductor leak), not
+      # fixed by mem-util tuning. Use tools-text.yml for those.
+      - --max-model-len
+      - "200000"
+      # 0.95 — backed off from 0.97 on 2026-05-01 PM after PN25 v3 closed
+      # Cliff 1 mech B at the FFN intermediate buffer (which now sits in
+      # PN12+PN25's resident FFNIntermediateCache pool). Pool residence
+      # adds ~140 MiB persistent VRAM, which tightened activation budget
+      # at DeltaNet `chunk_fwd_o` (`o = torch.empty_like(v)`, 50 MiB) —
+      # 0.97 left only 26 MiB free, OOM at 9.8K probe. 0.95 frees ~480 MiB
+      # activation budget, validated up to 30K probes.
+      - --gpu-memory-utilization
+      - "0.95"
+      - --max-num-seqs
+      - "1"
+      - --max-num-batched-tokens
+      - "4128"
+      - --kv-cache-dtype
+      - turboquant_3bit_nc
+      # Drops the vision tower → frees ~1 GB VRAM → lifts engine ceiling from
+      # 198K (long-vision at 0.98) to 218K (this variant at 0.985). This is the only difference vs long-vision.
+      - --language-model-only
+      - --trust-remote-code
+      - --reasoning-parser
+      - qwen3
+      - --enable-auto-tool-choice
+      - --tool-call-parser
+      - qwen3_coder
+      - --enable-prefix-caching
+      - --enable-chunked-prefill
+      - --no-scheduler-reserve-full-isl
+      # MTP K=3 spec-decode INTENTIONALLY DISABLED on this variant. Frees
+      # ~280 MiB of draft-model residency on top of #35975's ~444 MiB savings,
+      # unlocking max_model_len=200K admission (vs 180K cap with MTP-on).
+      # Trade-off: lose ~3-4x decode TPS speedup vs `long-text.yml` (MTP-on).
+      # Pick this variant for long single-shot RAG / codebase prompts where
+      # TTFT-bounded throughput matters more than per-token decode speed.
+      # - --speculative-config
+      # - '{"method":"mtp","num_speculative_tokens":3}'
+      - --host
+      - 0.0.0.0
+      - --port
+      - "8000"
--- a/models/qwen3.6-27b/vllm/compose/docker-compose.long-text.yml
+++ b/models/qwen3.6-27b/vllm/compose/docker-compose.long-text.yml
@@ -21,25 +21,46 @@
 #   - Any IDE coding agent
 #   - Any flow that puts tool/function schemas in the system prompt
 #
-# Stack: vLLM v0.20.1rc1.dev16 + Genesis v7.65 dev (commit d89a089) +
-# TurboQuant 3-bit KV + MTP n=3 + vision tower disabled. Migrated from
-# vLLM dev205 + Genesis v7.64 on 2026-05-01 PM. v0.20's revised TQ FA paths
-# close the synthetic 50K-token tool-prefill stress (which fired on dev205);
-# real IDE-agent shapes still hit the inductor compile-path leak above.
+# Stack: vLLM v0.20.1rc1.dev16 + Genesis v7.69 dev (commit 2db18df) +
+# TurboQuant 3-bit KV + MTP n=3 + vision tower disabled.
 #
-# v7.65 key patches active:
-#   - P38B (Genesis #14 fix): compile-safe in-source hook for
-#     _continuation_prefill — replaces our local patch_pn12_compile_safe_
-#     custom_op.py
-#   - P15B (Genesis #15 fix): FA varlen max_seqlen_k clamp at TQ wrapper
-#     boundary — replaces our local patch_fa_max_seqlen_clamp.py
-#   - PN25: Inductor-safe silu_and_mul opaque op (covers the same surface
-#     as our former patch_pn12_compile_safe_custom_op.py)
+# v7.69 key patches active:
+#   - P103 (Cliff 2 chunked fwd_h+fwd_o orchestrator) — chunk.py self-install
+#     hook survives `exec vllm serve` worker spawn (v7.69 F2 fix). Also
+#     applied: Codex r1 cu_seqlens=[0,T] dense-single-seq gate fix that
+#     allows the chunked path to engage on real serving.
+#   - PN30 v7.68 part1+2+3 — DS conv state layout + spec-decode AL>1 fix
+#     (drift-marker bug fixed in v7.69)
+#   - PN32 v2 — GDN _forward_core chunked-prefill (Cliff 2 outer split)
+#   - PN34 — workspace_lock relaxation (replaces our former
+#     patch_workspace_lock_disable.py sidecar)
 #   - PN26b: Genesis-original sparse-V Triton kernel for SM86 (Ampere
 #     consumer) — first SM86 sparse-V kernel in any public tree. 27B
 #     tuning: BLOCK_KV=8 num_warps=4 threshold=0.01.
-#   - Cliff 8 hardening: partial_apply_warnings counter in boot summary
-#     surfaces silent anchor-drift skips
+#   - P38B / P15B: Cliff 1 mech B compile-safe hook + FA varlen clamp
+#
+# Cliff 2 closure recipe (cross-rig validated 2026-05-02 PM):
+#   - vllm#35975 backport sidecar (`patch_inputs_embeds_optional.py`) frees
+#     ~444 MiB at boot on text-only models (skips inputs_embeds buffers
+#     for both gpu_model_runner.py + llm_base_proposer.py).
+#   - --gpu-memory-utilization 0.93 — middle ground between 0.92 (cliff
+#     just barely closed) and 0.95 (cliff fires). Recovers ~30K of KV
+#     pool concurrency vs 0.92 while keeping ~494 MiB margin at peak.
+#   - --max-model-len 180000 — engine pre-check admission ceiling on
+#     this config (200K rejected: KV pool needs 3.31 GiB, only 2.19 GiB
+#     available at 0.93). Cliff 2 single-prompt safe envelope: ~60K
+#     verified pass; ~70-80K is the OOM ceiling.
+#
+# Validation (2026-05-02 PM, branch v7.69-cliff2-test):
+#   - 60K MTP-on probe: HTTP 200 in 623s, recall correct, MTP AL=4.00
+#   - 90K MTP-on probe: HTTP 500 OOM at 112s (50 MiB / 24.5 MiB free)
+#   - Probes 2-6 (tool prefill, IDE-agent, multi-turn, LCB-coding,
+#     reasoning-heavy): all PASS
+#
+# For workloads that need >60K single-prompt or 200K declared context,
+# use docker-compose.long-text-no-mtp.yml (max-context safety variant
+# trades MTP decode speedup for prefill stability). For coding agents
+# on dual-card, use docker-compose.dual-turbo.yml.
 #
 # Recommended client max_tokens: 8192 (FREE thinking on this compose; budget
 # for hard reasoning runs). 16384 for competition-grade problems. See
@@ -93,10 +114,12 @@ services:
      - ../cache/triton:/root/.triton/cache
      - ../patches/genesis/vllm/_genesis:/usr/local/lib/python3.12/dist-packages/vllm/_genesis:ro
      - ../patches/patch_tolist_cudagraph.py:/patches/patch_tolist_cudagraph.py:ro
-      # workspace_lock_disable — relaxes vllm#39226 strict assertion for the TQ
-      # decode path on v0.20. Drop when Sandermage ships the P98 marker fix
-      # that auto-enables on v0.20 without our help. See docs/CLIFFS.md.
-      - ../patches/patch_workspace_lock_disable.py:/patches/patch_workspace_lock_disable.py:ro
+      # workspace_lock_disable DROPPED on v7.69 — Genesis PN34 covers the
+      # boot-time AND runtime _decode_attention paths. Opt-in via
+      # GENESIS_ENABLE_PN34_WORKSPACE_LOCK_RELAX=1 in environment block.
+      # patch_inputs_embeds_optional.py — backport of vllm#35975 (~64 MiB GPU +
+      # 64 MiB pinned CPU savings on text-only models). Drops when PR merges.
+      - ../patches/patch_inputs_embeds_optional.py:/patches/patch_inputs_embeds_optional.py:ro
      # NOTE: patch_pn12_ffn_pool_anchor.py + patch_pn12_compile_safe_custom_op.py
      # + patch_fa_max_seqlen_clamp.py all DROPPED — Genesis v7.65 covers their
      # surface natively (PN12 native + PN25 + PN17 + P15B). See migration
@@ -152,7 +175,20 @@ services:
      - GENESIS_ENABLE_P99=1
      - GENESIS_ENABLE_P100=1
      - GENESIS_ENABLE_P101=1
+      # ─── Cliff 2 closure recipe (v7.69, single-24GB-GPU TP=1) ──────────
+      # P103 closes the FLA inner h tensor via chunk.py self-install hook
+      # (v7.69 F2 fix — survives `exec vllm serve` worker spawn).
+      # PN32 v2 closes the outer FLA call via _forward_core direct patch
+      # + initial_state threading (v7.69 F3 fix — composes with P103).
+      # Both required for Cliff 2 ceiling > 50K on TP=1.
      - GENESIS_ENABLE_P103=1
+      - GENESIS_ENABLE_PN32_GDN_CHUNKED_PREFILL=1
+      - GENESIS_PN32_GDN_CHUNK_SIZE=8192
+      - GENESIS_PN32_GDN_CHUNK_THRESHOLD=16384
+      - GENESIS_FLA_FWD_H_MAX_T=16384
+      # PN34 (workspace_lock relaxation, opt-in default OFF). Replaces our
+      # patch_workspace_lock_disable.py sidecar.
+      - GENESIS_ENABLE_PN34_WORKSPACE_LOCK_RELAX=1
      - GENESIS_ENABLE_PN8_MTP_DRAFT_ONLINE_QUANT=1
      - GENESIS_ENABLE_PN9_INDEPENDENT_DRAFTER_ATTN=1
      - GENESIS_ENABLE_PN11_GDN_AB_CONTIGUOUS=1
@@ -236,10 +272,13 @@ services:
          exit 1
        fi
        python3 -m vllm._genesis.patches.apply_all
-        # workspace_lock_disable — relaxes vllm#39226 strict assertion until
-        # Sandermage's P98 marker fix lands.
-        python3 /patches/patch_workspace_lock_disable.py
+        # patch_tolist_cudagraph.py — still needed pending upstream
+        # vllm#40807 .tolist() capture-stream guard fix.
        python3 /patches/patch_tolist_cudagraph.py
+        # patch_inputs_embeds_optional.py — backport of vllm#35975
+        # (free ~64 MiB GPU + ~64 MiB pinned CPU on text-only models).
+        # Drops when PR merges upstream.
+        python3 /patches/patch_inputs_embeds_optional.py
        exec vllm serve "$@"
      - --
    command:
@@ -282,7 +321,7 @@ services:
      # 0.97 left only 26 MiB free, OOM at 9.8K probe. 0.95 frees ~480 MiB
      # activation budget, validated up to 30K probes.
      - --gpu-memory-utilization
-      - "0.95"
+      - "0.93"
      - --max-num-seqs
      - "1"
      - --max-num-batched-tokens
--- a/models/qwen3.6-27b/vllm/patches/patch_inputs_embeds_optional.py
+++ b/models/qwen3.6-27b/vllm/patches/patch_inputs_embeds_optional.py
@@ -0,0 +1,180 @@
+#!/usr/bin/env python3
+"""
+patch_inputs_embeds_optional.py — backport of vllm-project/vllm#35975
+
+Skips inputs_embeds GPU buffer allocation for text-only models. Saves
+~64 MiB GPU + ~64 MiB pinned CPU on a config that doesn't use multimodal
+inputs or prompt embeddings.
+
+Why we ship it locally:
+  PR #35975 is open upstream as of 2026-05-02 (awaiting code-owner approval
+  after author addressed reviewer feedback). On club-3090's TP=1 + 24GB
+  Qwen3.6-27B + MTP K=3 + 0.95 mem-util config, Cliff 2 at 60K fires with
+  ~24.5 MiB free at the time of the failing 50 MiB allocation. PR #35975
+  frees ~64 MiB GPU + ~64 MiB pinned CPU on text-only models — sufficient
+  margin to potentially close Cliff 2 at 60K MTP-on without reducing
+  context or mem-util.
+
+  This sidecar exists because we don't want to fork the docker image just
+  for one PR. When PR #35975 merges upstream and ships in our nightly tag,
+  this sidecar can be deleted.
+
+References:
+  - https://github.com/vllm-project/vllm/pull/35975
+  - club-3090 results/v0.20-migration/v769-codex-r1-test.summary
+
+Idempotent. Safe to re-run.
+"""
+import logging
+import re
+import sys
+from pathlib import Path
+
+log = logging.getLogger("inputs_embeds_optional")
+logging.basicConfig(level=logging.INFO, format="[%(name)s] %(message)s")
+
+PATCH_TAG = "[inputs_embeds_optional]"
+
+
+def patch_gpu_model_runner() -> bool:
+    """
+    File: vllm/v1/worker/gpu_model_runner.py
+
+    Before:
+        self.inputs_embeds = self._make_buffer(
+            self.max_num_tokens, self.inputs_embeds_size, dtype=self.dtype, numpy=False
+        )
+
+    After:
+        # vllm#35975: skip inputs_embeds buffer for text-only models.
+        self.inputs_embeds = None
+        if self.supports_mm_inputs or self.enable_prompt_embeds:
+            self.inputs_embeds = self._make_buffer(
+                self.max_num_tokens, self.inputs_embeds_size, dtype=self.dtype, numpy=False
+            )
+    """
+    target = Path(
+        "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py"
+    )
+    if not target.exists():
+        log.error(f"{PATCH_TAG} target not found: {target}")
+        return False
+
+    src = target.read_text()
+    if PATCH_TAG in src:
+        log.info(f"{PATCH_TAG} gpu_model_runner.py already patched, skipping.")
+        return True
+
+    # Find the exact allocation site. Use a regex that matches multi-line.
+    # Keep the `numpy=False` argument literal.
+    pattern = re.compile(
+        r"^( *)self\.inputs_embeds = self\._make_buffer\(\n"
+        r"\1    self\.max_num_tokens, self\.inputs_embeds_size, dtype=self\.dtype, numpy=False\n"
+        r"\1\)$",
+        re.MULTILINE,
+    )
+    match = pattern.search(src)
+    if not match:
+        log.error(
+            f"{PATCH_TAG} gpu_model_runner.py: anchor not found "
+            f"(self.inputs_embeds = self._make_buffer block). vLLM may have "
+            f"changed; review PR #35975 manually."
+        )
+        return False
+
+    indent = match.group(1)
+    replacement = (
+        f"{indent}# {PATCH_TAG} vllm#35975: skip inputs_embeds buffer for text-only models.\n"
+        f"{indent}self.inputs_embeds = None\n"
+        f"{indent}if self.supports_mm_inputs or self.enable_prompt_embeds:\n"
+        f"{indent}    self.inputs_embeds = self._make_buffer(\n"
+        f"{indent}        self.max_num_tokens, self.inputs_embeds_size, dtype=self.dtype, numpy=False\n"
+        f"{indent}    )"
+    )
+    new_src = pattern.sub(replacement, src, count=1)
+    target.write_text(new_src)
+    log.info(f"{PATCH_TAG} gpu_model_runner.py: applied")
+    return True
+
+
+def patch_llm_base_proposer() -> bool:
+    """
+    File: vllm/v1/spec_decode/llm_base_proposer.py
+    (PR #35975 originally targeted eagle.py; in this nightly the code lives
+     in the shared base class llm_base_proposer.py.)
+
+    Before:
+        self.inputs_embeds = torch.zeros(
+            (self.max_num_tokens, self.inputs_embeds_size),
+            dtype=self.dtype,
+            device=device,
+        )
+
+    After:
+        # vllm#35975: skip inputs_embeds tensor for text-only proposers.
+        self.inputs_embeds = None
+        if self.supports_mm_inputs:
+            self.inputs_embeds = torch.zeros(
+                (self.max_num_tokens, self.inputs_embeds_size),
+                dtype=self.dtype,
+                device=device,
+            )
+    """
+    target = Path(
+        "/usr/local/lib/python3.12/dist-packages/vllm/v1/spec_decode/llm_base_proposer.py"
+    )
+    if not target.exists():
+        log.error(f"{PATCH_TAG} target not found: {target}")
+        return False
+
+    src = target.read_text()
+    if PATCH_TAG in src:
+        log.info(f"{PATCH_TAG} llm_base_proposer.py already patched, skipping.")
+        return True
+
+    pattern = re.compile(
+        r"^( *)self\.inputs_embeds = torch\.zeros\(\n"
+        r"\1    \(self\.max_num_tokens, self\.inputs_embeds_size\),\n"
+        r"\1    dtype=self\.dtype,\n"
+        r"\1    device=device,\n"
+        r"\1\)$",
+        re.MULTILINE,
+    )
+    match = pattern.search(src)
+    if not match:
+        log.error(
+            f"{PATCH_TAG} llm_base_proposer.py: anchor not found "
+            f"(self.inputs_embeds = torch.zeros block). vLLM may have changed."
+        )
+        return False
+
+    indent = match.group(1)
+    replacement = (
+        f"{indent}# {PATCH_TAG} vllm#35975: skip inputs_embeds tensor for text-only proposers.\n"
+        f"{indent}self.inputs_embeds = None\n"
+        f"{indent}if self.supports_mm_inputs:\n"
+        f"{indent}    self.inputs_embeds = torch.zeros(\n"
+        f"{indent}        (self.max_num_tokens, self.inputs_embeds_size),\n"
+        f"{indent}        dtype=self.dtype,\n"
+        f"{indent}        device=device,\n"
+        f"{indent}    )"
+    )
+    new_src = pattern.sub(replacement, src, count=1)
+    target.write_text(new_src)
+    log.info(f"{PATCH_TAG} llm_base_proposer.py: applied")
+    return True
+
+
+def main() -> int:
+    ok = True
+    ok &= patch_gpu_model_runner()
+    ok &= patch_llm_base_proposer()
+    if ok:
+        log.info(f"{PATCH_TAG} all patches applied successfully")
+        return 0
+    log.error(f"{PATCH_TAG} one or more patches failed")
+    return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())
--- a/results/v0.20-migration/v769-codex-r1-test.summary
+++ b/results/v0.20-migration/v769-codex-r1-test.summary
@@ -0,0 +1,404 @@
+# v7.69 + Codex round 1 P103 gate fix — verification on TP=1 + 24GB
+Date: 2026-05-02 PM
+Genesis pin: 2db18df (v7.69 dev tip) + Codex's gate-fix patch applied to in-tree
+P103: patched to treat cu_seqlens=[0,T] as dense single-seq (not multi-seq varlen)
+PN32 v2: enabled
+PN34: enabled (workspace_lock relax)
+DS layout: enabled
+GENESIS_FLA_FWD_H_MAX_T: tested at both 16384 (default) and 2048
+
+## Ground truth from diagnostic log
+
+Added `print` at the gate to capture T, q.shape, cu_shape, _single_seq_cu, _true_varlen_multi_seq.
+
+### Run 1: GENESIS_FLA_FWD_H_MAX_T=16384 (Codex default)
+
+T distribution across 442 invocations on 60K probe:
+  T=4128: 394 invocations  (vLLM chunked-prefill chunk size)
+  T=64:    48 invocations  (cudagraph warmup or MTP verify)
+  T>4128:   0 invocations
+
+q.shape[0] = 1 always (B=1, single sequence)
+cu_shape = torch.Size([2]) always (single segment, [0, T])
+_single_seq_cu = True for every invocation
+_true_varlen_multi_seq = False for every invocation
+
+Gate verdict: gate-fix is semantically correct but `q.shape[1] <= _MAX_T`
+(4128 <= 16384) bypasses to original_fwd on EVERY call. Chunked path
+never engages.
+
+OOM: same trace as v7.69 baseline. 50.00 MiB alloc fails, 24.50 MiB free.
+
+### Run 2: GENESIS_FLA_FWD_H_MAX_T=2048 (force chunking on T=4128 calls)
+
+T distribution across 436 invocations:
+  T=4128: 340 invocations  (now ENTERING chunked path: 4128 > 2048)
+  T=64:    48 invocations
+  T=11:    48 invocations  (smaller residuals on chunked path)
+
+Per-call allocation: 24.00 MiB (half of previous 50.00 MiB)
+Cumulative active PyTorch: 23.01 GiB (vs 22.96 GiB on run 1)
+OOM trigger: failed at call ~336/340 (vs ~393/394 on run 1)
+
+Net change: chunking DID reduce per-call peak by half, but cumulative
+state across chunked sub-calls grew slightly (chained state buffers
+between chunks). OOM fires earlier in absolute call count.
+
+## Cumulative diagnosis
+
+The gate-fix Codex proposed is correct in principle:
+  cu_seqlens.shape == (2,) IS dense single-seq, not multi-seq varlen.
+  P103 v7.70 should NOT bypass on cu_seqlens != None when shape == (2,).
+
+But on our TP=1 + 24GB + MTP K=3 + 0.95 mem-util config, the chunked
+path:
+  (a) doesn't engage at MAX_T=16384 because vLLM's outer chunked
+      prefill already caps T at max_num_batched_tokens=4128
+  (b) does engage at MAX_T=2048 but doesn't help because the OOM
+      isn't a single-allocation-too-large problem — it's
+      cumulative-state-too-large
+
+Current allocation breakdown at OOM:
+  Total in use: 23.55 GiB / 23.56 GiB (99.9%)
+  PyTorch active: 23.01 GiB
+    Of which:
+      Model weights:       16.87 GiB (boot log)
+      MTP draft sharing:    0      GiB (shared via embedding+lm_head)
+      KV pool @ 60K TQ3:   ~1     GiB (rough estimate, ~17 KB/tok)
+      Mamba conv state:    ~0.5   GiB (DS layout + GDN forward)
+      PN12/PN25 pools:     ~0.2   GiB
+      FLA temp buffers:    ~0.2   GiB
+      cudagraph private:    0.024 GiB
+      Other PyTorch alloc: ~4     GiB (UNACCOUNTED — likely activations)
+
+The 4 GiB "unaccounted" is the actual headroom problem. Each prefill
+chunk's activations stick around in some lifetime overlap until the
+next chunk's allocation, fragmenting expandable-segments effective
+headroom.
+
+## Conclusions
+
+1. Codex's gate-fix should still ship to Sander as v7.70 — it's
+   semantically correct and unblocks the synthetic single-seq path.
+2. But it does NOT close real-config Cliff 2 at 60K on 24GB.
+3. The actual fix is one of:
+   (a) Lower mem-util to ~0.90 (frees ~700 MiB headroom, costs ~10K ctx)
+   (b) Cap max-model-len to 50K and route 50K+ to TP=2 (`dual.yml`)
+   (c) Genesis-side: tighter activation residency in PN12/PN25/PN30
+       pools + audit FLA temp buffer reuse
+   (d) vLLM-side: scheduler pause prefill chunks when VRAM headroom
+       drops below threshold
+
+Practical recommendation for users on TP=1 + 24GB:
+  Cliff 2 ceiling stays ~50-60K on this config regardless of v7.69
+  patches. Treat it as architectural + use dual.yml for longer prompts.
+
+## Round 2 prompt for Codex queued at /tmp/codex_prompt_cliff2_round2.md
+
+Asks ChatGPT/Codex to:
+  - Diagnose where the 4 GiB unaccounted active PyTorch allocation lives
+  - Recommend instrumentation to bisect (memory_summary per call?)
+  - Propose either Genesis-side or vLLM-side fix that doesn't require
+    per-call chunking (since chunking doesn't help here)
+
+## State of in-tree files
+
+- `models/qwen3.6-27b/vllm/patches/genesis/vllm/_genesis/wiring/hybrid/patch_103_fla_cliff2_chunked.py`
+  has Codex's gate-fix applied + temp diagnostic log. Will revert
+  diagnostic log before any commit.
+- `models/qwen3.6-27b/vllm/compose/docker-compose.long-text.yml`
+  has GENESIS_FLA_FWD_H_MAX_T=2048 (testing value). Needs revert to
+  16384 (or whatever final value) before commit.
+
+
+## ROUND 2 — Codex r2 + user PR research bisect (2026-05-02 PM)
+
+### Boot resident snapshot
+
+MTP-on  : 23,156 MiB (model weights + MTP draft 0.28 GiB shared + KV preallocation + Genesis pools)
+MTP-off : 23,164 MiB (same minus MTP, ~280 MiB savings — MTP draft shares embedding+lm_head)
+
+Headroom at 0.95 mem-util ≈ 1,400 MiB before any request.
+
+### MTP-off A/B at 60K — single-variable change
+
+Removed only `--speculative-config '{"method":"mtp",...}'`. All else
+unchanged: P103 (with Codex gate fix) + PN30 + PN32 v2 + PN34 + DS layout
+ MAX_T=16384 + max_num_batched_tokens=4128 + max-model-len=180000 +
+mem-util=0.95.
+
+Manual curl with 600s timeout:
+  HTTP 200 ✅ recalled "violet otter 48" correctly
+  Wall: 504s
+  Engine logs: prompt throughput 227 tok/s, gen 7.5 tok/s, KV usage 6.1%
+  No OOM, no errors
+
+verify-stress harness curl (5 min default timeout):
+  HTTP 000 (harness timed out before engine completed)
+  Engine itself returned 200 a few seconds after harness gave up — both
+  the harness curl AND the manual curl saw the same engine completion;
+  the harness's 5-min cap is the fail signal, not engine correctness.
+
+**Conclusion: MTP-off closes the 60K Cliff 2 cleanly.** Cliff 2 on
+TP1 + 24GB + 0.95 mem-util is dominated by MTP draft residency +
+activation footprint, not P103 cu_seqlens gate.
+
+### MTP-off at 90K — indeterminate within 15 min
+
+Manual curl with 900s (15 min) timeout:
+  HTTP 000 (curl timed out)
+  Wall 900s
+  No OOM trace
+  GPU 0 still at 100% utilization
+  No POST 200 in engine logs after 13:42:37
+  Engine never completed within practical wait
+
+Practical interpretation: 90K MTP-off is past the wall-clock-feasible
+ceiling on this config even without OOM. Throughput drops as
+cumulative state grows; what would take ~7 min at constant 227 tok/s
+takes 25+ min in practice (and may eventually OOM internally without
+reaching us). For users on TP1+24GB, **MTP-off shifts the practical
+ceiling from ~30-60K to ~60K**, but doesn't push to 90K.
+
+### Updated upstream PR research (provided by user)
+
+Three open PRs that could compose with our findings:
+
+| PR | What | Why it matters |
+|---|---|---|
+| [vllm#35975](https://github.com/vllm-project/vllm/pull/35975) | Skip inputs_embeds GPU buffer for text-only models | ~64 MiB GPU + 64 MiB pinned CPU savings. Direct match for our 50 MiB margin failure. **Highest-value test next.** |
+| [vllm#37521](https://github.com/vllm-project/vllm/pull/37521) | Spec-decode warmup memory accounting | Profiling/KV sizing leaves less false headroom. Worth testing for MTP-on path. |
+| [vllm#37429](https://github.com/vllm-project/vllm/pull/37429) | KV cache sizing for hybrid Mamba/attention | Architectural — could free substantial headroom on hybrid Qwen3-Next models. Bigger/riskier. |
+
+vLLM's official Qwen3-Next recipe documents `--tensor-parallel-size 4
+--gpu-memory-utilization 0.8 --no-enable-chunked-prefill` as happy
+path. Our 24GB TP1 long-context MTP case is OUTSIDE the documented
+configuration — context worth flagging to Sander.
+
+### Next: apply PR #35975 as text-patch + retest 60K MTP-on
+
+Plan: modify `vllm/v1/worker/gpu_model_runner.py:623-633` and
+`vllm/v1/spec_decode/eagle.py:192-201` to skip `inputs_embeds` buffer
+allocation when not needed (no MM inputs, no prompt_embeds). 64 MiB
+saved on text-only model + 64 MiB pinned CPU. If that turns 60K
+MTP-on from -25 MiB to +39 MiB free, we have a real closure proof.
+
+
+## ROUND 3 — vllm#35975 backport + mem-util tuning matrix (2026-05-02 PM)
+
+### Closure recipe: PR#35975 + mem-util 0.92 + MTP-on
+
+**60K probe RESULT: HTTP 200 OK, recall correct, MTP AL=4.00**
+
+| Config | Boot resident | OOM/Pass at 60K | Wall | Notes |
+|---|---|---|---|---|
+| MTP-off, 0.95 mem-util | 23,164 MiB | ✅ Pass | 504s | Already established (Round 2) |
+| MTP-on + #35975 + 0.95 mem-util | 22,720 MiB | ❌ Fails 46.5 MiB free / 50 needed | n/a | #35975 freed ~444 MiB at boot, but only ~22 MiB extra at peak |
+| **MTP-on + #35975 + 0.92 mem-util** | **21,980 MiB** | **✅ Pass** | **~11.5 min wall** | **Cliff 2 closed** with full MTP K=3 + correct recall |
+
+Notes on the MTP-on + #35975 + 0.92 run:
+- Boot resident dropped from 22,720 (at 0.95) to 21,980 (at 0.92) — saved 740 MiB
+- KV pool: 3.87 GiB / 305,472 tokens (vs 5.07 GiB / 363,264 at 0.95) — 1.43x → 1.25x concurrency at 180K
+- Wall ~11.5 min (vs 8.4 min on MTP-off + 0.95) — slower than expected; investigation TBD
+- MTP per-position acceptance: 1.000, 1.000, 1.000 (perfect on this prompt)
+
+### Codex r2 verdict (post-bisect)
+
+> #35975 is definitely relevant, but not sufficient alone. MTP-off proving 60K pass means draft residency is the dominant pressure. #35975 + gpu_memory_utilization=0.92 is now the right "keep MTP" candidate. P103 streaming output is lower priority unless 0.92 still fails or you need to recover more context at fixed 0.95.
+
+### Two shippable variants
+
+**Variant A — Long-text with MTP** (`long-text.yml`):
+- Genesis v7.69 + Codex P103 gate fix + PN30 + PN32 + PN34
+- vllm#35975 backport sidecar (`patch_inputs_embeds_optional.py`)
+- `--gpu-memory-utilization 0.92` (down from 0.95)
+- `--speculative-config '{"method":"mtp","num_speculative_tokens":3}'` (retained)
+- Cliff 2 closed at 60K. Probe 7 ceiling moves from 30-60K → 60K+.
+
+**Variant B — Long-text without MTP** (`long-text-no-mtp.yml` — TODO):
+- Same as A but `--speculative-config` REMOVED, `--gpu-memory-utilization 0.95` (recovered)
+- Cliff 2 closed at 60K with full mem-util (KV pool 1.43x at 180K)
+- Lower decode throughput (no MTP speedup)
+- Trade-off: more KV/context vs slower decode
+
+### Codex r2's recommended next bisect (open)
+
+1. ~~#35975 + MTP-on + 0.92: establish pass~~ ✅ DONE (this run)
+2. **#35975 + MTP-on + 0.93/0.935: find less conservative pass point** ← next
+3. ~~MTP-off + 0.95: keep as alternate throughput/context-safe variant~~ ✅ already validated
+4. Only consider P103 streaming output if MTP-on at 0.95 specifically required
+
+User concurs with Codex's plan. Step 2 (midpoint 0.93/0.935) gives margin
+recommendation for shippable.
+
+### Next: 60K test at #35975 + MTP-on + 0.93 (then 0.935 if 0.93 fails)
+
+
+## ROUND 4 — mem-util sweep at 60K (Codex r2 step 2-3)
+
+### 60K results across mem-util at MTP-on + #35975
+
+| mem-util | Boot resident | KV pool size | 60K MTP-on | Wall | Final GPU free | Notes |
+|---|---|---|---|---|---|---|
+| 0.95 | 22,720 MiB | 5.5 GiB / ~388K tok | ❌ OOM 50/46.5 | n/a | n/a | 22 MiB short of margin |
+| 0.93 | 22,260 MiB | 4.4 GiB / ~340K tok | ✅ HTTP 200 | 623s | ~494 MiB | **Best balanced point.** AL=4.0, recall correct |
+| 0.92 | 21,980 MiB | 3.87 GiB / ~305K tok | ✅ HTTP 200 | 689s | ~580 MiB | More margin but slower + less KV |
+
+**Conclusion: 0.93 is the recommended balanced shipping point.** Recovers
+~30K tokens of KV pool concurrency vs 0.92 while still passing 60K
+Cliff 2 with real end-of-run margin (~494 MiB free). Wall improvement
+likely from cudagraph friendlier allocator pattern at higher headroom.
+
+### Three explicit shippable variants (Codex r2 framing, post-bisect)
+
+**Balanced MTP** — `long-text.yml` updated:
+  - Genesis v7.69 + Codex P103 gate fix + PN30 + PN32 v2 + PN34
+  - vllm#35975 backport sidecar (`patch_inputs_embeds_optional.py`)
+  - mem-util 0.93 (down from 0.95)
+  - MTP K=3 retained
+  - Cliff 2 closed at 60K. KV concurrency at 180K: ~1.4x
+
+**Max-context safety** — `long-text-no-mtp.yml` (TODO):
+  - Same patches except no `--speculative-config`
+  - mem-util 0.95 (full KV pool)
+  - For long single-shot RAG / codebase prompts where slow decode is OK
+  - Cliff 2 ceiling: 60K confirmed; 90K probe with max_tokens=1 pending
+
+**Future upstream win**:
+  - vllm#37429 hybrid Mamba/attention KV cache sizing — could free more
+    residency without trading mem-util
+  - Worth testing on a separate branch when CI signals stabilize
+
+### Remaining tests (in progress)
+
+- 90K + max_tokens=1 at 0.93 — measures prefill ceiling without slow decode
+- max_model_len bump (200K? 220K?) at 0.93 (Codex says do this AFTER mem-util pick — admission capacity test, not safety)
+
+
+## ROUND 5 — 90K probe + admission capacity (final)
+
+### 90K MTP-on + #35975 + 0.93, max_tokens=1 (prefill-only timing)
+
+Result: **HTTP 500 OOM at 112s wall**
+- Trace: chunk_fwd_o `o = torch.empty_like(v)` 50 MiB allocation, 24.5 MiB free
+- Engine died (EngineDeadError)
+
+Conclusion: **Cliff 2 ceiling at 0.93 + #35975 + MTP-on = somewhere
+between 60K (passes) and 90K (fails)**. Likely 70-80K. For shipping
+purposes, document 60K as the safe single-prompt envelope.
+
+### Admission capacity test: max_model_len 200K at 0.93 + #35975
+
+Result: Engine pre-check **REJECTED**
+```
+ValueError: To serve at least one request with the models's max seq len
+(200000), (3.31 GiB KV cache is needed, which is larger than the
+available KV cache memory (2.19 GiB). Based on the available memory,
+the estimated maximum model length is 111456.
+```
+
+Per Codex r2 framing: this is *admission* capacity (would-it-fit), not
+*safety*. The 200K declared at 0.93 + #35975 doesn't fit in available
+KV pool.
+
+**180K stays as the practical max_model_len ceiling at 0.93 + #35975 +
+MTP-on.** Pushing higher requires either:
+- Higher mem-util (back toward 0.95) — but then Cliff 2 returns
+- Drop MTP — frees ~280 MiB residency from draft model
+- vllm#37429 hybrid Mamba/attention KV reduction (untested)
+
+### FINAL recommended recipe (Balanced MTP variant)
+
+```yaml
+# docker-compose.long-text.yml
+--max-model-len 180000
+--gpu-memory-utilization 0.93
+--speculative-config '{"method":"mtp","num_speculative_tokens":3}'
+--max-num-batched-tokens 4128
+--kv-cache-dtype turboquant_3bit_nc
+--enable-chunked-prefill
+--language-model-only
+
+# Genesis env (key Cliff 2 + closures):
+GENESIS_ENABLE_P103=1
+GENESIS_ENABLE_PN30_DS_LAYOUT_SPEC_DECODE=1
+GENESIS_ENABLE_PN32_GDN_CHUNKED_PREFILL=1
+GENESIS_PN32_GDN_CHUNK_SIZE=8192
+GENESIS_PN32_GDN_CHUNK_THRESHOLD=16384
+GENESIS_FLA_FWD_H_MAX_T=16384
+GENESIS_ENABLE_PN34_WORKSPACE_LOCK_RELAX=1
+VLLM_SSM_CONV_STATE_LAYOUT=DS
+
+# Sidecars (mounted):
+patch_tolist_cudagraph.py             # vllm#40807 workaround
+patch_inputs_embeds_optional.py       # vllm#35975 backport (~444 MiB savings)
+```
+
+### Shippable recipe summary
+
+| Variant | Config | 60K | 90K | KV @ 180K | Best for |
+|---|---|---|---|---|---|
+| **Balanced MTP** (this recipe) | MTP-on + 0.93 + #35975 | ✅ | ❌ ~70-80K cliff | 1.4x | Default. Long-context with spec-decode. |
+| Max-context safety | MTP-off + 0.95 + #35975 | ✅ | TBD (longer test) | 1.43x | RAG / single-shot codebase analysis |
+| Future upstream | + vllm#37429 | TBD | TBD | TBD | When PR matures |
+
+
+## ROUND 6 — Max-context variant (MTP-off + 0.95 + 200K + #35975)
+
+### Setup
+
+Same Genesis v7.69 stack + Codex P103 gate fix + #35975 sidecar, but:
+- `--speculative-config` REMOVED (no MTP K=3 draft)
+- `--gpu-memory-utilization 0.95` (recovered)
+- `--max-model-len 200000` (admission test)
+
+Boot resident: **21,004 MiB** (lower than MTP-on variants because draft model
+residency is gone). KV pool: 5.1 GiB / 272,288 tokens. Concurrency at 200K:
+**1.20x**. **200K ADMISSION PASSES** (vs 180K cap on MTP-on path).
+
+### 60K probe — MTP-off + 0.95 + 200K + #35975
+
+Result: **HTTP 200 OK in 537s wall** (vs 504s without #35975 in earlier
+test — slightly slower, possibly due to 200K ctx vs 180K). Recall correct
+("violet otter 48"). No OOM. Probe matches the closure pattern of round 2's
+MTP-off + 180K test, plus the extra 20K ctx admission headroom.
+
+### 90K probe (max_tokens=1) — MTP-off + 0.95 + 200K + #35975
+
+Result: **HTTP 000 at 1500s curl timeout** (25-min budget). Engine kept
+computing (GPU 100% util, no OOM trace), but never delivered response
+within practical bound. Same pattern as round 2 MTP-off + 180K 90K probe
+(also indeterminate within 15 min budget there).
+
+Practical Cliff 2 ceiling on max-context variant: **60K confirmed**.
+90K is past wall-clock-feasibility on this hardware regardless of
+mem-util/MTP/admission tweaks.
+
+### Final shippable matrix
+
+Both variants share the same single-prompt safety envelope (~60K).
+Differ on declared context, decode TPS, and concurrency:
+
+| Variant | max_model_len | mem-util | MTP | KV @ ctx | 60K | 90K |
+|---|---|---|---|---|---|---|
+| Balanced MTP (`long-text.yml`) | **180K** | 0.93 | ✅ K=3 | 1.40x | ✅ 623s | ❌ OOM 112s |
+| Max-context (`long-text-no-mtp.yml`) | **200K** | 0.95 | ❌ off | 1.20x | ✅ 537s | indeterm. |
+
+### Codex r2 verdict (final)
+
+> Cliff 2 at 60K on 24GB, TP1, Qwen3-Next TQ3, DS layout is not caused by
+> the P103 cu_seqlens gate after the semantic fix. With MTP enabled,
+> draft-model residency removes enough activation headroom that late
+> FLA/GDN prefill fails. Disabling MTP lets the same request complete with
+> HTTP 200; the earlier failure mode was headroom, while the verify
+> harness timeout was client-side.
+
+Plus: vllm#35975 backport frees enough headroom that MTP-on can also
+close 60K when paired with mem-util 0.93.
+
+### Open future tests
+
+- vllm#37429 (hybrid Mamba/attention KV cache sizing) — could free more
+  resident memory without trading mem-util
+- P103 streaming output variant — Codex r2 lower-priority recommendation;
+  on-disk implementation deferred
--- a/results/v0.20-migration/v769-test.summary
+++ b/results/v0.20-migration/v769-test.summary
@@ -0,0 +1,177 @@
+# v7.69 Retest — Branch v7.69-cliff2-test
+Date: 2026-05-02 PM
+Genesis pin: 2db18df (v7.69 dev tip)
+vLLM image: nightly-7a1eb8ac2ec4ea69338c51dc7afd4b15010abfa8 (v0.20.1rc1.dev16)
+Hardware: 1× RTX 3090 (24 GB), SM 8.6 Ampere consumer, TP=1
+Compose: long-text.yml @ 180K + 0.95 (back to original target — DS layout active)
+
+## Env bundle (Cliff 2 closure recipe per Sander's v7.69 reply)
+
+GENESIS_ENABLE_P103=1
+GENESIS_ENABLE_PN32_GDN_CHUNKED_PREFILL=1
+GENESIS_PN32_GDN_CHUNK_SIZE=8192
+GENESIS_PN32_GDN_CHUNK_THRESHOLD=16384
+GENESIS_FLA_FWD_H_MAX_T=16384
+GENESIS_ENABLE_PN34_WORKSPACE_LOCK_RELAX=1
+VLLM_SSM_CONV_STATE_LAYOUT=DS
+
+## Sidecars dropped on this branch
+- patch_pn25_genesis_register_fix.py    (replaced by Sander's PN25 v7.68)
+- patch_pn30_dst_shaped_temp_fix.py     (replaced by Sander's PN30 v7.68 + v7.69 part3 marker fix)
+- patch_workspace_lock_disable.py       (replaced by Sander's PN34, opt-in env)
+
+## ✅ F1 — PN30 v7.68 part3 drift-marker fix CONFIRMED WORKING
+v7.68 dev tip (18e65e3) had part3 skip with `upstream_merged` because
+its drift-markers list contained the bare prefix `[Genesis PN30` which
+also matched parts 1+2's own markers. v7.69 (2db18df) tightened part3's
+drift markers to `[Genesis PN30 v7.68 dst-shaped]` — specific enough to
+not collide.
+
+Boot log:
+  [PN30 v1/worker/mamba_utils.py — collect_mamba_copy_meta dst-shaped DS
+   temp (issue #17, v7.68)] applied 1 sub-patches:
+   pN30_collect_mamba_copy_meta_dst_shaped_temp
+  [Genesis] applied: PN30 DS conv state + spec-decode AL>1 (issue #17)
+   — PN30 v7.68 applied: ...
+
+DS layout active (`Setting SSM conv state layout to DS`). Part1 + part2
+ part3 all APPLY clean, no upstream_merged false-positive. Apply_all
+elapsed: clean, 0 failures.
+
+## ✅ F2 — P103 chunk.py self-install hook CONFIRMED FIRES
+v7.68 P103 logged "rebound at 0 caller sites" because `setattr` ran in
+the entrypoint shell process and was lost on `exec vllm serve` (worker
+spawn replaced the process image).
+
+v7.69 P103 ships a text-patched self-install hook appended to
+chunk.py end-of-file:
+
+  try:
+      import os as _genesis_p103_os
+      if _genesis_p103_os.environ.get("GENESIS_ENABLE_P103", ...):
+          from vllm._genesis.wiring.hybrid.patch_103_fla_cliff2_chunked
+              import _genesis_p103_install_at_import as _genesis_p103_install
+          _genesis_p103_install(globals())
+  except Exception:
+      pass
+
+Confirmed firing on TP=1 spawn. Cliff 2 trace path:
+
+  vllm/model_executor/layers/fla/ops/chunk.py:111 forward
+    → vllm/_genesis/wiring/hybrid/patch_103_fla_cliff2_chunked.py:203
+       chunked_fwd  ← P103 wrapper engaged at runtime
+        → return original_fwd(...)  ← BUT falls through to fallback
+
+The wrap IS engaged in workers. F2 is verified at the
+"hook fires correctly" level. Boot log:
+  [Genesis P103 self-install] wrapper installed in chunk.py at
+   module-import time (survives `exec vllm serve` + worker spawn)
+
+## ⚠️ F3 — Cliff 2 NOT closed at 60K despite P103 + PN32 v2
+
+Probes pass / fail summary:
+  [1/7] Long-context needle 10K  ✅ recalled 'silver otter 16'
+  [1/7] Long-context needle 30K  ✅ recalled 'amber iguana 95'
+                                    (was crashing on v7.68; v7.69 fix)
+  [2/7] Tool prefill 25K          ✅ tool_call clean
+  [3/7] IDE-agent one-shot        ✅ 66 completion tokens
+  [4/7] Multi-turn agent          ✅
+  [5/7] LCB-coding shape          ✅
+  [6/7] Reasoning max=8192        ✅
+  [7/7] Long-context needle 60K   ❌ HTTP 500 (Cliff 2 OOM)
+  [7/7] Long-context needle 90K   ❌ HTTP 500
+
+Cliff 2 OOM trace (probe 7 60K rung):
+
+  File "vllm/model_executor/layers/fla/ops/chunk.py", line 111, in forward
+    g, o, A, final_state, w, h, v_new = chunk_gated_delta_rule_fwd(
+  File "vllm/_genesis/wiring/hybrid/patch_103_fla_cliff2_chunked.py",
+       line 203, in chunked_fwd
+    return original_fwd(  ← P103 fallback path
+  File "vllm/model_executor/layers/fla/ops/chunk.py", line 72,
+       in chunk_gated_delta_rule_fwd
+    o = chunk_fwd_o(
+  File "vllm/model_executor/layers/fla/ops/chunk_o.py", line 161,
+       in chunk_fwd_o
+    o = torch.empty_like(v)
+  torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 50.00 MiB.
+  GPU 0 has a total capacity of 23.56 GiB of which 24.50 MiB is free.
+
+### Why P103's chunked path didn't engage
+
+P103's gate is `T <= MAX_T or cu_seqlens != None`. At 60K prompt
+serving:
+  - T = 60000 (well above MAX_T=16384)
+  - But cu_seqlens != None (vLLM serving ALWAYS passes cu_seqlens)
+→ P103 falls through to original_fwd
+
+### Why PN32 v2 didn't engage
+
+PN32 v2's docstring says "Multi-seq bypasses to original" — i.e. it
+only chunks when there's exactly one sequence. With MTP K=3
+spec-decode, the dispatcher reports `uniform_query_len=4` per
+P66 cudagraph filter log. Whether PN32's gate sees this as
+"multi-seq" wasn't traceable at runtime, but the trace doesn't
+show PN32's chunked path.
+
+### Net: F3 closure works only when cu_seqlens=None AND single-seq
+
+Both P103 and PN32 v2 have gates that require cu_seqlens=None
+and/or single-sequence to engage. vLLM serving always passes
+cu_seqlens (it's the per-sequence boundary marker that enables
+batched serving). So neither chunking path fires under real
+serving, only synthetic test scenarios.
+
+## Other engine state observations on v7.69
+
+Engine survived probe 7 OOM gracefully (HTTP 500 + EngineDeadError
+exception → API server shutdown). Compare to v7.66 where Cliff 2 OOM
+killed the EngineCore process abruptly. Suggests vLLM's
+`async_llm.output_handler` exception handling is cleaner on v0.20+.
+Engine still came back up clean on next request batch (could re-test
+small probes after large probe failed).
+
+## Cross-rig regressions: NONE
+TPS on smoke benchmark equivalent to v7.66 + sidecars (same engine
+path under cu_seqlens != None, no overhead from PN32 v2 in fallback
+mode).
+
+## Recommendation back to Sander
+
+Two options for v7.70:
+
+(a) Ship F3 fix that handles cu_seqlens != None — slice metadata
+    along with tensors per chunk; thread initial_state correctly
+    across chunk boundaries even when cu_seqlens describes a
+    multi-sequence batch. Per Sander's own caveat in v7.69 reply:
+    "Multi-seq bypasses to original (chunking across cu_seqlens
+    boundaries needs inner state-cache surgery not exposed at this
+    layer)" — this is the surgery that needs to happen.
+
+(b) Keep current "synthetic-only" gate but document explicitly
+    that P103 + PN32 only close Cliff 2 in benchmarks where
+    cu_seqlens=None is passed manually. Real serving stays
+    behind the cliff. Operators on TP=1 + 24GB should plan for
+    50-60K single-prompt ceiling and route longer prompts to
+    TP=2 (`dual.yml`) or llama.cpp.
+
+Our preference: (a) — but we recognize this is non-trivial. Happy to
+engage ChatGPT/Codex CLI on the cu_seqlens-aware chunking design and
+share back as we did for PN30 dst-shaped + PN25 import-time
+registration.
+
+## What stayed working
+- All 4 of v7.68's other accepted-and-folded patches (PN25 v7.68
+  TP=1 register, PN34 workspace_lock, PN30 v7.68 part1+part2, P101)
+  remain solid.
+- Tool calls clean across IDE-agent shapes (probes 3 + 4).
+- 30K single-prompt now safe (was unsafe on v7.68 due to PN30
+  being broken; v7.69 closes that regression).
+
+## Net: v7.69 is a clear improvement over both v7.66 and v7.68 dev tip
+
+If we accept that Cliff 2 is fundamentally still architectural for
+single-24GB-GPU long-prompt and route around it, v7.69 is our
+best stable single-card config to date. Recommend Sander tag v7.69
+stable and we move to master once the 30K-now-safe data is
+cross-confirmed on Sander's planned A5000-rig.
--- a/scripts/setup.sh
+++ b/scripts/setup.sh
@@ -106,23 +106,37 @@ echo "Model dir:    ${MODEL_DIR}"
 # vllm/_genesis package + per-patch env opts). Newer composes mount the package;
 # the legacy compose still references the v7.13 shim.
 # Pin Genesis to the exact commit our published numbers were measured against.
-# Currently pointing at v7.66 dev tip (commit fc89395, 2026-05-02 AM). Bumped
-# from v7.64 (64dd18b) for the v7.65 patch set:
-#   - P38B / P15B — close the Cliff 1 mech B cascade (issues #14 + #15) via
-#     compile-safe in-source hook + FA varlen workspace clamp.
-#   - PN25 — Inductor-safe silu_and_mul opaque op (replaces our local
-#     patch_pn12_compile_safe_custom_op.py — now removed).
-#   - PN26b — Genesis-original sparse-V Triton kernel for SM86 (Ampere
-#     consumer). First sparse-V kernel in any public tree for SM86. Default
-#     ON in v0.20+ composes (BLOCK_KV=8 num_warps=4 threshold=0.01 per
-#     Sandermage's 27B-specific tuning).
-#   - PN28 — merge_attn_states NaN guard backport (vllm#39148).
-#   - Cliff 8 hardening (partial_apply_warnings counter in boot summary).
-# Pinned to dev SHA fc89395 because v7.66 is feature-complete on dev but not
-# yet tagged; SHA pin is immutable.
-# Bumping GENESIS_PIN requires re-running verify-full.sh against your composes
+# Currently pointing at v7.69 dev tip (commit 2db18df, 2026-05-02 PM). Bumped
+# from v7.66 (fc89395) for the v7.69 patch set, which addresses the 3
+# regressions the v7.68 cross-rig retest found ([club-3090#19] and our
+# v7.68-cliff2-test branch summary):
+#   - F1 (PN30 part3 drift-marker bug) — fixed via specific marker
+#     `[Genesis PN30 v7.68 dst-shaped]` so part3 idempotency check no longer
+#     collides with part1+2 markers in the same file.
+#   - F2 (P103 setattr lost on `exec vllm serve`) — fixed via self-install
+#     hook text-patched into chunk.py end-of-file. Survives any startup
+#     mechanism (workers, fork, spawn, exec). The "rebound at 0 caller sites"
+#     log message in v7.68 was misleading — internal callers DID get the
+#     setattr in the entrypoint shell process, but `exec` replaced the image
+#     and lost it. v7.69 hook fires every time chunk.py imports.
+#   - F3 (PN32 v1 chunked at wrong level) — rewritten as PN32 v2 to patch
+#     `_forward_core` directly + thread initial_state via prior chunk's
+#     last_recurrent_state. Composes with P103: v2 chunks the OUTER FLA
+#     call, P103 chunks INSIDE the FLA inner h tensor.
+# Recommended Cliff 2 closure env bundle for single-24GB-GPU:
+#   GENESIS_ENABLE_P103=1                          (close inner FLA h tensor)
+#   GENESIS_ENABLE_PN32_GDN_CHUNKED_PREFILL=1      (close outer FLA call buffer)
+#   GENESIS_PN32_GDN_CHUNK_SIZE=8192               (default)
+#   GENESIS_PN32_GDN_CHUNK_THRESHOLD=16384         (default)
+#   GENESIS_FLA_FWD_H_MAX_T=16384                  (P103 default)
+# v7.69 also retains v7.68's accept-and-fold of our 3 cross-rig sidecars:
+# PN25 v7.68 (TP=1 worker-spawn registration), PN30 v7.68 (DS conv-state
+# layout dst-shaped temp), PN34 (vllm#39226 runtime workspace_lock).
+# Pinned to dev SHA 2db18df because v7.69 is feature-complete on dev pending
+# our retest validation; if clean, Sander will tag stable.
+# Bumping GENESIS_PIN requires re-running verify-stress.sh against your composes
 # to confirm the new commit works on your config.
-GENESIS_PIN="${GENESIS_PIN:-fc89395}"
+GENESIS_PIN="${GENESIS_PIN:-2db18df}"

 if [[ "${SKIP_GENESIS:-0}" != "1" ]]; then
  if [[ -d "${GENESIS_DIR}/.git" ]]; then
@@ -143,38 +157,11 @@ if [[ "${SKIP_GENESIS:-0}" != "1" ]]; then
  fi
  echo "[genesis] Pinned to ${GENESIS_PIN} ($(cd "${GENESIS_DIR}" && git rev-parse --short HEAD))"

-  # PN25 worker-spawn registration fix — local backport.
-  #
-  # Sandermage shipped his own fix in d92bcb3 (hasattr global-registry guard
-  # in `_register_op_once`), but cross-rig validation on our TP=1 single-card
-  # showed it doesn't work — `torch.ops.genesis.silu_and_mul_pooled` doesn't
-  # exist in spawned workers on TP=1 (whereas it does on his TP=2 PROD).
-  # Reported back as comment on Sandermage/genesis-vllm-patches#16.
-  #
-  # Our local v3 patch takes a different approach: register at activation.py
-  # import time as a module-level cached global, BEFORE any dynamo trace runs.
-  # Survives worker spawn correctly on TP=1.
-  #
-  # Idempotent. Safe to re-run.
-  if [[ -f "${ROOT_DIR}/models/qwen3.6-27b/vllm/patches/patch_pn25_genesis_register_fix.py" ]]; then
-    (cd "${ROOT_DIR}" && python3 models/qwen3.6-27b/vllm/patches/patch_pn25_genesis_register_fix.py) || {
-      echo "[genesis] WARN: PN25 register fix did not apply cleanly. PN25 may not work in workers." >&2
-    }
-  fi
-
-  # PN30 DS conv-state layout fix — local correction for Genesis issue #17.
-  #
-  # Sander's PN30 avoided vLLM's DS+spec-decode NotImplementedError by
-  # compacting state[src_block, :, offset:] and raw-memcpying it into the
-  # destination block. That corrupts DS row strides. Our sidecar patches PN30
-  # so collect_mamba_copy_meta builds a full destination-shaped temp block,
-  # copies the source tail into the dst prefix, then reuses PN30's temp-list
-  # lifetime handling.
-  if [[ -f "${ROOT_DIR}/models/qwen3.6-27b/vllm/patches/patch_pn30_dst_shaped_temp_fix.py" ]]; then
-    (cd "${ROOT_DIR}" && python3 models/qwen3.6-27b/vllm/patches/patch_pn30_dst_shaped_temp_fix.py) || {
-      echo "[genesis] WARN: PN30 dst-shaped temp fix did not apply cleanly. Keep PN30 disabled or use SD layout." >&2
-    }
-  fi
+  # v7.69 ships PN25 + PN30 + PN34 directly (Sander's accept-and-fold of our
+  # cross-rig sidecars). Local patch_pn25_genesis_register_fix.py +
+  # patch_pn30_dst_shaped_temp_fix.py + patch_workspace_lock_disable.py are
+  # now redundant. Sidecar Python files retained in vllm/patches/ for
+  # rollback if any v7.69 patch regresses on your config.
 else
  echo "[genesis] SKIP_GENESIS=1 — not cloning."
 fi