Six rounds of cross-rig bisect with Codex/ChatGPT diagnosis closes
Cliff 2 at 60K on TP=1 + 24GB. Two shippable variants:
Balanced MTP (long-text.yml updated):
- Genesis v7.69 dev tip (commit 2db18df)
- Codex r1 P103 gate fix applied to nested Genesis tree (cu_seqlens=[0,T]
treated as dense single-seq, not multi-seq varlen). Sent to Sander as
v7.70 proposal — diff in /tmp/cliff2_v770_cu_seqlens_response.md.
- vllm#35975 backport sidecar (patch_inputs_embeds_optional.py)
- mem-util 0.93 (down from 0.95)
- max_model_len 180000 (admission ceiling at this mem-util)
- MTP K=3 retained
- 60K probe: HTTP 200 in 623s, recall correct, MTP AL=4.00
Max-context safety (long-text-no-mtp.yml — NEW):
- Same patches, but MTP off + mem-util 0.95
- max_model_len 200000 (admission unlocked by removing MTP residency)
- 60K probe: HTTP 200 in 537s, recall correct
- 90K probe: indeterminate within 25-min curl budget
- For long single-shot RAG / codebase analysis
Diagnostic chain:
1. Codex r1 identified P103 gate as too broad (cu_seqlens != None
bypasses chunking even for single-seq [0,T] case). Applied fix.
2. T=4128 distribution showed chunked path never engages on real
serving (vLLM's outer chunked-prefill caps T well below MAX_T).
3. Codex r2: real Cliff 2 source is residency, not gate logic.
4. PR #35975 backport (skip inputs_embeds for text-only) frees
444 MiB at boot — necessary but not sufficient at 0.95 mem-util.
5. mem-util sweep at 0.92/0.93 with MTP+#35975 closed Cliff 2 at 60K.
6. MTP-off + 0.95 + 200K admission validated max-context variant.
Codex's P103 gate fix is semantically correct and worth shipping in
Genesis v7.70 even though it's not what closes 60K Cliff 2 on this
config (the FLA call sees T=4128 already, well below MAX_T).
Full diagnostic trail: results/v0.20-migration/v769-codex-r1-test.summary
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
f6613c869a