b875624f2d
PR #23 + PRa62ad78+ PR9af1a52shipped PN25 v3 + PN30 dst-shaped temp fix on long-text only. This commit extends the same patch stack to the remaining 3 TQ3 composes (long-vision, bounded-thinking, dual-turbo) and validates each independently. What changed ------------ - **long-vision.yml**: 198K + 0.98 → **145K + 0.95** + DS layout + PN25 v3 + PN30. Vision tower residence forces deeper backoff than long-text (engine pre-check returned `estimated max 148608` at 175K + 0.95, settled at 145K with safety margin). - **bounded-thinking.yml**: 214K + 0.985 → **180K + 0.95** + DS layout + PN25 v3 + PN30. Parity with long-text — same patch stack, same backoff, structured-CoT grammar still works on top. - **dual-turbo.yml**: 262K context preserved + DS layout + PN25 v3 + PN30. TP=2 splits state across both cards, fits the patch stack at 0.85 mem-util cleanly. Validation per compose (verify-stress.sh, 7 probes) --------------------------------------------------- | Variant | Pass count | Failure | |--------------------|------------|----------------------| | long-text | 6 / 7 | Cliff 2 architectural | | long-vision | 6 / 7 | Cliff 2 architectural | | bounded-thinking | 6 / 7 | Cliff 2 architectural | | dual-turbo (TP=2) | 6 / 7 | Cliff 2 architectural | All non-architectural probes pass — IDE-agent one-shot, multi-turn agent, LCB-coding, reasoning-heavy, 25K tool RETURN, small-rung longctx. Cliff 2 (60K+ single prompt DeltaNet GDN forward state OOM) fails on every variant including TP=2 because GDN state is per-rank not split — that's fundamental, not addressable on this config class. Per-config summaries written to: - results/v0.20-migration/long-vision-pn30.summary - results/v0.20-migration/bounded-thinking-pn30.summary - results/v0.20-migration/dual-turbo-pn30.summary Docs partial update (SINGLE_CARD.md): updated TL;DR table with new ctx ceilings and removed the Cliff 1 mech B "limitation to know" since both mechanisms (PN12 eager + PN25 v3 compile) now close it. Kept Cliff 2 as the one remaining limitation. More doc updates in follow-up commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
29 lines
1.2 KiB
Plaintext
29 lines
1.2 KiB
Plaintext
variant: dual-turbo (TP=2)
|
|
date: 2026-05-02
|
|
context: 262144
|
|
gpu_mem_util: 0.85
|
|
kv_dtype: turboquant_3bit_nc
|
|
tp: 2
|
|
genesis_pin: 753344b (v7.65 dev tip)
|
|
patches: PN25 v3 (local) + PN30 dst-shaped temp fix (local) + DS layout enabled
|
|
verify_stress_results: |
|
|
[1/7] longctx small (10K + 30K): PASS
|
|
[2/7] 25K tool RETURN: PASS
|
|
[3/7] IDE-agent one-shot: PASS (66 tokens, finish=stop)
|
|
[4/7] multi-turn agent: PASS
|
|
[5/7] LCB-coding: PASS
|
|
[6/7] reasoning 8192: PASS (8192 tokens, finish=length)
|
|
[7/7] longctx large (60K+): FAIL (Cliff 2 architectural — DeltaNet GDN forward state OOM, even on TP=2 single-prompt)
|
|
score: 6/7
|
|
notes: |
|
|
TP=2 splits state but doesn't avoid Cliff 2 on single-prompt 60K+ since
|
|
the GDN state buffer is per-rank and grows with seq_len, not split across
|
|
ranks. Cliff 2 is not addressable on this config class regardless of
|
|
TP factor. Rule for users: dual-turbo for 4-stream concurrent serving at
|
|
≤50K each, OR llamacpp/default 262K for single-prompt big inputs.
|
|
PN30 dst-shaped temp fix passes cleanly on TP=2 — multi-turn agent +
|
|
LCB-coding shapes both work that previously crashed with Sander's
|
|
upstream PN30 a9977d8.
|
|
kv_pool_tokens: 1523232
|
|
max_concurrency: 4.67
|