noonghunnaandClaude Opus 4.8
fabf9ce1ea
feat(qwen): flip multi-max to fp8/e4m3 KV + weights-conditional Ampere compat
Follow-up to #594 (which flipped dual-max int8-PTH -> fp8). Mirrors it on the
4-card tier and makes the registry/compat model reflect reality:
- multi4/fp8/mtp.yml: KV default int8_per_token_head -> fp8 (+ header, mirrors
dual-max: fp8->e4m3, FlashInfer, scale=1.0, not-e5m2)
- registry kv_format: dual-max + multi-max int8_per_token_head -> fp8_e4m3
(jesse's #594 changed only the compose, leaving the registry stale) + de-stale
the status_notes (fp8 KV, 8-pack 109 ties 107, soak p50 125.5)
- compat.py C5: weights-conditional rule. fp8_e4m3 KV is allowed on Ampere
(sm>=8.6) ONLY for fp8-weights checkpoints (FlashInfer path). Non-fp8 weights
(Gemma W4A16) still route to Triton (needs SM89+) and stay correctly rejected.
No rtx-3090 profile change -> Gemma protection intact via the profile.
- test-launch-compat: assert BOTH directions (Qwen fp8-weights passes, Gemma
gemma-mtp-tp1 still fails C5) so the rule + registry stay in sync.
- DTYPE_MATRIX: cite vLLM's fp8-KV blog (scale=1.0 = near-lossless lower bound;
Qwen3.5-27B matches baseline AUC @1M) + document the weights-conditional rule.
Validated: dual-max + multi-max validate-variant PASS with fp8_e4m3 on 2x/4x
3090; gemma-mtp-tp1 still rejected. 11 compat/registry guards green.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
2026-07-06 14:55:01 +00:00
..
2026-06-06 00:46:13 +00:00
2026-06-14 17:04:47 +05:00
2026-07-04 14:17:55 +00:00
2026-07-05 06:16:59 +00:00
2026-07-03 03:50:14 +00:00
2026-07-02 10:03:44 +00:00
2026-07-06 09:51:11 +00:00
2026-05-17 04:19:53 +00:00
2026-05-16 05:13:00 +00:00
2026-05-18 16:43:28 +00:00
2026-05-18 16:43:28 +00:00
2026-07-06 14:55:01 +00:00
2026-07-06 14:55:01 +00:00
2026-05-18 16:43:28 +00:00
2026-07-05 17:12:10 +00:00
2026-06-11 05:31:36 +00:00
2026-05-17 01:28:22 +00:00
2026-05-17 00:12:26 +00:00
2026-07-05 06:16:59 +00:00
2026-07-04 10:35:00 +00:00
2026-05-18 16:43:28 +00:00
2026-07-05 03:23:49 +00:00
2026-05-18 19:29:43 +00:00
2026-07-05 23:20:06 +00:00
2026-05-18 16:43:28 +00:00
2026-07-04 16:22:48 +00:00
2026-05-18 18:29:58 +00:00
2026-07-02 10:03:44 +00:00
2026-06-18 01:10:01 +05:00
2026-06-18 01:10:01 +05:00
2026-05-18 17:49:00 +00:00
2026-05-17 12:01:25 +00:00
2026-07-03 03:50:14 +00:00