club-3090

alex/club-3090

Fork 0

Commit Graph

Author	SHA1	Message	Date
noonghunna	e6660afe49	rebench: add fail-fast verify-full preflight + fix runtime skip set (#306 ) rebench-full: run verify-full as step 0 before the multi-hour matrix. If the endpoint isn't functional (not serving, 404 on a name mismatch, dead tool-calls, warmup gibberish), abort in ~2 min instead of grinding through bench → soak on a broken server. Skipped cleanly by --skip verify-full or --resume (run_step returns 0 on skip → no abort); only a real run-and-fail aborts. set -e is suppressed inside the `if !` condition, matching how run_step already behaves under the callers' `\|\| true`. rebench-runtime: fix the skip set broken by #303. It hard-coded BASE_SKIP="quality-full,aider-polyglot" — but #303 removed the aider step and added quality-thinking (think-ON). So the runtime tier was silently RUNNING the ~60-90 min think-ON 8-pack it's designed to skip. Corrected to "quality-full,quality-thinking"; verify-full is intentionally NOT skipped so the runtime tier inherits the fail-fast preflight (a runtime knob can break serving). Header/timing refreshed. Co-authored-by: noonghunna <10742901+noonghunna@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 21:00:16 +05:00
noonghunna	f1bb069eb5	feat(models): add ik_llama Qwen3.6-27B IQ4_KS composes — text 262K + vision 160K (#180 ) ik_llama.cpp serving path for Qwen3.6-27B (ubergarm MTP-IQ4_KS GGUF), validated on a single RTX 3090 (task #402): - compose/single/iq4ks-mtp.yml — text, 262K ctx (full native n_ctx_train), q4_0 KV, MTP n=2, froggeric v19. ~59 narr / 65 code TPS; verify-stress 7/7 to 90K; soak-continuous PASS. MTP n=2 vs n=3 A/B: n=2 wins (narrative +3.7%, code a wash within noise). - compose/single/iq4ks-mtp-vision.yml — vision, 160K ctx + full-res (4M-px / 2048²) images at -b 1024; bench + verify-stress + soak all PASS. 160K chosen for headroom (172K served images but OOM-crashed under heavy prefill). - patches/froggeric-chat-template/ — v19 template + PROVENANCE. - scripts/rebench-runtime.sh — bench+verify-stress+soak "runtime tier" suite (rebench-full minus the quality legs; for config-only changes). Engine quirk documented in-header: this ik_llama build forces n_ubatch = n_batch, so only -b moves the compute buffer (-ub is a no-op here — distinct from mainline llama.cpp). Eval track; not yet wired into COMPOSE_REGISTRY/switch.sh. Co-authored-by: noonghunna <10742901+noonghunna@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-22 04:45:44 +05:00

Author

SHA1

Message

Date

noonghunna

e6660afe49

rebench: add fail-fast verify-full preflight + fix runtime skip set (#306 )

rebench-full: run verify-full as step 0 before the multi-hour matrix. If the
endpoint isn't functional (not serving, 404 on a name mismatch, dead
tool-calls, warmup gibberish), abort in ~2 min instead of grinding through
bench → soak on a broken server. Skipped cleanly by --skip verify-full or
--resume (run_step returns 0 on skip → no abort); only a real run-and-fail
aborts. set -e is suppressed inside the `if !` condition, matching how
run_step already behaves under the callers' `|| true`.

rebench-runtime: fix the skip set broken by #303. It hard-coded
BASE_SKIP="quality-full,aider-polyglot" — but #303 removed the aider step and
added quality-thinking (think-ON). So the runtime tier was silently RUNNING
the ~60-90 min think-ON 8-pack it's designed to skip. Corrected to
"quality-full,quality-thinking"; verify-full is intentionally NOT skipped so
the runtime tier inherits the fail-fast preflight (a runtime knob can break
serving). Header/timing refreshed.

Co-authored-by: noonghunna <10742901+noonghunna@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-03 21:00:16 +05:00

noonghunna

f1bb069eb5

feat(models): add ik_llama Qwen3.6-27B IQ4_KS composes — text 262K + vision 160K (#180 )

ik_llama.cpp serving path for Qwen3.6-27B (ubergarm MTP-IQ4_KS GGUF), validated
on a single RTX 3090 (task #402):

- compose/single/iq4ks-mtp.yml — text, 262K ctx (full native n_ctx_train),
  q4_0 KV, MTP n=2, froggeric v19. ~59 narr / 65 code TPS; verify-stress 7/7
  to 90K; soak-continuous PASS. MTP n=2 vs n=3 A/B: n=2 wins (narrative +3.7%,
  code a wash within noise).
- compose/single/iq4ks-mtp-vision.yml — vision, 160K ctx + full-res (4M-px /
  2048²) images at -b 1024; bench + verify-stress + soak all PASS. 160K chosen
  for headroom (172K served images but OOM-crashed under heavy prefill).
- patches/froggeric-chat-template/ — v19 template + PROVENANCE.
- scripts/rebench-runtime.sh — bench+verify-stress+soak "runtime tier" suite
  (rebench-full minus the quality legs; for config-only changes).

Engine quirk documented in-header: this ik_llama build forces n_ubatch =
n_batch, so only -b moves the compute buffer (-ub is a no-op here — distinct
from mainline llama.cpp). Eval track; not yet wired into COMPOSE_REGISTRY/switch.sh.

Co-authored-by: noonghunna <10742901+noonghunna@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-22 04:45:44 +05:00

2 Commits