alex/club-3090
1
0
Fork 0
Code Issues Pull Requests Actions 38 Packages Projects Releases Wiki Activity

Bump llama.cpp pin to server-cuda-b9967 across all composes (+4 think-ON) (#680)

Browse Source 
b9246 was 52 days / 721 builds old. Validated on Tess dual (full quality
A/B, 2026-07-11): think-OFF total tie 115=115 (greedy-leg engine
isolation — totals neutral, though 4 scenarios flipped each way),
think-ON 122 vs the 118x3 pin — the lift is thinking-path-specific;
3 previously-100%-reproducible failures now pass (CLI-31 safety refusal,
CLI-20 archive precision, RM-04 math). Boots clean, draft-mtp works.

Also unifies the pin landscape: Deckard's b9570 and the ROLLING tags on
gemma-12b/vibethinker (a #187-policy violation — rolling crash-looped at
b9282) all move to the same validated build. Engine-profile spec now
matches the composes. Non-Tess slugs: boot re-validation is spot-checked
(smoke to follow on PR); bench numbers stay dated to their measured pin;
baselines rows go stale by design (the staleness badge is the feature).


Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm

Co-authored-by: noonghunna <10742901+noonghunna@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
noonghunna
2026-07-12 01:00:29 +05:00
committed by GitHub
parent 0b970ab0a7
commit ee0624dd1b
12 changed files with 13 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/WSL_SETUP.md
View File

@@ -212,7 +212,7 @@ You still do steps **1–3** (WSL + driver/passthrough + `.wslconfig` RAM) and *
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build --config Release -j
```
…or grab a prebuilt CUDA binary. To match the MTP / spec-decode support the Docker image ships, track a recent build — the composes pin `ghcr.io/ggml-org/llama.cpp:server-cuda-b9246` (or newer).
…or grab a prebuilt CUDA binary. To match the MTP / spec-decode support the Docker image ships, track a recent build — the composes pin `ghcr.io/ggml-org/llama.cpp:server-cuda-b9967` (or newer).
2. **Run `llama-server` with the flags the compose uses.** The compose is the source of truth — lift them from [`models/qwen3.6-27b/llama-cpp/compose/single/unsloth-q4km/mtp.yml`](../models/qwen3.6-27b/llama-cpp/compose/single/unsloth-q4km/mtp.yml). The equivalent native invocation:
```bash

2
models/gemma-4-12b/llama-cpp/compose/single/unsloth-q8kxl/base.yml
View File

@@ -41,7 +41,7 @@
services:
llama-cpp-gemma4-12b:
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda}
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9967}
container_name: "${ESTATE_CONTAINER:-llama-cpp-gemma4-12b}"
restart: unless-stopped
ports:

2
models/qwen3.6-27b/llama-cpp/README.md
View File

@@ -22,7 +22,7 @@ For full pros/cons + general llama.cpp tuning, see [`/docs/engines/LLAMA_CPP.md`
## Docker compose (recommended)
Three compose variants in [`compose/single/`](compose/single/) — all use the official `ghcr.io/ggml-org/llama.cpp` image (CUDA), **no custom build needed**, **no club-3090 patches** (unlike our vLLM track). MTP PR #22673 has merged upstream so this image has it natively. The composes are **pinned to build `server-cuda-b9246`** (validated 2026-05-20) — *not* the rolling `:server-cuda` tag, because that tag regressed at `b9282` (broken lib packaging → crash loop, [#187](https://github.com/noonghunna/club-3090/issues/187)). To follow a newer build, override `LLAMACPP_IMAGE=ghcr.io/ggml-org/llama.cpp:server-cuda-bXXXX` (validate it first). Bench numbers were measured on `b9246`; expect ±5% drift on newer builds.
Three compose variants in [`compose/single/`](compose/single/) — all use the official `ghcr.io/ggml-org/llama.cpp` image (CUDA), **no custom build needed**, **no club-3090 patches** (unlike our vLLM track). MTP PR #22673 has merged upstream so this image has it natively. The composes are **pinned to build `server-cuda-b9967`** (validated 2026-07-11 via a full quality A/B on Tess: think-OFF total tie, think-ON +4 — the improvement is thinking-path-specific; earlier pin b9246 validated 2026-05-20) — *not* the rolling `:server-cuda` tag, because that tag regressed at `b9282` (broken lib packaging → crash loop, [#187](https://github.com/noonghunna/club-3090/issues/187)). To follow a newer build, override `LLAMACPP_IMAGE=ghcr.io/ggml-org/llama.cpp:server-cuda-bXXXX` (validate it first). Bench numbers in BENCHMARKS were measured on the pin current at their date; expect ±5% drift across builds.
### `single/unsloth-q4km/mtp.yml` — MTP n=2, 200K ctx, no vision

2
models/qwen3.6-27b/llama-cpp/compose/single/pi-reasoning-q4km/mtp.yml
View File

@@ -87,7 +87,7 @@ services:
# Pinned to b9246 (the validated mainline MTP build; rolling :server-cuda regressed at
# b9282, #187). Bump via env once a newer build is validated:
# LLAMACPP_IMAGE=ghcr.io/ggml-org/llama.cpp:server-cuda-bXXXX
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9246}
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9967}
container_name: "${ESTATE_CONTAINER:-llama-cpp-pi-reasoning}"
restart: unless-stopped
ports:

2
models/qwen3.6-27b/llama-cpp/compose/single/unsloth-q4km/bounded-thinking.yml
View File

@@ -53,7 +53,7 @@ services:
# regressed at b9282 (broken lib packaging: `libllama-common.so.0: cannot
# open shared object file` → crash loop) — see #187. Bump via env when a
# newer build is validated: LLAMACPP_IMAGE=ghcr.io/ggml-org/llama.cpp:server-cuda-bXXXX
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9246}
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9967}
container_name: "${ESTATE_CONTAINER:-llama-cpp-qwen36-27b-bounded-thinking}"
restart: unless-stopped
ports:

2
models/qwen3.6-27b/llama-cpp/compose/single/unsloth-q4km/mtp-vision.yml
View File

@@ -92,7 +92,7 @@ services:
# regressed at b9282 (broken lib packaging: `libllama-common.so.0: cannot
# open shared object file` → crash loop) — see #187. Bump via env when a
# newer build is validated: LLAMACPP_IMAGE=ghcr.io/ggml-org/llama.cpp:server-cuda-bXXXX
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9246}
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9967}
container_name: "${ESTATE_CONTAINER:-llama-cpp-qwen36-27b-vision}"
restart: unless-stopped
ports:

2
models/qwen3.6-27b/llama-cpp/compose/single/unsloth-q4km/mtp.yml
View File

@@ -83,7 +83,7 @@ services:
# regressed at b9282 (broken lib packaging: `libllama-common.so.0: cannot
# open shared object file` → crash loop) — see #187. Bump via env when a
# newer build is validated: LLAMACPP_IMAGE=ghcr.io/ggml-org/llama.cpp:server-cuda-bXXXX
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9246}
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9967}
container_name: "${ESTATE_CONTAINER:-llama-cpp-qwen36-27b}"
restart: unless-stopped
ports:

2
models/qwen3.6-40b-deckard/llama-cpp/compose/dual/piehsoft-q6k/mtp.yml
View File

@@ -80,7 +80,7 @@
services:
llama-cpp-deckard-40b:
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9570}
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9967}
container_name: "${ESTATE_CONTAINER:-llama-cpp-deckard-40b}"
restart: unless-stopped
ports:

2
models/tess-4-27b/llama-cpp/compose/dual/migtissera-q4km/mtp.yml
View File

@@ -93,7 +93,7 @@
services:
llama-cpp-tess-4-27b:
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9246}
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9967}
container_name: "${ESTATE_CONTAINER:-llama-cpp-tess-4-27b}"
restart: unless-stopped
ports:

2
models/vibethinker-3b/llama-cpp/compose/single/prithivmlmods-q8/q8kv.yml
View File

@@ -65,7 +65,7 @@
# LLAMACPP_IMAGE engine image (default: rolling server-cuda; launcher injects engine pin)
services:
llama-cpp-vibethinker-3b:
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda}
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9967}
container_name: "${ESTATE_CONTAINER:-llama-cpp-vibethinker-3b}"
restart: ${CLUB3090_RESTART:-unless-stopped}
ports:

4
scripts/lib/profiles/engines/llama-cpp-mainline.yml
View File

@@ -5,7 +5,7 @@ type: llama.cpp
stability: stable
install:
method: docker_image
spec: ghcr.io/ggml-org/llama.cpp:server-cuda
spec: ghcr.io/ggml-org/llama.cpp:server-cuda-b9967
min_sm: 6.0
supported_model_families:
- qwen3-next-hybrid
@@ -27,5 +27,5 @@ supported_weight_formats:
required_overlays: []
vendored_overlays: []
required_genesis: false
notes: "Cliff-immune single-card fallback for Qwen GGUF serving. Uses upstream `ghcr.io/ggml-org/llama.cpp`, pinned to build `server-cuda-b9246` (NOT the rolling `:server-cuda` tag — it regressed at b9282 with a broken-lib crash loop, #187). MTP PR #22673 merged upstream + draft-eagle3 + draft-mtp spec types available natively. Bump via env once a newer build is validated: `LLAMACPP_IMAGE=ghcr.io/.../server-cuda-bXXXX`. No club-3090 patches on llama.cpp; we follow upstream as-is."
notes: "Cliff-immune single-card fallback for Qwen GGUF serving. Uses upstream `ghcr.io/ggml-org/llama.cpp`, pinned to build `server-cuda-b9967` (bumped from b9246 2026-07-11 after a full quality A/B on Tess dual: think-OFF total tie 115=115, think-ON 122 vs the 118x3 pin — the lift is thinking-path-specific; boots clean, draft-mtp works. NOT the rolling `:server-cuda` tag — it regressed at b9282 with a broken-lib crash loop, #187; always pin validated builds). MTP PR #22673 merged upstream + draft-eagle3 + draft-mtp spec types available natively. Bump via env once a newer build is validated: `LLAMACPP_IMAGE=ghcr.io/.../server-cuda-bXXXX`. No club-3090 patches on llama.cpp; we follow upstream as-is."

2
scripts/tests/test-preflight-compose-deps.sh
View File

@@ -80,7 +80,7 @@ llama_compose="${TMP_DIR}/llama.yml"
cat > "$llama_compose" <<'YAML'
services:
llama:
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9246}
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9967}
command: >-
-m /models/${GGUF_FILE:-qwen3.6-27b-gguf/unsloth-mtp-q4km/Qwen3.6-27B-Q4_K_M.gguf}
YAML