llama-cpp: replace orphan llama-cpp:local with upstream pinned image (#170)

v0.8.3 shipped composes (llamacpp/default, llamacpp/mtp, llamacpp/mtp-vision) all
reference `image: llama-cpp:local`, a custom image that exists ONLY on the
maintainer's rig. There is no Dockerfile, no build script, and no setup.sh hook
to produce it for users. Anyone running `bash scripts/switch.sh llamacpp/mtp`
on a fresh clone hits "image llama-cpp:local not found" and dies at boot.

The custom image was a v0.8.3-dev artifact from when MTP PR #22673 was bleeding
edge. The official upstream `ghcr.io/ggml-org/llama.cpp:server-cuda` now has it
merged (build b9246 = commit 871b0b70f, 2026-05-20) — pinning to b9246 reproduces
the v0.8.3 numbers (50.25 narr / 58.04 code on single 3090, vs shipped 51.28/59.72).

Surfaced by @zemaphore in discussion #170. README.md was also lying: claimed
"both use the official ghcr.io image, no custom build needed" while composes
referenced llama-cpp:local.

Override the pin via `LLAMACPP_IMAGE=ghcr.io/.../server-cuda-bXXXX` env if you
want to follow upstream master.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
noonghunna
2026-05-20 12:37:31 +00:00
parent db9c5e1bb9
commit c3e7c7ed80
5 changed files with 6 additions and 6 deletions

View File

@@ -22,7 +22,7 @@ For full pros/cons + general llama.cpp tuning, see [`/docs/engines/LLAMA_CPP.md`
## Docker compose (recommended)
Two compose files in [`compose/`](compose/) — both use the official `ghcr.io/ggml-org/llama.cpp:server-cuda` image, no custom build needed.
Three compose variants in [`compose/single/`](compose/single/) — all use the official `ghcr.io/ggml-org/llama.cpp:server-cuda` image (pinned at build `b9246`, contains MTP PR #22673), **no custom build needed**. Override the pinned tag via `LLAMACPP_IMAGE` env if you want a newer build (do verify MTP behavior hasn't drifted before relying on changed numbers).
### `docker-compose.yml` — max context, single slot, vision

View File

@@ -111,7 +111,7 @@
services:
llama-cpp-qwen36-27b:
image: llama-cpp:local
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9246}
container_name: "${ESTATE_CONTAINER:-llama-cpp-qwen36-27b}"
restart: unless-stopped
ports:

View File

@@ -68,7 +68,7 @@
services:
llama-cpp-qwen36-27b-mtp-vision:
image: llama-cpp:local
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9246}
container_name: "${ESTATE_CONTAINER:-llama-cpp-qwen36-27b}"
restart: unless-stopped
ports:

View File

@@ -72,7 +72,7 @@
services:
llama-cpp-qwen36-27b-mtp:
image: llama-cpp:local
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9246}
container_name: "${ESTATE_CONTAINER:-llama-cpp-qwen36-27b}"
restart: unless-stopped
ports:

View File

@@ -5,7 +5,7 @@ type: llama.cpp
stability: stable
install:
method: docker_image
spec: llama-cpp:local
spec: ghcr.io/ggml-org/llama.cpp:server-cuda-b9246
min_sm: 6.0
supported_model_families:
- qwen3-next-hybrid
@@ -22,5 +22,5 @@ supported_weight_formats:
required_overlays: []
vendored_overlays: []
required_genesis: false
notes: "Cliff-immune single-card fallback for Qwen GGUF serving. Built from ggerganov/llama.cpp @ d14ce3dab (MTP PR #22673)."
notes: "Cliff-immune single-card fallback for Qwen GGUF serving. Upstream `ghcr.io/ggml-org/llama.cpp:server-cuda` pinned at build b9246 (commit 871b0b70f, 2026-05-20) — MTP PR #22673 merged + draft-eagle3 + draft-mtp spec types available natively. Override via env: `LLAMACPP_IMAGE=ghcr.io/.../server-cuda-bXXXX`."