llama-cpp: replace orphan llama-cpp:local with upstream pinned image (#170)
v0.8.3 shipped composes (llamacpp/default, llamacpp/mtp, llamacpp/mtp-vision) all reference `image: llama-cpp:local`, a custom image that exists ONLY on the maintainer's rig. There is no Dockerfile, no build script, and no setup.sh hook to produce it for users. Anyone running `bash scripts/switch.sh llamacpp/mtp` on a fresh clone hits "image llama-cpp:local not found" and dies at boot. The custom image was a v0.8.3-dev artifact from when MTP PR #22673 was bleeding edge. The official upstream `ghcr.io/ggml-org/llama.cpp:server-cuda` now has it merged (build b9246 = commit 871b0b70f, 2026-05-20) — pinning to b9246 reproduces the v0.8.3 numbers (50.25 narr / 58.04 code on single 3090, vs shipped 51.28/59.72). Surfaced by @zemaphore in discussion #170. README.md was also lying: claimed "both use the official ghcr.io image, no custom build needed" while composes referenced llama-cpp:local. Override the pin via `LLAMACPP_IMAGE=ghcr.io/.../server-cuda-bXXXX` env if you want to follow upstream master. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -22,7 +22,7 @@ For full pros/cons + general llama.cpp tuning, see [`/docs/engines/LLAMA_CPP.md`
|
||||
|
||||
## Docker compose (recommended)
|
||||
|
||||
Two compose files in [`compose/`](compose/) — both use the official `ghcr.io/ggml-org/llama.cpp:server-cuda` image, no custom build needed.
|
||||
Three compose variants in [`compose/single/`](compose/single/) — all use the official `ghcr.io/ggml-org/llama.cpp:server-cuda` image (pinned at build `b9246`, contains MTP PR #22673), **no custom build needed**. Override the pinned tag via `LLAMACPP_IMAGE` env if you want a newer build (do verify MTP behavior hasn't drifted before relying on changed numbers).
|
||||
|
||||
### `docker-compose.yml` — max context, single slot, vision
|
||||
|
||||
|
||||
@@ -111,7 +111,7 @@
|
||||
|
||||
services:
|
||||
llama-cpp-qwen36-27b:
|
||||
image: llama-cpp:local
|
||||
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9246}
|
||||
container_name: "${ESTATE_CONTAINER:-llama-cpp-qwen36-27b}"
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
|
||||
@@ -68,7 +68,7 @@
|
||||
|
||||
services:
|
||||
llama-cpp-qwen36-27b-mtp-vision:
|
||||
image: llama-cpp:local
|
||||
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9246}
|
||||
container_name: "${ESTATE_CONTAINER:-llama-cpp-qwen36-27b}"
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
|
||||
@@ -72,7 +72,7 @@
|
||||
|
||||
services:
|
||||
llama-cpp-qwen36-27b-mtp:
|
||||
image: llama-cpp:local
|
||||
image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda-b9246}
|
||||
container_name: "${ESTATE_CONTAINER:-llama-cpp-qwen36-27b}"
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
|
||||
@@ -5,7 +5,7 @@ type: llama.cpp
|
||||
stability: stable
|
||||
install:
|
||||
method: docker_image
|
||||
spec: llama-cpp:local
|
||||
spec: ghcr.io/ggml-org/llama.cpp:server-cuda-b9246
|
||||
min_sm: 6.0
|
||||
supported_model_families:
|
||||
- qwen3-next-hybrid
|
||||
@@ -22,5 +22,5 @@ supported_weight_formats:
|
||||
required_overlays: []
|
||||
vendored_overlays: []
|
||||
required_genesis: false
|
||||
notes: "Cliff-immune single-card fallback for Qwen GGUF serving. Built from ggerganov/llama.cpp @ d14ce3dab (MTP PR #22673)."
|
||||
notes: "Cliff-immune single-card fallback for Qwen GGUF serving. Upstream `ghcr.io/ggml-org/llama.cpp:server-cuda` pinned at build b9246 (commit 871b0b70f, 2026-05-20) — MTP PR #22673 merged + draft-eagle3 + draft-mtp spec types available natively. Override via env: `LLAMACPP_IMAGE=ghcr.io/.../server-cuda-bXXXX`."
|
||||
|
||||
|
||||
Reference in New Issue
Block a user