alex/club-3090
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: 4 stale refs missed in 2026-05-10 reorg push (caught by RobH589 #116)

Browse Source 
After the GGUF dir move (/mnt/models/gguf/qwen3.6-27b/ → /mnt/models/
huggingface/qwen3.6-27b-gguf/), four refs were not updated and led to
a path-mismatch loop reported in club-3090#116:

- scripts/preflight.sh `hf download` hint pointed at qwen3.6-27b/, but
  the compose default expects qwen3.6-27b-gguf/. Same for the mv hint
  for mmproj relocation, and the in-container mmproj default at line 329.
- models/qwen3.6-27b/llama-cpp/README.md example command still said
  `MODEL_DIR=/mnt/models/gguf` (now /mnt/models/huggingface).
- models/qwen3.6-27b/llama-cpp/compose/single/{docker-compose,concurrent}.yml
  header comment said "Q5_K_XL" but the actual default has been Q3_K_XL
  for a while (this one predates the reorg — just stale doc).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
noonghunna
2026-05-10 17:06:20 +00:00
parent 00366a58d7
commit cf7f1959fd
4 changed files with 7 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
models/qwen3.6-27b/llama-cpp/README.md
View File

@@ -30,7 +30,7 @@ Showcase: full **262K context** on one 3090 with vision + q4_0 KV.
```bash
cd models/qwen3.6-27b/llama-cpp/compose
MODEL_DIR=/mnt/models/gguf docker compose up -d
MODEL_DIR=/mnt/models/huggingface docker compose up -d
```
Memory budget: 14.5 GB (Q3_K_XL) + 4.5 GB KV @ 262K + 0.8 GB mmproj ≈ 20 GB / 24 GB.

2
models/qwen3.6-27b/llama-cpp/compose/single/concurrent.yml
View File

@@ -1,6 +1,6 @@
# ===========================================================================
# Profile (at-a-glance):
# Model: Qwen3.6-27B (Unsloth Q5_K_XL GGUF)
# Model: Qwen3.6-27B (Unsloth Q3_K_XL GGUF)
# Engine: llama.cpp (NOT vLLM)
# Topology: Single 3090 (TP=1)
# Drafter: none

2
models/qwen3.6-27b/llama-cpp/compose/single/docker-compose.yml
View File

@@ -1,6 +1,6 @@
# ===========================================================================
# Profile (at-a-glance):
# Model: Qwen3.6-27B (Unsloth Q5_K_XL GGUF)
# Model: Qwen3.6-27B (Unsloth Q3_K_XL GGUF)
# Engine: llama.cpp (NOT vLLM — different engine, different memory model)
# Topology: Single 3090 (TP=1)
# Drafter: none (vanilla llama.cpp; MTP via PR #22673 not adopted yet)

8
scripts/preflight.sh
View File

@@ -325,8 +325,8 @@ preflight_compose_deps() {
mmproj_in_container="${mmproj_in_container//\$\{MMPROJ_FILE:-/}"
mmproj_in_container="${mmproj_in_container%\}}"
[[ -z "$gguf_in_container" ]] && gguf_in_container="qwen3.6-27b/unsloth-q3kxl/Qwen3.6-27B-UD-Q3_K_XL.gguf"
[[ -z "$mmproj_in_container" ]] && mmproj_in_container="qwen3.6-27b/mmproj-F16.gguf"
[[ -z "$gguf_in_container" ]] && gguf_in_container="qwen3.6-27b-gguf/unsloth-q3kxl/Qwen3.6-27B-UD-Q3_K_XL.gguf"
[[ -z "$mmproj_in_container" ]] && mmproj_in_container="qwen3.6-27b-gguf/mmproj-F16.gguf"
if [[ -n "${GGUF_FILE:-}" ]]; then gguf_in_container="$GGUF_FILE"; fi
if [[ -n "${MMPROJ_FILE:-}" ]]; then mmproj_in_container="$MMPROJ_FILE"; fi
@@ -375,9 +375,9 @@ preflight_compose_deps() {
if [[ $hint_gguf -eq 1 ]]; then
echo "[preflight] hf download unsloth/Qwen3.6-27B-GGUF \\" >&2
echo "[preflight] Qwen3.6-27B-UD-Q3_K_XL.gguf mmproj-F16.gguf \\" >&2
echo "[preflight] --local-dir ${model_dir}/qwen3.6-27b/unsloth-q3kxl" >&2
echo "[preflight] --local-dir ${model_dir}/qwen3.6-27b-gguf/unsloth-q3kxl" >&2
echo "[preflight] # mmproj lands at unsloth-q3kxl/ — move it up so the default --mmproj path resolves:" >&2
echo "[preflight] # mv ${model_dir}/qwen3.6-27b/unsloth-q3kxl/mmproj-F16.gguf ${model_dir}/qwen3.6-27b/" >&2
echo "[preflight] # mv ${model_dir}/qwen3.6-27b-gguf/unsloth-q3kxl/mmproj-F16.gguf ${model_dir}/qwen3.6-27b-gguf/" >&2
echo "[preflight] (~16 GB total. setup.sh today only fetches the vLLM AutoRound weights;" >&2
echo "[preflight] GGUF must be fetched separately for any llamacpp/* variant.)" >&2
fi