alex/club-3090
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: fix v0.8.0 docs-fidelity gaps (trc-ack first-run heads-up, exit-code honesty, GGUF message claim)

Browse Source 
From the v0.8.0 docs-fidelity test (#369) — align docs with shipped CLI:

- PULL.md Quickstart + FAQ: first-run heads-up that common archs
  (Qwen2ForCausalLM &c) hard-block at needs-trust-remote-code-ack even
  with --dry-run; add --trust-remote-code (after vetting the code) to
  clear it. (Was a silent new-user wall.)
- PULL.md exit-codes: documented honestly — argparse usage/arg errors
  exit 2 (shared with honest hard-stop); 64 is reserved, arg-parser
  errors do not currently reach it (tracked CLI follow-up, #370).
- FAQ GGUF claim: "clear message" → accurate "aborts as
  unsupported-format (generic message; clearer GGUF message is a
  tracked v0.8.1 follow-up), not a crash".

Additive, leak-clean, links resolve, curated path untouched. Docs-only
(triggers no CI). Follows the (b) cross-link pass afe56f7.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
noonghunna
2026-05-17 20:39:33 +00:00
parent afe56f763f
commit 78a7dee247
2 changed files with 6 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/FAQ.md
View File

@@ -66,11 +66,11 @@ We tried EAGLE — it's blocked on Qwen3-Next (the family Qwen3.5/3.6 belong to)
### The model I want isn't in the supported list — can I still run it?
Yes, if it's a **safetensors** repo. As of v0.8.0, `scripts/pull.sh <org/Model> --profile-like vllm/minimal --dry-run` evaluates *any* safetensors HF repo against this stack's KV math — no download — and tells you honestly whether it fits and at what confidence. Drop `--dry-run` (add `--yes`) and, if it passes the gates, it downloads, generates a minimal compose, and boots it. Non-fits stop with a precise reason, not a crash. Full guide: [docs/PULL.md](PULL.md). Limits: safetensors + vLLM only; GGUF / `.bin` repos hard-block with a clear message (see next Q).
Yes, if it's a **safetensors** repo. As of v0.8.0, `scripts/pull.sh <org/Model> --profile-like vllm/minimal --dry-run` evaluates *any* safetensors HF repo against this stack's KV math — no download — and tells you honestly whether it fits and at what confidence. Drop `--dry-run` (add `--yes`) and, if it passes the gates, it downloads, generates a minimal compose, and boots it. Non-fits stop with a precise reason, not a crash. Full guide: [docs/PULL.md](PULL.md). One heads-up: many common archs (e.g. `Qwen2ForCausalLM`) stop at `needs-trust-remote-code-ack` on the first try even with `--dry-run` — add `--trust-remote-code` (after checking what code the repo runs) to clear it. Limits: safetensors + vLLM only; GGUF / `.bin` repos abort at derive as `unsupported-format` (not a crash) — see next Q.
### Why not GGUF on vLLM for this model?
Multiple gates blocked. Qwen3.6-27B GGUF on vLLM hits a chain of "fixed but-not-quite" issues — multimodal config routing, ParallelLMHead skip, the `Qwen35TensorProcessor._reverse_reorder_v_heads` weight loader producing garbage output on the 27B layout (transformers PR #45283 only validated on 0.8B). Tracked in [INTERNALS.md](../models/qwen3.6-27b/INTERNALS.md#qwen36-27b-gguf-on-vllm). Use llama.cpp for GGUF on this model. **Note (v0.8.0):** `pull` evaluates *safetensors* repos only — GGUF→llama.cpp is **not** served via `pull` (it stays the curated/manual path; cross-engine generation is deliberately deferred). A GGUF repo hard-blocks at derive with a clear message, not a crash.
Multiple gates blocked. Qwen3.6-27B GGUF on vLLM hits a chain of "fixed but-not-quite" issues — multimodal config routing, ParallelLMHead skip, the `Qwen35TensorProcessor._reverse_reorder_v_heads` weight loader producing garbage output on the 27B layout (transformers PR #45283 only validated on 0.8B). Tracked in [INTERNALS.md](../models/qwen3.6-27b/INTERNALS.md#qwen36-27b-gguf-on-vllm). Use llama.cpp for GGUF on this model. **Note (v0.8.0):** `pull` evaluates *safetensors* repos only — GGUF→llama.cpp is **not** served via `pull` (it stays the curated/manual path; cross-engine generation is deliberately deferred). A GGUF/`.bin` repo aborts cleanly at the deriver stage as `unsupported-format` (the message is generic — it does not yet say "GGUF, use llama.cpp"; a clearer message is a tracked v0.8.1 follow-up), not a crash.
### Why AutoRound INT4 not GPTQ / AWQ?