diff --git a/CHANGELOG.md b/CHANGELOG.md index d90dc0e2..a5a1a902 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,14 @@ Changes that span the entire stack — engine version pins, script behavior, repo structure. Per-model dated history lives in `models//CHANGELOG.md`. +## 2026-04-29 — Disable Genesis P68/P69 in shipped composes (silent-stop bugfix) + +- **`tools-text.yml`** + **`fast-chat.yml`** — `GENESIS_ENABLE_P68_AUTO_FORCE_TOOL` and `GENESIS_ENABLE_P69_LONG_CTX_TOOL_REMINDER` are now commented out (default-off). Caused silent finish_reason=stop with empty content + no tool_calls on greetings and clarifying questions when the prompt exceeded 8000 chars (the patches' threshold). Affected every realistic IDE-agent setup (Cline, Cursor, OpenCode, Copilot Gateway). +- **Bisected via cross-rig data on club-3090 issue #2** (HoodOG1 + tenitram). State A (all 4 env vars on) reproduced the silent stop. State B (P68 off, P69 on) still broken — model loops on "I cannot respond with plain text" then stops mid-reasoning. State D (both P68 and P69 off) — clean: greeting → "Hello! How can I help you today?", tool request → clean `read_file` call. +- **P64 and PN8 stay enabled** — both are real targeted bugfixes (P64 = qwen3coder MTP streaming early-return fix from kotori-yan vllm#39598 backport; PN8 = FP8+MTP draft online-quant memory savings, vllm#40849 backport). Neither overrides user intent. +- Mechanism: `vllm/_genesis/middleware/long_ctx_tool_adherence.py:227` — P68 silently sets `request.tool_choice = "required"` when prompt > 8000 chars; P69 appends "must use a tool" text to the user message. Either alone makes "hi there" + tools fail. Together they're worse. +- Threshold of 8000 chars is too low for IDE agents (typical context: 20-50K). We may file an upstream issue with Sandermage suggesting either raising the default or making opt-in+threshold mandatory-explicit. + ## 2026-04-29 — UX polish: pre-flight checks + cards-first wizard + PNG embeds + per-page chart split - **`scripts/preflight.sh`** (new) — sourceable library of `preflight_docker`, `preflight_gpu [min]`, `preflight_disk `, `preflight_gpu_idle`, `preflight_running`. Each prints actionable `Fix:` hints on failure rather than a cryptic mid-run crash. diff --git a/docs/FAQ.md b/docs/FAQ.md index 1098b473..f2e0d6c6 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -120,7 +120,9 @@ bash scripts/switch.sh vllm/tools-text There's a second wrinkle: Copilot's LLM Gateway sometimes sends very low `max_tokens` (e.g. 64) on probe-style requests. With `tool_choice: required` (which Copilot enforces via `minItems: 1` on its structured-outputs schema), the model must emit a tool-call JSON that wraps a real argument like a file path — and 64 tokens isn't enough to fit `{"name": "read_file", "parameters": {"filePath": "/long/abs/path"}}`. The truncated JSON arrives at the gateway as "empty response." If you see this pattern, it's a client-side limit, not the server. Other OpenAI-compat clients (Cline / Continue.dev / Cursor) tend to send realistic max_tokens by default and don't hit this. -Background + debug-log analysis: [club-3090 #2](https://github.com/noonghunna/club-3090/issues/2). +**Server-side fix landed 2026-04-29:** the Genesis P68/P69 long-context tool-adherence patches were silently overriding `tool_choice: auto → required` and injecting "must use a tool" reminders whenever prompt > 8000 chars. That made greetings + clarifying questions stall on every IDE-agent setup (Cline, Cursor, OpenCode, and Copilot Gateway combined). We disabled both in `tools-text.yml` and `fast-chat.yml`. Behavior now: greeting → plain-text reply ("Hello! How can I help you today?"); tool request → clean `read_file({"path": "..."})` call. P64 and PN8 stay enabled (real targeted bugfixes, no user-intent override). + +Background + bisection: [club-3090 #2](https://github.com/noonghunna/club-3090/issues/2#issuecomment-4346345554). --- diff --git a/models/qwen3.6-27b/vllm/compose/docker-compose.fast-chat.yml b/models/qwen3.6-27b/vllm/compose/docker-compose.fast-chat.yml index 8f0c0c0d..fd648d13 100644 --- a/models/qwen3.6-27b/vllm/compose/docker-compose.fast-chat.yml +++ b/models/qwen3.6-27b/vllm/compose/docker-compose.fast-chat.yml @@ -63,8 +63,11 @@ services: # We don't enable P4 (TurboQuant-hybrid) — fp8 KV doesn't need it. # We don't enable P65/P67 — fp8 KV doesn't trigger #40880 cudagraph bug. - GENESIS_ENABLE_P64_QWEN3CODER_MTP_STREAMING=1 - - GENESIS_ENABLE_P68_AUTO_FORCE_TOOL=1 - - GENESIS_ENABLE_P69_LONG_CTX_TOOL_REMINDER=1 + # P68/P69 deliberately DISABLED (2026-04-29) — see tools-text.yml header + # for full reasoning. Short version: 8000-char threshold is too low for + # IDE agents and breaks greetings / clarifying questions silently. + # - GENESIS_ENABLE_P68_AUTO_FORCE_TOOL=1 + # - GENESIS_ENABLE_P69_LONG_CTX_TOOL_REMINDER=1 - GENESIS_ENABLE_PN8_MTP_DRAFT_ONLINE_QUANT=1 # v7.62.x — opt-in test shm_size: "16gb" ipc: host diff --git a/models/qwen3.6-27b/vllm/compose/docker-compose.tools-text.yml b/models/qwen3.6-27b/vllm/compose/docker-compose.tools-text.yml index 9109a3bb..59b12cc6 100644 --- a/models/qwen3.6-27b/vllm/compose/docker-compose.tools-text.yml +++ b/models/qwen3.6-27b/vllm/compose/docker-compose.tools-text.yml @@ -52,8 +52,14 @@ services: - VLLM_MARLIN_USE_ATOMIC_ADD=1 # Genesis v7.14+ opt-in patches (fp8 KV path — no P4/P65/P67 needed) - GENESIS_ENABLE_P64_QWEN3CODER_MTP_STREAMING=1 - - GENESIS_ENABLE_P68_AUTO_FORCE_TOOL=1 - - GENESIS_ENABLE_P69_LONG_CTX_TOOL_REMINDER=1 + # P68/P69 deliberately DISABLED (2026-04-29). They override user + # tool_choice / inject "must use a tool" reminders when the prompt + # exceeds 8000 chars — IDE-agent system prompts blow past that on + # the first message and get silent stops on greetings / clarifying + # questions. Bisected on club-3090 issue #2 (HoodOG1 + tenitram). + # Set to 1 only if you specifically want forced-tool-call behavior. + # - GENESIS_ENABLE_P68_AUTO_FORCE_TOOL=1 + # - GENESIS_ENABLE_P69_LONG_CTX_TOOL_REMINDER=1 # PN8 (v7.62.x): MTP draft online-quant propagation. Targets fp8+MTP # path specifically — frees ~600 MiB by inheriting target quant config # into the draft head. Sandermage's claim: closes Cliff 1 on this path.