alex/club-3090
1
0
Fork 0
Code Issues Pull Requests Actions 38 Packages Projects Releases Wiki Activity

docs(FAQ): add 5-step triage ladder before symptom-matching

Browse Source 
When users hit boot OOMs, MTP weirdness, or TQ3/long-context budget
issues, our previous troubleshooting flow jumped straight to
symptom-matching. That misses the systematic narrowing pass: validate
the simplest stack works first, then add one variable per step until
the failing layer is isolated.

New leading section in Troubleshooting:

- Step 1 — `vllm/minimal` (32K + fp8, no Genesis, no spec-decode):
  validates hardware, driver, Docker, NVIDIA Container Toolkit, model
  files, base vLLM. Strips out every layer that could be the cause.

- Step 2 — `vllm/tools-text` (75K + fp8 + MTP + Genesis): adds Genesis
  + MTP K=3. Still fp8 KV (no TQ3 yet). Most common failure here is
  GENESIS_PIN-vs-tree mismatch — re-run setup.sh.

- Step 3 — `vllm/long-text` (180K + TQ3 + MTP + full Genesis): adds
  TurboQuant 3-bit KV + long-context. The production-target single-card
  config. Failure here narrows to TQ3 setup, GDN >60K hardware wall,
  or Cliff 1 mech B (closed since v7.69 PN25).

- Step 4 — `vllm/dual` (262K + fp8 + TP=2 + 2 streams, Genesis-less):
  adds TP=2 NCCL + multi-GPU memory split. Crucially removes Genesis,
  since dual.yml is intentionally Genesis-less. Failure here despite
  step 3 working narrows to TP=2 NCCL specifically. WSL2 is the most
  common trigger (its vGPU layer adds memory accounting wrinkles that
  bare-metal Linux doesn't have).

- Step 5 — `vllm/dual-turbo` (262K + TQ3 + TP=2 + Genesis): the full
  multi-card stack. Failure here despite step 4 narrows to the
  TQ3-on-TP=2-with-Genesis intersection.

The ladder works for both single and dual-card users because steps 1-3
isolate stack layers regardless of GPU count, and steps 4-5 add TP=2
surface separately. A dual-card user hitting issues should still run
steps 1-3 on a single card first — it's the only way to tell apart
"single-card stack issue that also breaks dual" vs "TP=2 NCCL specific."

Existing symptom-pattern-matching list demoted to "Quick recognition
guide" sub-section, kept verbatim — still useful for users who already
know the failure surface and want to skip the ladder.

Going forward we'll point to this ladder explicitly in triage replies
on issues / discussions, replacing the ad-hoc "have you tried X?" pattern.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
noonghunna
2026-05-02 23:45:58 +00:00
parent 1f09a059d5
commit 9560efd1f7
1 changed files with 102 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

103
docs/FAQ.md
View File

@@ -160,7 +160,108 @@ The [bug report template](https://github.com/noonghunna/club-3090/issues/new?tem
## Troubleshooting
Quick recognition guide for common failure modes:
### Before symptom-matching — boot the simplest stack first
If you're hitting boot OOMs, weird MTP behavior, or memory-budget issues
on TQ3 / long-context configs, validate that your hardware + driver +
container runtime + model files are fundamentally sound by booting the
simplest variant first. Each step adds one variable on top of the
previous; if step N works and step N+1 fails, the new variable is the
cause.
**Step 1 — `vllm/minimal` (32K + fp8 + no Genesis + no spec-decode)**
```bash
bash scripts/launch.sh --variant vllm/minimal
```
Tests: hardware, driver, Docker, NVIDIA Container Toolkit, model files,
base vLLM. Strips out everything that could be the cause.
- ✅ Boots cleanly → your stack is fundamentally sound. Continue to step 2.
- ❌ Fails — the issue is fundamental (driver mismatch, model files
missing or corrupt, container runtime, base vLLM image). Fix at this
layer before trying anything else. Symptom-match against the table
below or run `bash scripts/report.sh > my-rig.md` and file a bug.
**Step 2 — `vllm/tools-text` (75K + fp8 + MTP + Genesis)**
```bash
bash scripts/switch.sh vllm/tools-text
```
Adds: Genesis patches + MTP K=3 spec-decode. Still fp8 KV (no TQ3 yet).
- ✅ Boots cleanly → Genesis + MTP layer is sound. Continue to step 3.
- ❌ Fails — narrow to Genesis or MTP specifically. Most common gap:
on-disk Genesis tree at `models/qwen3.6-27b/vllm/patches/genesis/`
out of sync with `GENESIS_PIN` in `scripts/setup.sh`. Re-run
`bash scripts/setup.sh qwen3.6-27b` to refresh the tree.
**Step 3 — `vllm/long-text` (180K + TQ3 + MTP + full Genesis)**
```bash
bash scripts/switch.sh vllm/long-text
```
Adds: TurboQuant 3-bit KV + long-context handling. This is the
production-target single-card config.
- ✅ Boots cleanly → single-card stack fully validated. If you only
need single-card, stop here — this is what we ship as the IDE-agent
default.
- ❌ Fails — narrow to TQ3 or long-context specifically. If `tools-text`
worked but `long-text` doesn't, the issue is in TQ3 KV setup, GDN
cliff envelope (>60K single prompts hit the hardware wall on 24 GB),
or Cliff 1 mech B compile-path (closed since v7.66 + PN25 — confirm
Genesis tree is at v7.69 = `2db18df`).
**Step 4 — `vllm/dual` (262K + fp8 + TP=2 + 2 streams, Genesis-less)**
For dual-card users only. `dual.yml` is **intentionally Genesis-less**
(per its YAML header) — fp8 KV + TP=2 doesn't trigger the cudagraph
bug class Genesis was built to patch.
```bash
bash scripts/switch.sh vllm/dual
```
Adds: TP=2 NCCL coordination + multi-GPU memory split. Removes Genesis.
- ✅ Boots cleanly with steps 1-3 also passing → TP=2 path works. If
`long-text` (single-card with Genesis) AND `dual` (TP=2 without
Genesis) both work but `dual-turbo` (TP=2 + TQ3 + Genesis) doesn't,
the bug is specifically in the TQ3-on-TP=2-with-Genesis intersection.
- ❌ Fails despite step 3 working — the issue is in TP=2 NCCL
coordination or multi-GPU memory budget. WSL2 is the most common
trigger here (its vGPU layer adds memory accounting wrinkles that
bare-metal Linux doesn't have); native Linux + 2× 3090 PCIe is
well-tested. If you're on WSL2 and hitting this, native Linux or
switching to single-card `long-text` is the off-ramp.
**Step 5 — `vllm/dual-turbo` (262K + TQ3 + TP=2 + 4 streams, full Genesis)**
```bash
bash scripts/switch.sh vllm/dual-turbo
```
Adds: TQ3 KV + Genesis on top of TP=2 + 4-stream concurrency.
- ✅ Boots and verify-stress passes → full dual-card stack validated.
- ❌ Fails despite steps 3 and 4 working — the bug is specifically in
the multi-card TQ3+Genesis intersection. File a bug with `report.sh`
output; this is a narrow surface we'd want to debug carefully.
### Why this works for both single and dual-card users
The first 3 steps isolate stack layers (base → Genesis+MTP+fp8 →
TQ3+long-ctx). Steps 4-5 add TP=2 surface separately. A user on dual
hardware who's hitting issues should still run steps 1-3 on a single
card first — it's the only way to tell apart "issue in single-card
stack that also breaks dual" from "issue specific to TP=2 NCCL /
multi-GPU coordination."
### Quick recognition guide for common failure modes
- **Container dies at boot with `GPTQ_MARLIN_MIN_THREAD_N (64) > out_features`** — dual-card vllm#40361 patch didn't apply. Confirm `/opt/ai/vllm-src/` exists with the patched marlin kernel files.
- **Container dies during DFlash boot** — vllm#40334 dtype mismatch. Verify the compose has `--dtype bfloat16`.