b07b2f99e7
Conceptual narrative (hardware → engine → model size → quant → templates) that the doc set lacked: GETTING_STARTED is commands-only, GLOSSARY is isolated definitions, FAQ is troubleshooting. Cross-linked from README, GETTING_STARTED, and GLOSSARY. Carries a scope banner (stack is NVIDIA/3090-tested; other vendors are context, not a support commitment) and an RTX 3090 FP8 caveat (no FP8 weight compute on Ampere; AWQ/GPTQ for weights, fp8_e5m2 KV only). Links to PULL.md / KV_MATH.md for the stack's actual fit-math path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1.6 KiB
1.6 KiB
Getting started — zero to curl in 5 minutes
The fastest path from git clone to serving your first response. No decisions, no menus — just commands.
New to local AI and the terms below feel like jargon? Read LOCAL_AI_PRIMER.md first — how hardware, engines, model sizes, and quants fit together in plain English.
# 1. Clone
git clone https://github.com/noonghunna/club-3090.git
cd club-3090
# 2. Download the model (Qwen3.6-27B, ~18 GB)
bash scripts/setup.sh qwen3.6-27b
# 3. Boot the default config (single-card chat, 48K context)
bash scripts/launch.sh --variant vllm/default
# 4. Test it
curl -sf http://localhost:8020/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"qwen3.6-27b-autoround","messages":[{"role":"user","content":"Capital of France?"}],"max_tokens":200}'
If you see Paris in the response, you're up and running.
Next steps
| You want | Go here |
|---|---|
| Pick a config by workload (long context, vision, dual-card, etc.) | docs/SINGLE_CARD.md or docs/DUAL_CARD.md |
| Understand the jargon (TPS, KV, MTP, TP) | docs/GLOSSARY.md |
| Client code snippets (Python, curl, IDE setup) | docs/EXAMPLES.md |
| Run the canonical benchmark | bash scripts/bench.sh |
| Update to the latest | bash scripts/update.sh |
| Hardware questions (power caps, NVLink, 4090/5090) | docs/HARDWARE.md |
| File an issue or share bench numbers | bash scripts/report.sh --full > my-rig.md and open an issue |