ec704e4e2e
- setup.sh: GENESIS_PIN now defaults to commit bf667c7 (Genesis HEAD as of 2026-04-27, semver "v7.54"). This is the exact tree our published TPS numbers were measured against; tagged v7.51-stable was one minor older but came up first because the SHA isn't durable. Switch to commit pin removes the doc-vs-runtime mismatch. Clone strategy adjusted since --branch + --depth 1 doesn't accept SHAs. - .env.example: documents MODEL_DIR / HF_TOKEN / CUDA_VISIBLE_DEVICES / MEM_UTIL / MAX_MODEL_LEN / GENESIS_PIN / SKIP_GENESIS / URL / WARMUPS / RUNS with the same defaults the composes ship. Pure opt-in. - .github/ISSUE_TEMPLATE/: bug-report.yml requires docker logs --tail 100, verify-full.sh output, nvidia-smi, GPU config, compose variant, repo commit. numbers-from-your-rig.yml structures cross-rig TPS contributions with rig spec, bench output, VRAM, max ctx, and notes. config.yml routes Q&A to Discussions. - .gitignore: drop trailing slash on genesis pattern so it also ignores local symlinks that some of us point at out-of-tree clones. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
88 lines
3.3 KiB
Plaintext
88 lines
3.3 KiB
Plaintext
# club-3090 environment overrides
|
|
#
|
|
# Copy to `.env` (or export inline) and edit only the values you want to
|
|
# change. Every variable below has a sensible default in the compose files
|
|
# and scripts — `.env` is purely for opt-in overrides.
|
|
#
|
|
# cp .env.example .env
|
|
# $EDITOR .env
|
|
#
|
|
# Variables are read by `docker compose ...` automatically when run from the
|
|
# directory that contains a `.env`. For scripts/setup.sh you can either
|
|
# `export` them in your shell or prefix the command:
|
|
#
|
|
# MODEL_DIR=/scratch/models bash scripts/setup.sh qwen3.6-27b
|
|
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# Model storage
|
|
# -----------------------------------------------------------------------------
|
|
|
|
# Where weights live. Default is <repo>/models-cache. Override if you keep
|
|
# weights on a separate disk (e.g. /mnt/models). Path is resolved as-is —
|
|
# absolute paths recommended.
|
|
# MODEL_DIR=/mnt/models
|
|
|
|
# HuggingFace token. Only needed for gated/private repos. The Lorbus
|
|
# AutoRound INT4 weights and unsloth GGUFs we use are public, so this is
|
|
# usually blank.
|
|
# HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxx
|
|
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# GPU selection
|
|
# -----------------------------------------------------------------------------
|
|
|
|
# Which GPUs to expose to docker. Single-card composes use just `0`,
|
|
# dual-card composes use `0,1`. The compose files set this themselves;
|
|
# override here if your physical layout differs (e.g. you run dual-card
|
|
# on cards 2,3).
|
|
# CUDA_VISIBLE_DEVICES=0,1
|
|
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# vLLM tuning knobs
|
|
# -----------------------------------------------------------------------------
|
|
|
|
# GPU memory utilization fraction. The shipped composes use:
|
|
# single-card default (48K): 0.92
|
|
# single-card long-vision/text: 0.98
|
|
# dual-card: 0.92
|
|
# Lowering this leaves more headroom for prefill spikes — see
|
|
# docs/INTERNALS.md "prefill cliffs". Raising above 0.95 is risky on
|
|
# the 48K config but fine on long-* configs because the model + KV
|
|
# fully consume the card anyway.
|
|
# MEM_UTIL=0.92
|
|
|
|
# Max sequence length. Shipped defaults: 48K (chat-safe), 192K (long-vision),
|
|
# 205K (long-text). Going beyond these on a 24GB card needs --mem-fraction
|
|
# tuning; see docs/USE_CASES.md for the curve.
|
|
# MAX_MODEL_LEN=48000
|
|
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# Genesis patch tree (vLLM only)
|
|
# -----------------------------------------------------------------------------
|
|
|
|
# Pin Sandermage/genesis-vllm-patches to a specific ref. Default is the exact
|
|
# commit our published TPS numbers were measured against. Bumping requires
|
|
# re-running scripts/verify-full.sh to confirm your new ref still passes.
|
|
# GENESIS_PIN=bf667c7
|
|
|
|
# Skip cloning Genesis (e.g. you only intend to run llama.cpp / SGLang).
|
|
# SKIP_GENESIS=1
|
|
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# verify-full.sh / verify-stress.sh
|
|
# -----------------------------------------------------------------------------
|
|
|
|
# Endpoint to test against. Default localhost:8020 matches every shipped
|
|
# compose's host port.
|
|
# URL=http://localhost:8020
|
|
|
|
# Number of warmup + measured runs in the bench script. Defaults are
|
|
# 3 warmup + 5 measured. Lower these for a faster smoke test.
|
|
# WARMUPS=3
|
|
# RUNS=5
|