setup-ai-studio.sh's director download honored MODEL_DIR, but the ComfyUI
assets (image/video/audio) defaulted to a hardcoded /mnt/models/comfyui — so
any rig whose models don't live under /mnt failed mid-download with
"mkdir: Permission denied" (sumo Discord report, 2026-06-27). Two more walls
behind it: the disk-space preflight checked the hardcoded /mnt path and had no
override (an idempotent resume under-threshold was blocked), and gpu-mode.sh —
which setup calls at step 3 — hardcoded CLUB3090_DIR=/opt/ai/github/club-3090,
breaking on any other clone.
- New services/comfyui/comfyui-paths.sh: derive COMFYUI_ROOT / COMFYUI_MODELS_DIR
as a "comfyui" sibling of MODEL_DIR (read from repo-root .env when unset).
Backward-compatible on the rig (/mnt/models/huggingface -> /mnt/models/comfyui);
explicit overrides respected. Sourced by setup-ai-studio.sh,
download_studio_models.sh, and gpu-mode.sh so the download target, the disk
check, and the container mounts all agree.
- setup-ai-studio.sh: disk check uses the derived path; add SKIP_DISK_CHECK=1 to
bypass the free-space preflight independently of SKIP_DOWNLOAD.
- gpu-mode.sh: auto-detect CLUB3090_DIR from the script's real location (resolving
the /usr/local/bin symlink), overridable via env — removes the hardcoded path.
- The 3 legacy hardcoded download scripts (flux2/hunyuan_llava/models) honor
COMFYUI_MODELS_DIR.
- New test-comfyui-paths.sh guards the derivation; docs/ai-studio config note.
Full suite green (59/59).
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Co-authored-by: noonghunna <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
The nvidia-power-cap.service unit shipped in #485 caps both cards at 250W (rtx-3090.yml prefill operating point), but gpu-mode.sh's comments, echoes, and the service-missing fallback (-pl 230) all still said 230W. Align them so the messaging + fallback match the unit.
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Co-authored-by: noonghunna <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
* Director placement lever: env-driven CPU / GPU0 / GPU1 (backend)
The studio director's GPU/CPU placement is now a single lever,
STUDIO_DIRECTOR_DEVICE (gpu0 | gpu1 | cpu, default gpu0), read from the rig
.env by gpu-mode's start_studio_director and translated into the compose
env (-ngl + CUDA_VISIBLE_DEVICES + device_ids):
- gpu0 (default): -ngl 99, GPU0 — fast craft (~50-100 tok/s), ~4.6 GB,
coexists with the image lanes. Unchanged from before.
- gpu1: -ngl 99, GPU1 — only when GPU1 has room (NOT during a video render;
GPU1 is the DisTorch DiT donor).
- cpu: -ngl 0, CUDA_VISIBLE_DEVICES="" — frees ~4.6 GB off GPU0 (lifts the
single-card Wan window 121→161 frames) at ~single-digit tok/s craft.
Compose now reads ${DIRECTOR_NGL:-99} + ${STUDIO_DIRECTOR_CUDA-0} (no-colon
so an explicit empty value = CPU survives). Default (no override) preserves
current GPU0 behaviour exactly.
Live-validated: CPU mode starts with GPU0 full (gemma12b), adds 0 MiB VRAM
to GPU0, serves on :8090, generates (~5 tok/s CPU). The c3 Settings field
that writes STUDIO_DIRECTOR_DEVICE follows in the next commit.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
* c3 Settings: Director placement (CPU / GPU0 / GPU1) + CPU thread cap
Exposes the director-placement lever in the c3 Settings modal so users pick
GPU0 (fast, default) / GPU1 / CPU without hand-editing the .env.
- services.py: director_device() reads STUDIO_DIRECTOR_DEVICE from the repo
.env (default gpu0, validates the value); set_repo_env_var() upserts a key
in place (preserves other lines, no duplicates, creates the file if absent).
- app.py: SettingsScreen gains a "Director placement" Select; apply_settings
persists the choice to the repo .env (the SHARED config gpu-mode reads —
distinct from c3-settings.json for MODEL_DIR/HF_TOKEN). Applies on the next
ai-studio start.
- compose: CPU thread cap — -t ${DIRECTOR_THREADS:-8} bounds CPU use so the
director doesn't starve OWUI's embedder/reranker (also CPU). The ~2.6 GB
GGUF loads into system RAM (mmap'd; resident in page cache, not run from SSD).
- tests: +6 data-layer (TestDirectorPlacement) + 1 headless apply-settings
round-trip (persists STUDIO_DIRECTOR_DEVICE, idempotent re-apply). Full
suite green (728), settings/director subset 13/13.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
* tests: fix stale scene names in gpu-mode-list EXPECT
The scene catalog renamed its dispatch keywords to qwen27b / gemma-31b,
but the test's EXPECT spot-check map still referenced the old 27b / gemma
short names — so the JSON-shape assertion had been red on master. Point
EXPECT at the canonical names the catalog now emits.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
* studio: run director in chat scene as the catalog-support layer
Bring the uncensored director up in the `chat` scene (honoring the
STUDIO_DIRECTOR_DEVICE placement knob), framing chat as the supporting-
infra home — OWUI + LiteLLM + Qdrant + SearXNG + director — for ad-hoc
Catalog models launched via `switch.sh --owui`.
A CPU-placed director uses no GPU, so it's the always-on path: it survives
scene switches and stays live in OWUI. New _director_evict_if_gpu helper
frees only a GPU-resident director when a dual-card LLM scene claims the
cards; mode_off stops it outright. Also brings mode_gemma_int8 in line with
its dual-card siblings (it was missing the studio teardown entirely).
Docs: requirements.md gains a "Chat scene — the Catalog-support layer"
section + reframes director placement around the unified knob / c3 Setting;
video.md note synced.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
* studio: disable director thinking on CPU only (latency)
The uncensored director is an "Aggressive" reasoning fine-tune that emits
a full <think> trace before every answer. On GPU that's cheap and the
trace lands in reasoning_content (content stays clean), so leave it on.
On CPU (~14 tok/s) the trace dominates latency, so gpu-mode now passes
`--jinja --reasoning off` for the cpu placement only — forcing the
template's enable_thinking=false (this fine-tune ignores /no_think and
--reasoning-budget 0, but honors --reasoning off).
Wired via a new DIRECTOR_THINK_ARGS compose param (empty on GPU). Live:
CPU director now answers in one pass, no reasoning trace, craft quality
intact (full cinematic spec, finish=stop).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
* c3: Containers-pane director start honors placement + enumerate sidecars
Two consistency fixes for the Containers tab:
1. Starting studio-director from Containers now honors STUDIO_DIRECTOR_DEVICE
instead of falling back to the GPU0/thinking-on compose default. New
director_compose_env() mirrors gpu-mode's start_studio_director translation
(NGL/CUDA/GPU/THINK_ARGS), injected as an `env K=V …` prefix on the compose
up cmd (process env wins over --env-file). cpu → -ngl 0 + --reasoning off.
2. The nested studio sidecars (director/gallery/orchestrator/image-shim/
step-voice/tts) now enumerate when STOPPED, so they're startable rows — not
only visible while running. New STUDIO_SIDECARS map is the single SoT for
resolving the container-name → services/studio/<sub>/ project (fixing the
director↔enhancer name mismatch that previously returned None → docker
restart, which fails on a fresh install).
+8 tests (director_compose_env cpu/gpu, director resolves to enhancer with the
env prefix, sidecar enumeration). Live: c3 service_start plan starts the director
CPU + no-think (argv -ngl 0 --reasoning off, GPU0 free, clean generation).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
---------
Co-authored-by: noonghunna <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
* Lazy-load the step-voice service + scene-wire it into ai-studio
Make the premium-voice lane "just work" in the studio without pre-pinning
GPU1: the step-voice container is now part of the ai-studio scene, but the
~14 GB Step-Audio-EditX model loads on first use instead of at boot, and is
evicted before a video render so it never OOMs against the LTX/Wan DiT
(both want GPU1, the DisTorch donor is ~21.9 GB).
server.py — lazy lifecycle (STEP_VOICE_LAZY=1, default):
- model loads on the FIRST /clone or /edit (lock-guarded), not at import →
the container boots at ~0 GB GPU, so the OWUI voice lane appears as soon
as ai-studio starts it (the pipe gates the lane on :8193 reachability).
- new POST /unload frees the model from GPU; an idle-unload timer
(STEP_VOICE_IDLE_UNLOAD_S, default 300 s) reclaims GPU1 automatically.
- /health reports {ready, loaded, lazy}; transcribe+clone moved off the
event loop into the worker. STEP_VOICE_LAZY=0 restores eager boot.
build_studio_pipe.py / studio_pipe.py — deterministic voice⊕video mutex:
- video lanes (ltx/sulphur/10eros/wan) POST /unload before submitting the
render, so a freshly-used voice model can't collide with the video DiT.
gpu-mode.sh — scene wiring:
- start_step_voice in mode_ai_studio (lazy → cheap); stop_step_voice
wherever stop_comfyui is called (every LLM scene + mode_off) so GPU1 is
freed for a dual-card model. ai-studio --list-modes row + helpers added.
docker-compose.yml — STEP_VOICE_LAZY + STEP_VOICE_IDLE_UNLOAD_S env,
restart policy note (scene-managed), header rewritten for lazy behaviour.
Live-validated via the real compose (--build bakes server.py): lazy boot
loaded:false @ ~4 MiB GPU1, first /clone loads (~13 GB) + clean audio,
POST /unload frees back to ~0, reload clean; gemma12b on GPU0 untouched.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
* ai-studio scene: start qdrant (OWUI's vector DB) + list it
Every model scene starts qdrant (start_service qdrant) but mode_ai_studio
didn't — it predates OWUI's Qdrant vector-DB wiring. So in ai-studio OWUI
ran without its vector store started by the scene (document RAG / web-search
embeddings would have no backend on a clean ai-studio launch from mode_off).
Add start_service qdrant to mode_ai_studio and qdrant:6333 to its
--list-modes service/port row, so it actually starts AND shows up in c3's
ai-studio scene preview alongside studio-step-voice.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
---------
Co-authored-by: noonghunna <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
New `gpu-mode gemma12b` scene (alias gemma-12b) serves
vllm/gemma-12b-single-int8-mtp on :8038 — Gemma 4 12B (gemma4_unified
arch) AutoRound INT8 + bf16 KV + MTP n=2, single card (frees the other
GPU). Stops all dual models + ai-studio on entry; the single-card stop
is folded into the shared stop_all_gemma helper so every other model
scene auto-evicts it, with an explicit line in mode_gemma_int8 (the one
mode that doesn't call stop_all_gemma).
Wires it end-to-end: --list-modes TSV + JSON row (c3 Operate picks it
up), :8038 status probe, all-off detection chain, usage help, and a
litellm gateway route gemma-4-12b-int8 -> :8038.
Live-validated: boots and serves gemma-4-12b-int8 on :8038, coherent
completion direct and through the litellm gateway (:4000).
Note: the gemma4_unified image tag is ephemeral (arch-preview) — pin a
digest before treating this scene as durable.
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
Co-authored-by: noonghunna <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
More descriptive scene names (the bare 27b/gemma were ambiguous). New canonical
names in the dispatch + the --list-modes TSV (what c3 Operate displays) + help;
OLD names kept as aliases so `gpu-mode 27b` / `gemma` / `35b-a3b` (+ a3b/35b)
still work — zero broken muscle memory.
27b → qwen27b (alias: 27b)
35b-a3b → qwen35b-a3b (alias: 35b-a3b, a3b, 35b)
gemma → gemma-31b (alias: gemma)
c3 reads --list-modes dynamically + invokes scenes by that name, so it shows
the new names and runs `gpu-mode qwen27b` with no code change. deckard unchanged.
bash -n clean; --list-modes carries the new names.
qwen3.6-35b-a3b-autoround had a litellm route (:8051) + was serveable via c3
Run·Catalog, but had no gpu-mode SCENE — so it never appeared in c3's Operate
scene list, unlike the other three LLMs.
Adds the full scene: A3B_DUAL_DIR, start/stop_35b_a3b_dual helpers (on the
production slug vllm/qwen-35b-a3b-dual, dual/autoround-int4/fp8.yml, :8051),
mode_35b_a3b, the `35b-a3b|a3b|35b` dispatch, the --list-modes TSV row (so it
shows in c3 Operate), the :8051 status probe, and a help line. Cross-mode stops:
every model mode (27b/gemma/deckard/ai-studio/off) now stops 35b-a3b so a switch
doesn't OOM — and mode_27b was also missing stop_deckard, fixed here too. litellm
:8051 route already correct.
Live-validated: `gpu-mode 35b-a3b` stops the running model + starts
vllm-qwen36-35b-a3b-dual on :8051; --list-modes --json carries the scene;
bash -n clean; no undefined start/stop/mode calls remain.
Switching to gemma stopped 27b but never started gemma. Root cause: `set -e`
+ mode_gemma_int8 called three functions deleted when the dflash/awq gemma
scenes were pruned (stop_gemma_dflash / _dflash_int8 / _awq) — the first
undefined call (`stop_gemma_dflash: command not found`) aborted the function
*after* the 27b teardown but *before* start_gemma_int8. Exactly the reported
"27b unloads, gemma fails to load".
- mode_gemma_int8: drop the 3 undefined calls; add the DEFINED stop_gemma_int8
(clean re-switch). Live-validated: `gpu-mode gemma` now starts
vllm-gemma-4-31b-mtp-int8 on :8032 and serves.
- Finish the partial prune: remove the 3 dead/unreachable gemma modes
(mode_gemma_dflash / _dflash_int8 / _awq — 0 dispatch refs) that referenced
the same deleted functions, + the orphaned GEMMA_DUAL_AWQ_DIR. No dangling
refs remain; bash -n clean.
- litellm: gemma-4-31b-autoround route :8030 → :8032 (the :8030 gemma-mtp scene
was pruned; the live scene serves :8032). End-to-end OWUI→litellm→gemma = 200.
Replace the separate `image-studio` / `video-studio` / `comfyui` scenes with
a single `ai-studio` scene: ComfyUI holds both GPUs always, and image / video /
audio are picked as *lanes* in OWUI rather than gpu-mode switches. Removes the
gemma-12b chat brain from the studio bundle (kept on disk, just unwired) and
points OWUI's DEFAULT_MODELS at the qwen director.
- gpu-mode.sh: `mode_ai_studio` replaces the two studio modes; drop the gemma
start/stop + the GPU0-pin split; preflight now reads the shared manifest.
- scripts/lib/studio-models.tsv: single source-of-truth model manifest (modality,
label, root, rel_path, size, installer) consumed by both gpu-mode preflight and
the c3 cockpit, so the two can't drift.
- download_{director,video,audio,ace_step,stable_audio,studio}_models.sh: idempotent
fetchers; download_studio_models.sh orchestrates "grab everything".
- setup-video-studio.sh + repointed setup-image-studio.sh → `gpu-mode ai-studio`.
- litellm: drop the gemma-4-12b :8069 route; comfyui compose/entrypoint GPU-pin
comments corrected (no scene sets the pin now).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
gpu-mode only STARTS the studio bundle; the models are fetched by
setup-image-studio.sh. So `gpu-mode image-studio` (or the cockpit, which calls
gpu-mode) on a box that skipped the installer booted broken services. Gate both
paths on the models being present.
- gpu-mode: preflight_studio_models() runs at the top of mode_image_studio /
mode_video_studio — checks the director GGUF (both) + the Ideogram set (image),
and on a miss prints `Fix: bash scripts/setup-image-studio.sh` and refuses to
start, instead of tearing down the LLMs then booting weightless containers.
Mirrors preflight_compose_deps; skip with STUDIO_NO_PREFLIGHT=1.
- cockpit: the studio guard now checks studio_ready() = comfyui-local image AND
the director GGUF on disk (was image-only), so it catches a missing director
BEFORE the scene-switch confirm. Cached flag renamed _comfyui_ready →
_studio_ready; STUDIO_DIRECTOR_REL kept in lock-step with the setup download +
the gpu-mode preflight.
Full cockpit suite 706 passed; bash gates 56/0.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01PoXV1FuKmTiDPB15qQvLjB
Four changes from review:
1. comfyui shows engine "comfyui" (not the generic "studio") — both running
(_docker_ps_stack_containers) and stopped (_merge_known_services); other GPU
rig services / studio stacks stay "studio".
2. Orchestration: remove the bottom Power-cap strip — the cap already shows on the
GPU cards at the top ("(cap NNNW)"), so the strip duplicated it. populate_power_cap
keeps caching the per-GPU cap for the cards; the [c] menu still adjusts it.
3. gpu-mode: remove the standalone `comfyui` scene (mode_comfyui + dispatch + TSV
row + usage help) — comfyui runs via image-studio / video-studio now. test EXPECT
+ cockpit _STUDIO_SCENES updated.
4. Containers now lists the STOPPED studio sidecars (studio-director/-gallery/
-orchestrator/-image-shim/-tts) + comfyui: _stopped_studio_containers reads
`docker container ls -a` for scene-managed service/stack containers that exist but
aren't running and aren't top-level services/ dirs (engines excluded as noise).
[s] on them uses service_start_or_restart — compose-up for a real service dir,
`docker restart` for an existing sidecar (its dir name ≠ container name).
Full cockpit suite 702 passed; bash gates 56/0.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01PoXV1FuKmTiDPB15qQvLjB
Ollama was dropped from serving 2026-05-10 (Qwen/Gemma route through LiteLLM
directly), but its services/ollama/ compose dir was kept "for manual spin-up" —
so the cockpit Containers tab (which lists every services/*/docker-compose.yml)
surfaced it as a stopped service that nothing in the stack starts. Remove it.
- Delete services/ollama/.
- gpu-mode.sh: drop the 9 defensive `stop_service ollama` calls, the :11434
status readout, and fix the stale `chat` usage-help line (it listed Ollama +
omitted SearXNG; the authoritative scene TSV was already correct).
- services/openwebui: drop the dead OLLAMA_BASE_URL env; set ENABLE_OLLAMA_API
=false so Open WebUI doesn't probe a now-dead :11434 on boot.
- cockpit services.py: drop ollama from the supporting-service docstring/comment
examples (it's no longer a known service).
The SERVICES=() array already excluded ollama, so no scene behaviour changes.
bash gates 56/0; cockpit container/known-service tests green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01PoXV1FuKmTiDPB15qQvLjB
gemma-int8 was a literal alias for gemma (same INT8 PTH container :8032), and
gemma-mtp (the bf16/32K/no-overlay fallback) isn't needed as a scene. Keep only
the 'gemma' scene (INT8 PTH KV, 262K, vision, :8032 — mode_gemma_int8).
Removed: the gemma-int8 + gemma-mtp TSV rows, their dispatch cases + help lines,
mode_gemma (the bf16 fallback mode) and start_gemma_mtp. Kept stop_gemma_mtp /
stop_gemma_int8 / stop_all_gemma — mode_off + the other gemma modes call them to
defensively tear down a stray bf16/int8 gemma container (the bf16 compose stays
serveable via its catalog slug).
test-gpu-mode-list EXPECT trimmed; repo gates 56/0. Cockpit untouched (the scene
table is data-driven — the dropped scenes just disappear from --list-modes).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Scene-catalog cleanup for the Orchestration tab (maintainer-directed):
- Regroup: 'serving' group → 'models'; 'chat' moves serving→ops (browser-chat
infra, no GPU model); 'diffusiongemma' moves studio→models (it IS a model).
Groups are now models / studio / ops. gpu-mode's plain + --json renders are
data-driven, so this is a TSV edit + the contract comment.
- Remove 'bigmodel': it was ~redundant with 'off' (both stop everything). Its one
useful extra — the freed VRAM/RAM readout — is folded into mode_off; the cache
drop was marginal (Linux auto-reclaims page cache; nothing for VRAM) and the
llama-server hint belongs in docs.
- Remove the 'diffusiongemma'/'dgemma' SCENE (TSV/dispatch/help/mode_/start_): a
niche dLLM whose scene duplicated the catalog slug vllm/diffusiongemma-dual (the
way to serve it). stop_diffusiongemma is KEPT — mode_off + the studio modes
call it to defensively clear a stray dgemma container.
- Cockpit scene filter switched from "ops-except-off" to a deny-list of the COMMAND
modes (power-cap / prune / prune-all) so 'chat' (now ops) still shows; off/chat +
models/studio scenes appear, the command-modes don't.
Updated test-gpu-mode-list EXPECT/GROUPS + the cockpit filter test. Repo gates
56/0; cockpit suite 673 (+1 skip) unchanged (cockpit code untouched by the
gpu-mode.sh scene removals).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Reworks the Orchestration tab's control set per maintainer direction:
- Power cap → a MENU ([c] opens PowerCapMenuScreen) instead of a stateful on/off
toggle: [d] apply default cap (230W) · [u] clear cap (uncap to hw default) ·
[m] custom wattage (type a value). Each choice routes through the standard
confirm gate. gpu-mode.sh power-cap now ALSO accepts a numeric wattage (applied
to both cards, validated against each card's [min,max]); power_cap_set(<W>)
builds it. (The menu blurs its Input on open + uses priority shortcut bindings
so d/u/m reach the screen — a focused Input otherwise swallows printable keys.)
- Prune / prune-all REMOVED from the cockpit (was Orchestration-only): binding,
action, palette entry, help/footer text, the prune() service method + its tests.
`gpu-mode prune[-all]` stays on the CLI for manual cleanup.
- Power-cap SWEEP [w] moved Orchestration → Doctor (a tuning/diagnostic bench, not
a live-estate control): context-gate → tab-doctor, action guard updated, surfaced
in the Doctor hint + help.
- Help/docstrings refreshed (Orchestration + the stale Doctor help line now lists
the real v/V/R/F verbs + the relocated [w]).
Scene start/stop path unchanged (⏎ → reconcile-gated scene_switch → gpu-mode
<mode>). +3 power-cap tests (menu→default/clear/custom, cap-sweep on Doctor),
power_cap_set custom-wattage + reject-invalid tests; per-tab check_action updated.
Suite: serve-cockpit 670 (+1 skip), tui-core 66.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Data-layer contracts the cockpit (and any jq user) consumes — all strictly
additive (existing human output byte-identical), full guard suite green (54/54):
- registry-emit.sh --json : {variants,defaults,profiles{engines,models,hardware,drafters}}
- tools/kv-calc.py --fit <slug|model> --card <gpu> --json : structured fit verdict
- gpu-mode.sh --list-modes [--json] : scene catalog (serving/studio/ops)
- estate_cli.py report-state/diagnose --json : structured estate read
- pull.sh --profile-like --dry-run --json : structured swap_path (not a message blob)
- health.sh CONTAINER= : Doctor probes any engine container (was qwen36-27b-hardcoded)
- switch.sh --explain <slug> [--json] : joined registry/engine/model/hw/drafter + fit + bench
Built + adversarially reviewed via workflow. The review caught a real
switch<->kv-calc seam defect (switch fed hyphenated 'rtx-3090', kv-calc matched
only 'rtx3090' -> fit silently 'unavailable' on the 3090 rig); fixed: kv-calc
accepts hyphenated hardware-profile ids + hyphen-strip fallback; switch surfaces
kv-calc's structured verdict regardless of RC and renders its real keys; both
tests now exercise the seam. No shared-module edits.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Groups all Club 3090 AI Studio docs under docs/ai-studio/ and fills the
gaps (no front-door overview; audio undocumented; image doc stale).
- NEW docs/ai-studio/README.md — the high-level overview: the director-driven
flow, the full 8-lane matrix, lanes-vs-modes, the shared substrate
(director/gallery/orchestrator/shim/tts/step-voice/gpu-mode), the VRAM
truth, productization framing.
- NEW docs/ai-studio/audio.md — voices (Kokoro narration), premium voice
(Step-Audio-EditX clone+edit), music (ACE-Step), SFX (Stable Audio). Notes
the future Understand pillar (ASR/diarize/separate) kept in private design.
- docs/ai-studio/image.md (was IMAGE_STUDIO.md) — REFRESHED to the current
lanes: HiDream-O1 + Ideogram-4 + Chroma + the native-button shim (the old
doc still described the retired gemma-12b image-studio mode).
- docs/ai-studio/video.md (was VIDEO_STUDIO.md) — TRIMMED to video-only;
image/audio sections moved to their own docs; cross-links added.
- Rewrote every reference across the repo (~13 files: root README, FAQ, docs
index, gpu-mode.sh, the pipe builder + studio service composes/py) to the
new paths. docs/README.md index gains the overview + audio entries.
LOCAL_AI_PRIMER.md stays at docs/ (general). The realtime voice-agent design
stays private until built.
Co-authored-by: noonghunna <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
Phase 3a of the Studio audio plan: a voiceover layer for the video lanes
(not a dedicated audio mode). Add a "voiceover: …" / 'narration: "…"' /
"say: …" directive to the message and the pipe generates a voice and mixes
it over the rendered clip.
New service services/studio/tts/ (:8192, aiohttp):
- Kokoro-82M (ONNX) on CPU — never touches the GPUs, ~1-2 s to synth a
sentence, zero VRAM pressure (better than the original "small TTS on
GPU0" plan). Runs after the render.
- Layer-aware ffmpeg mixdown, Path A: keep LTX/Sulphur's native ambient as
the bed, mix the voice over it, DUCK the bed under the voice
(sidechaincompress), loudness-normalize the master (loudnorm -16 LUFS),
cap audio to the clip length. Structured around an ordered layer list so
generated music / SFX (the future audio-studio) slot in without a rewrite
— only the voice layer is filled today. Honours the pro sound-design order
(bed -> SFX -> voice -> mix) the user flagged.
Pipe v0.9.0:
- _narration() parses the spoken line out of the message (and strips it from
the video prompt / duration parse); _narrate() POSTs /narrate after the
render (both single-clip and long-clip paths); silent-clip fallback if the
TTS service is down. Valves: enable_narration, tts_url, narrate_voice.
- gpu-mode video-studio starts studio-tts.
Two gotchas found + fixed: os.replace /tmp->/output is cross-device (use
shutil.move); sidechaincompress ends when the shorter input (the voice) runs
out, truncating the mux — apad the voice so the sidechain spans the clip,
then -t caps to the video duration.
Validated on the rig: /tts 1.7 s CPU; /narrate mixes the voice over a real
LTX-audio clip, ducked + normalized, duration-matched. Model files
(kokoro-v1.0.onnx + voices-v1.0.bin) obtained separately into
models/tts/kokoro/ — see docs/VIDEO_STUDIO.md "Integrated audio".
Co-Authored-By: Claude Opus 4.8 <[email protected]>
OWUI's native image button sends plain text to Ideogram-4, which is trained
on structured JSON captions and renders an "Image blocked by safety filter"
placeholder for off-schema input. OWUI's own image-prompt-generation can't
fix it: it returns {"prompt":"<string>"} and nesting the Ideogram JSON inside
that string defeats the task models (gemma-12b 0/3, qwen-4b 1/3 valid escapes).
Add services/studio/image-shim/ (:8191) — a transparent ComfyUI reverse-proxy.
OWUI's COMFYUI_BASE_URL points at it and OWUI image-prompt-generation is turned
off, so the raw prompt reaches the shim. The shim proxies everything through
(including the /ws progress socket) EXCEPT POST /prompt: there it reads the
plain-text prompt node, asks the director (qwen :8090) for a rich Ideogram-4
JSON caption, and rewrites the node — escaping done in Python (reliable), so the
model never produces nested JSON. Retries the director once; populated-caption
fallback if it's unusable. Blast radius = image gen only; OWUI title/tag/etc.
task-generation is untouched (this is a ComfyUI proxy, not a task-model override).
Wiring:
- gpu-mode video-studio + image-studio start the shim; image-studio also starts
the director (qwen GPU0 ~4.6GB) it depends on.
- services/openwebui/imagegen.env: COMFYUI_BASE_URL -> :8191,
ENABLE_IMAGE_PROMPT_GENERATION=false (fresh deploys; existing volumes set it in
the DB / Admin -> Images).
- docs/VIDEO_STUDIO.md "Native image button" + README image-shim row.
Validated on the rig end-to-end through the shim's /prompt: plain "a mountain
landscape at sunrise" / "a serene lake" / a logo all render clean (the director
crafts the caption); shim passthrough (/system_stats, /history, /view) + /ws relay
work; OWUI container reaches the shim at the configured base_url.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
mode_video_studio and mode_comfyui both claim to free the GPUs for
all-GPU rendering, but called only stop_all_gemma — which covers the
31B vLLM gemmas (mtp/int8), not the gemma-4-12b llama.cpp chat brain
that image-studio pins to GPU1. So an image-studio -> video-studio (or
-> comfyui) switch left gemma-4-12b resident: ~14 GB stuck on GPU1.
Measured on the rig: after `gpu-mode video-studio` GPU1 still held
13988 MiB (gemma PID unchanged across the switch). A video render's
DisTorch donor needs ~21.9 GB on GPU1 -> 14 + 21.9 = OOM. The bug only
bites the cross-mode path (image-studio is the only mode that starts
the 12b chat), which is exactly the path the unified-Studio switching
work exercises.
Fix: add stop_gemma_12b_chat to both modes, matching what
mode_diffusiongemma and mode_image_studio's siblings already do.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Wires the proven extend-chain into the chat flow. The OWUI pipe can't ffmpeg or
read the output dir, so a small host-side orchestrator service owns the chaining:
- services/studio/orchestrator/ (Dockerfile python+ffmpeg, orchestrator.py, compose):
async job API on :8190. POST /extend {prompt,lane,segments,frames} -> job_id;
GET /job/<id> -> {status,progress,filename}. Renders seg1 t2v -> extracts last
frame -> i2v segN -> ffmpeg-concat -> one mp4 in ComfyUI's output dir (gallery-served).
No GPU (ComfyUI does the work); always-on sidecar.
- Pipe: parses a duration from the user's text ('a 40-second ...'); >15s -> calls the
orchestrator, streams 'segment k/N', returns one combined video. Falls back to a
single capped clip if the orchestrator is unreachable. <=15s unchanged.
- gpu-mode video-studio now starts the orchestrator too.
- docs/VIDEO_STUDIO.md + README: orchestrator in the diagram + 'Longer videos' section.
Validated end-to-end: 2-segment job via the orchestrator API -> 20.1s combined clip,
clean join (forest-path neutral prompt). Builds on the seamless-join finding from
extend_chain. Capped at max_seconds (default 120s / 12 segments).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Brings the chat-driven video studio into the repo (was ad-hoc /tmp + manual
containers). Open WebUI -> a 'director' LLM crafts a casual prompt into a
cinematic spec -> ComfyUI renders text/image->video with audio on LTX-2.3
(stock) or Sulphur (uncensored), the 22B DiT split across both 3090s via
DisTorch.
New:
- services/studio/build_studio_pipe.py (+ generated studio_pipe.py): the OWUI
Function. Two lanes x t2v/i2v. Craft-and-go (no approval gate) + conversational
refine: it shows the crafted prompt and carries it forward so a follow-up
('make it night') evolves it. Single-stage 8-step graph (the 2-stage dev path
injects a lattice-mesh artifact on this hardware) in workflows/.
- services/studio/gallery/: always-on nginx (:8189) over ComfyUI's output dir, so
media + links survive ComfyUI being down (with HTTP Range for video seeking).
- services/studio/enhancer/: the director LLM (:8090, GPU0), OpenAI-compatible.
- scripts/gpu-mode.sh: 'video-studio' mode = ComfyUI (both GPUs) + director +
gallery + Open WebUI as one unit.
- docs/VIDEO_STUDIO.md (+ index): architecture, lanes, the measured length ceiling
(10s crisp / 15s cap / 20s collapses), VRAM split, model manifest, uncensored note.
Measured on 2x 3090: default 241 frames (~10s), hard-capped 361 (~15s) — a 20s
single pass silently corrupts. VRAM is not the limiter (weights on GPU1 ~22GB;
GPU0 compute ~14GB). The uncensored capability is in the model weights (Sulphur
DiT + director); point the pipe's chat_model valve at a stock LLM to craft via an
aligned model, the DiT still renders uncensored.
Refs #475 (gpu-mode video-studio), #477 (commit the studio). Pipe/gallery
live-validated; director crafting + frame sweep verified.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Adds `gpu-mode dgemma` (alias `diffusiongemma`): stops all other GPU models
(it's TP=2, both cards — shared 8199 slot with deckard) then launches the
DiffusionGemma compose (base.yml, PORT=8199) + wires it into Open WebUI.
Mirrors mode_deckard. Also stops it in mode_off.
Dual-card, so before switching to another model run `gpu-mode off` first
(single-card modes don't individually stop it — noted in usage). The mode
launches whatever base.yml is on the tree (the official :gemma image once
the dgemma image-adoption PR lands).
Validated live on 2x RTX 3090: `gpu-mode dgemma` stops peers, recreates the
container, starts services, owui-registers :8199, serves. bash -n clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
`gpu-mode` is a fixed mode dispatcher; there was no way to launch the
newly-promoted Deckard-40B through it. Add a `deckard` mode mirroring the
27b/gemma pattern:
- DECKARD_DIR + start_deckard/stop_deckard helpers (compose_at on the
piehsoft-q6k/mtp.yml compose; MODEL_DIR resolves from .env via --env-file,
same as the vLLM modes).
- mode_deckard: stops all other GPU models (Deckard layer-splits across both
cards), boots Deckard on :8199, brings up the support services, and wires it
into Open WebUI via the switch.sh --owui helper (best-effort; Deckard isn't
in the LiteLLM gateway config).
- stop_deckard added after every `stop_all_27b` call site (all 10 GPU-exclusive
modes) so switching AWAY from Deckard frees both cards first.
- usage() + dispatch case arm.
Live-validated: `gpu-mode deckard` boots the piehsoft-q6k project, container
healthy on :8199 serving deckard-40b, mount /mnt/models/huggingface, OWUI
connection recognized. bash -n clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Both 3090s normally run capped at 230W (quiet/cool), enforced at boot by
the enabled nvidia-power-cap.service oneshot. The cap suppresses benchmark
TPS — a documented artifact (e.g. an ik_llama "+18-20%" result that was
really just 230W vs 370W) — so maintainers need a quick way to uncap for a
true-TPS bench, then re-cap.
Adds `gpu-mode power-cap <on|off|status>` (alias: `powercap`):
- on: restart nvidia-power-cap.service (reusing the service as the single
source of truth for 230W). restart, NOT start: the unit is already
active from boot, and `start` on a RemainAfterExit oneshot is a no-op
that would silently fail to re-apply. Falls back to direct
`nvidia-smi -pl 230` if the service is missing.
- off: read each card's Default Power Limit from nvidia-smi and apply it.
Defaults differ per card here (GPU0 370W, GPU1 420W), so the value is
read, never hardcoded; nvidia-smi has no reset flag. Session-scoped:
a reboot or driver reload re-caps via the service (never disabled).
- status: print per-GPU enforced/default/min/max limits.
Also surfaces a one-line per-card cap state in `gpu-mode status` and echoes
the resulting enforced limit after on/off. Errors loudly (no nvidia-smi, no
sudo) in the script's existing style.
Live-tested on the rig: status -> off (370/420) -> status -> on (230/230),
verified each transition with `nvidia-smi -q -d POWER`; left at 230W.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Match gpu-mode.sh to the #279 dual prune: remove the 27b-turbo/27b-dflash/27b-dflash-noviz modes (they launched deprecated composes; defensive stop_ helpers kept), and repoint Gemma so gemma -> vllm/gemma-int8 default (:8032, 262K+vision), new gemma-mtp -> bf16 32K fallback (:8030), gemma-int8 kept as alias. bash -n clean; suite 38/38.
Every gemma-4-31b vLLM compose pinned a purged Docker Hub nightly
(e47c98ef / bf610c2f), un-bootable for fresh users (#250, #167 class).
Prune the 9-variant set to 3 and repoint survivors onto the immutable
stable release tag vllm/vllm-openai:v0.21.0 (never :latest).
Survivors: vllm/gemma-mtp (bf16 dual, default), vllm/gemma-int8 (int8-PTH
dual), vllm/gemma-mtp-tp1 (fp8 single). Dropped: gemma-dflash,
gemma-dflash-int8, gemma-int8-tq3, gemma-bf16, gemma-awq; the 262K path
folds into gemma-int8 via a CTX env override. Gemma is decoupled from the
shared Qwen nightly profiles via a new vllm-gemma-stable engine profile;
Qwen nightly profiles are untouched.
Diagnose decoupling: the pruned gemma dflash patch dirs doubled as the
diagnose tool's cross-model disk-source proxies for two Qwen overlays
(vllm-pr41703-dflash, vllm-pr42102-dflash-kv-quant). Those capabilities
are image-baked in the pinned nightly, not mountable files, so mark them
image_baked and have diagnose skip the disk-source check for image-baked
overlays. Runtime-neutral (no Qwen compose mounts them).
switch.sh launcher: handle the new VLLM_IMAGE engine-pin export (the twin
loop in launch.sh had it; switch.sh did not -> "unexpected engine pin
export"), and guard the VLLM_NIGHTLY_SHA echo for the image-only stable
profile (was unbound under set -u).
Validated on 2x RTX 3090 (Ampere, v0.21.0): full scripts/tests/*.sh suite
35/0; live boots of gemma-mtp (bf16) and gemma-int8 serve coherent output,
qwen vllm/dual regression-clean. gemma-mtp-tp1 (fp8_e4m3) is correctly
SM-gated (required_sm=9.0) -- confirmed Ampere-incompatible (Triton
"fp8e4nv not supported on sm_86"), a documented 32 GB+/Hopper variant;
beellama remains the single-card gemma path on Ampere.
Refs #451, #250, #167.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
PR-A (#231) inserted the <quant>/ layer; two compose-path references
outside the doc sweep were left stale:
- scripts/gpu-mode.sh (the rig mode-switcher, symlinked into
/usr/local/bin) cd'd into the dual/ topology dir and -f'd bare
filenames (turbo.yml, docker-compose.yml, ...) that PR-A moved under
<quant>/. Repointed DUAL_27B_DIR / GEMMA_DUAL_DIR at the autoround-int4
quant dir (mount-safe — same cd-into-compose-dir invocation switch.sh
uses), added GEMMA_DUAL_AWQ_DIR for the awq mode, and renamed the two
ex-docker-compose.yml defaults (qwen -> fp8-mtp.yml, gemma -> bf16-mtp.yml).
All 13 mode targets verified to resolve; bash -n clean.
- vllm-pr40798 / vllm-pr40914 patch READMEs cited dual/int8-tq3.yml (the
pre-PR-A name). Commit 6182922 split int8-tq3 into the tq3-* family; the
READMEs describe "TQ3 KV + MTP n=3" = tq3-mtp.yml (whose tombstone header
cites #40914) -> dual/autoround-int4/tq3-mtp.yml.
Surfaced (not fixed here — PR-B's registry-derivation eliminates it):
launch.sh LAUNCH_VARIANT_COMPOSE[vllm/gemma-mtp] points at a non-existent
dual/autoround-int4/fp8-mtp.yml; gemma dual default is bf16-mtp.yml (the
registry is correct). test-launch-compat doesn't sweep launch.sh paths.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
All llama.cpp + ik single composes shared one default container_name, so a
second variant hit a docker "name in use" conflict (ESTATE_CONTAINER override
existed but was undocumented). Give each non-base variant a distinct default:
- llamacpp/mtp → llama-cpp-qwen36-27b (base, unchanged)
- llamacpp/mtp-vision → llama-cpp-qwen36-27b-vision
- ik/iq4ks-mtp → ik-llama-qwen36-27b (base, unchanged)
- ik/iq4ks-mtp-vision → ik-llama-qwen36-27b-vision
- ik/iq4ks-two-stage → ik-llama-qwen36-27b-two-stage
The base names (mtp / iq4ks-mtp = the default workhorses) are kept so existing
hardcoded refs and benchlocal `docker logs <base>` keep working.
launch.sh: track the two renamed variants in LAUNCH_DEFAULT_CONTAINER, and add
the missing ik-llama/iq4ks-two-stage entry (it fell back to the wrong default
container before). gpu-mode.sh: comment-only update for the new naming.
Audit (all clear): the autodetect globs in preflight/soak/switch/health/update
are prefix-anchored (`^llama-cpp-qwen36-27b`, no end anchor) and gpu-mode uses
`== llama-cpp-*`, so the suffixed names still match — verified.
Ports still default to 8020 (concurrency uses ESTATE_PORT, per the issue's own
example) — documented with a "run two variants at once" snippet in SINGLE_CARD.
Validated: `docker compose config` resolves each distinct name; all names still
match the autodetect glob; bash -n + test-launch-compat pass.
Closes#169.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Post-v0.8.3 the llama.cpp profile family lands containers on :8020
(llamacpp/default/mtp) and :8030 (llamacpp/mtp-vision). The status block was
missing :8020 entirely and hardcoded :8030 as "gemma-mtp / Gemma 4 31B".
Add :8020 probe, and on :8030 detect the active container so the label tracks
whichever engine (vLLM Gemma vs llama.cpp Qwen) is actually serving.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Before this fix, gpu-mode off only stopped services it knew about
(stop_all_27b vLLM variants, stop_all_gemma vLLM variants, stop_comfyui,
and the SERVICES array). Instances booted via launch.sh --estate or
--estate-file persist via Docker `restart: unless-stopped` and so kept
running after gpu-mode off — appearing to "auto-restart". The user had
to know about launch.sh --down-estate to clean them up.
New stop_estate function:
- Reads ~/.club3090/estate.yml (constant ESTATE_YAML)
- No-ops if file missing, launch.sh unavailable, or estate list empty
- Otherwise calls launch.sh --down-estate <yaml> which cleanly removes
each estate-managed container + its docker network
Called from mode_off only — keeps blast radius small. Other mode
switchers (mode_27b, mode_gemma, etc.) aren't touched on purpose:
estate instances and gpu-mode classic services can in principle share
a rig at TP=1 + small contexts, so the user should choose explicitly
whether to keep them up when switching gpu-mode targets. mode_off is
the "stop everything" intent — clearly should include estate.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
The dev rig had grown across three home dirs (`/opt/ai/compose/`, `/opt/ai/github/`,
`/home/wasif/`) and three repos (single-3090, dual-3090, club-3090). Disk-out
on `/` (97% used) forced a cleanup; rather than just prune, we consolidated
the layout across the whole stack while we were at it. This commit captures
what landed inside this repo.
Services consolidation:
- services/{ollama,openwebui,litellm,qdrant,searxng}/ migrated in from
/opt/ai/compose/<svc>/ (zero functional change — same docker-compose.yml).
- services/litellm/config.yaml rewritten: explicit routes for current
primaries (qwen3.6-27b-autoround → :8010, gemma-4-31b-autoround → :8030).
Removed `* → ollama/*` wildcard.
- services/comfyui/ migrated in (was /opt/ai/compose/comfyui/) — wired into
gpu-mode with full mutex against vLLM/SGLang.
scripts/gpu-mode.sh under git:
- Was a loose /opt/ai/gpu-mode.sh outside any repo. Symlinked at
/usr/local/bin/gpu-mode.
- Five Gemma 4 31B modes added: gemma, gemma-dflash, gemma-int8,
gemma-dflash-int8, gemma-awq.
- One ComfyUI mode (mutex with all LLM serving).
- prune / prune-all subcommands (safe image prune; aggressive variant adds
build cache --keep-storage 5GB + dangling networks).
- gpu-mode status now shows Docker disk + /var/lib/docker + /tmp sizes.
- compose_at() passes --env-file <repo>/.env so MODEL_DIR resolves
regardless of which compose dir gpu-mode cd's into. Fixes the recurring
"MODEL_DIR not set, defaulting to ../../../../../models-cache" warning.
- stderr no longer swallowed by compose_at() (real errors surface).
- Cross-model VRAM mutex: every Qwen mode stop_all_gemma + stop_comfyui
and vice-versa.
scripts/maintenance/ — new hygiene-tools subdir:
- list-image-pins.sh: engine-agnostic pin auditor. Scans every compose's
`image:` line, groups by `<repo>:<tag>`, flags pin-drift (multiple tags
per repo), ranks composes by patch surface.
Pin tracking:
- docs/UPSTREAM.md gains a "Pinned images" section: table of every pinned
image, why each pin exists, retirement candidate criteria.
- docs/NIGHTLY_BUMP_RUNBOOK.md (new): 7-step procedure for bumping pinned
engine images (scope → branch → patch survival → boot → verify-full +
verify-stress → bench delta → land → retire). Engine-specific notes
for vLLM nightly hashes, llama.cpp digest pinning, SGLang variants.
Path updates from the engine + model dir consolidation:
- /opt/ai/vllm-src/ → /opt/ai/engines/vllm/primary/
(in setup.sh, INTERNALS.md, several patch READMEs, docs/HARDWARE.md,
docs/FAQ.md, docs/DUAL_CARD.md, docs/UPSTREAM.md, models/qwen3.6-27b/
CHANGELOG.md)
- /mnt/models/gguf/qwen3.6-27b/ → /mnt/models/huggingface/qwen3.6-27b-gguf/
(in models/qwen3.6-27b/llama-cpp/{compose/single/*.yml, recipes/*.sh,
README.md}, docs/engines/LLAMA_CPP.md)
CHANGELOG.md narrative gap fill:
- 2026-05-10 entry for this reorg.
- 2026-05-09 entry for compose convention formalization (topology
promoted to dir level, profile schema, Status enum + Caveats, cliff
CI swap, Discord launch).
- 2026-05-08 entry for Gemma 4 INT8 PTH unblock + 262K validation.
- 2026-05-07 entry for power-cap-sweep campaign + HARDWARE.md cross-rig
charts + cross-rig benchmark rows.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
