master
2 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
noonghunna
|
47ac38385a |
Director placement lever: CPU / GPU0 / GPU1 (c3 Settings) (#473)
* Director placement lever: env-driven CPU / GPU0 / GPU1 (backend)
The studio director's GPU/CPU placement is now a single lever,
STUDIO_DIRECTOR_DEVICE (gpu0 | gpu1 | cpu, default gpu0), read from the rig
.env by gpu-mode's start_studio_director and translated into the compose
env (-ngl + CUDA_VISIBLE_DEVICES + device_ids):
- gpu0 (default): -ngl 99, GPU0 — fast craft (~50-100 tok/s), ~4.6 GB,
coexists with the image lanes. Unchanged from before.
- gpu1: -ngl 99, GPU1 — only when GPU1 has room (NOT during a video render;
GPU1 is the DisTorch DiT donor).
- cpu: -ngl 0, CUDA_VISIBLE_DEVICES="" — frees ~4.6 GB off GPU0 (lifts the
single-card Wan window 121→161 frames) at ~single-digit tok/s craft.
Compose now reads ${DIRECTOR_NGL:-99} + ${STUDIO_DIRECTOR_CUDA-0} (no-colon
so an explicit empty value = CPU survives). Default (no override) preserves
current GPU0 behaviour exactly.
Live-validated: CPU mode starts with GPU0 full (gemma12b), adds 0 MiB VRAM
to GPU0, serves on :8090, generates (~5 tok/s CPU). The c3 Settings field
that writes STUDIO_DIRECTOR_DEVICE follows in the next commit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
* c3 Settings: Director placement (CPU / GPU0 / GPU1) + CPU thread cap
Exposes the director-placement lever in the c3 Settings modal so users pick
GPU0 (fast, default) / GPU1 / CPU without hand-editing the .env.
- services.py: director_device() reads STUDIO_DIRECTOR_DEVICE from the repo
.env (default gpu0, validates the value); set_repo_env_var() upserts a key
in place (preserves other lines, no duplicates, creates the file if absent).
- app.py: SettingsScreen gains a "Director placement" Select; apply_settings
persists the choice to the repo .env (the SHARED config gpu-mode reads —
distinct from c3-settings.json for MODEL_DIR/HF_TOKEN). Applies on the next
ai-studio start.
- compose: CPU thread cap — -t ${DIRECTOR_THREADS:-8} bounds CPU use so the
director doesn't starve OWUI's embedder/reranker (also CPU). The ~2.6 GB
GGUF loads into system RAM (mmap'd; resident in page cache, not run from SSD).
- tests: +6 data-layer (TestDirectorPlacement) + 1 headless apply-settings
round-trip (persists STUDIO_DIRECTOR_DEVICE, idempotent re-apply). Full
suite green (728), settings/director subset 13/13.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
* tests: fix stale scene names in gpu-mode-list EXPECT
The scene catalog renamed its dispatch keywords to qwen27b / gemma-31b,
but the test's EXPECT spot-check map still referenced the old 27b / gemma
short names — so the JSON-shape assertion had been red on master. Point
EXPECT at the canonical names the catalog now emits.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
* studio: run director in chat scene as the catalog-support layer
Bring the uncensored director up in the `chat` scene (honoring the
STUDIO_DIRECTOR_DEVICE placement knob), framing chat as the supporting-
infra home — OWUI + LiteLLM + Qdrant + SearXNG + director — for ad-hoc
Catalog models launched via `switch.sh --owui`.
A CPU-placed director uses no GPU, so it's the always-on path: it survives
scene switches and stays live in OWUI. New _director_evict_if_gpu helper
frees only a GPU-resident director when a dual-card LLM scene claims the
cards; mode_off stops it outright. Also brings mode_gemma_int8 in line with
its dual-card siblings (it was missing the studio teardown entirely).
Docs: requirements.md gains a "Chat scene — the Catalog-support layer"
section + reframes director placement around the unified knob / c3 Setting;
video.md note synced.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
* studio: disable director thinking on CPU only (latency)
The uncensored director is an "Aggressive" reasoning fine-tune that emits
a full <think> trace before every answer. On GPU that's cheap and the
trace lands in reasoning_content (content stays clean), so leave it on.
On CPU (~14 tok/s) the trace dominates latency, so gpu-mode now passes
`--jinja --reasoning off` for the cpu placement only — forcing the
template's enable_thinking=false (this fine-tune ignores /no_think and
--reasoning-budget 0, but honors --reasoning off).
Wired via a new DIRECTOR_THINK_ARGS compose param (empty on GPU). Live:
CPU director now answers in one pass, no reasoning trace, craft quality
intact (full cinematic spec, finish=stop).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
* c3: Containers-pane director start honors placement + enumerate sidecars
Two consistency fixes for the Containers tab:
1. Starting studio-director from Containers now honors STUDIO_DIRECTOR_DEVICE
instead of falling back to the GPU0/thinking-on compose default. New
director_compose_env() mirrors gpu-mode's start_studio_director translation
(NGL/CUDA/GPU/THINK_ARGS), injected as an `env K=V …` prefix on the compose
up cmd (process env wins over --env-file). cpu → -ngl 0 + --reasoning off.
2. The nested studio sidecars (director/gallery/orchestrator/image-shim/
step-voice/tts) now enumerate when STOPPED, so they're startable rows — not
only visible while running. New STUDIO_SIDECARS map is the single SoT for
resolving the container-name → services/studio/<sub>/ project (fixing the
director↔enhancer name mismatch that previously returned None → docker
restart, which fails on a fresh install).
+8 tests (director_compose_env cpu/gpu, director resolves to enhancer with the
env prefix, sidecar enumeration). Live: c3 service_start plan starts the director
CPU + no-think (argv -ngl 0 --reasoning off, GPU0 free, clean generation).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
---------
Co-authored-by: noonghunna <10742901+noonghunna@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
noonghunna
|
11984d19da |
docs: AI Studio requirements + Wan tuning + director placement
- requirements.md (new): hardware / software / disk / RAM to self-host AI Studio, expressed generically (2× 24 GB Ampere+, PCIe ok / no NVLink; ~120 GB disk; 32 GB+ RAM), single-vs-dual-card, and the director-placement VRAM lever. Linked from README's deep-dive index. - video.md: a "Wan2.2 — tuning & limits" section (recipe + the sampler finding, 480p default / 720p-DisTorch valve, the corrected length-ceiling table, i2v-seeded chaining) + Wan model files in the manifest. The ceiling table now shows the director as the single-card swing factor (121↔161) and adds the DisTorch both-card column. - Director placement documented as a lever (default GPU0; STUDIO_DIRECTOR_GPU / -ngl 0 relocate it), with the caveat that GPU1 is unsafe for the LTX lanes (their 22 GB donor) — CPU is universal. - The director carries its model name (Qwen3.5-4B-Uncensored) in the services manifest, matching the lane-naming convention. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm |