noonghunnaandClaude Opus 4.8
13c27d114a
Adds a third lane to the unified Studio pipe — `🖼️ Studio · Image` — rendering stills on Ideogram-4 fp8 via the same qwen director, gallery, and SPEC-refine machinery as the video lanes. Single-device GPU0 (~18.5 GB @1024²), so it runs in EITHER gpu-mode with no switch: in video-studio the 22B DiT weights sit on GPU1 (DisTorch donor), leaving GPU0 for the image + the ~4.6 GB director (≈23 GB @1024², fits). Capped at image_max_edge=1024 (2048² + director = OOM). Root cause solved along the way: Ideogram-4 is trained on STRUCTURED JSON captions and denoises off-schema plain text to a gray "Image blocked by safety filter" placeholder — an in-weights fallback, not a ComfyUI filter (grepped: none; the canonical template note confirms). Measured: plain "a red apple" → 100% blocked (4/4 neutral prompts); the same apple as a JSON caption → clean photoreal render (~80 s warm). So the image director (DIRECTOR_IMG_SYS) is category-aware (logo / poster / UI / photo / illustration) and emits the JSON caption, not prose; _coerce_caption() parses it (stripping ``` fences) and falls back to wrapping plain text in a minimal caption so off-schema input never reaches the model. qwen-4b produced valid JSON first-try for "Ember logo" + "headphone product photo" → both rendered clean. This also exposes a latent bug in Open WebUI's native 🖼️ image button: it templates plain text into imagegen.env's workflow, so it hits the same placeholder. Documented in VIDEO_STUDIO.md as a follow-up (needs a JSON-wrap step); the Studio · Image lane is the working path meanwhile. Validated live: pipe installed to the OWUI function table, OWUI healthy, function loads clean. Gate 41/42 (the 1 red is the pre-existing nex-n2-mini disk pollution, unrelated to services/studio). Co-Authored-By: Claude Opus 4.8 <[email protected]>
17 lines
1.8 KiB
JSON
17 lines
1.8 KiB
JSON
{
|
|
"unet_main": {"class_type": "UNETLoader", "inputs": {"unet_name": "ideogram4_fp8_scaled.safetensors", "weight_dtype": "default"}},
|
|
"unet_uncond": {"class_type": "UNETLoader", "inputs": {"unet_name": "ideogram4_unconditional_fp8_scaled.safetensors", "weight_dtype": "default"}},
|
|
"clip": {"class_type": "CLIPLoader", "inputs": {"clip_name": "qwen3vl_8b_fp8_scaled.safetensors", "type": "ideogram4"}},
|
|
"vae": {"class_type": "VAELoader", "inputs": {"vae_name": "flux2-vae.safetensors"}},
|
|
"pos": {"class_type": "CLIPTextEncode", "inputs": {"text": "placeholder", "clip": ["clip", 0]}},
|
|
"neg": {"class_type": "ConditioningZeroOut", "inputs": {"conditioning": ["pos", 0]}},
|
|
"guider": {"class_type": "DualModelGuider", "inputs": {"model": ["unet_main", 0], "positive": ["pos", 0], "cfg": 3.5, "model_negative": ["unet_uncond", 0], "negative": ["neg", 0]}},
|
|
"sigmas": {"class_type": "Ideogram4Scheduler", "inputs": {"steps": 20, "width": 1024, "height": 1024, "mu": 0.5, "std": 1.75}},
|
|
"sampler": {"class_type": "KSamplerSelect", "inputs": {"sampler_name": "euler"}},
|
|
"noise": {"class_type": "RandomNoise", "inputs": {"noise_seed": 42}},
|
|
"latent": {"class_type": "EmptyFlux2LatentImage", "inputs": {"width": 1024, "height": 1024, "batch_size": 1}},
|
|
"samp": {"class_type": "SamplerCustomAdvanced", "inputs": {"noise": ["noise", 0], "guider": ["guider", 0], "sampler": ["sampler", 0], "sigmas": ["sigmas", 0], "latent_image": ["latent", 0]}},
|
|
"decode": {"class_type": "VAEDecode", "inputs": {"samples": ["samp", 0], "vae": ["vae", 0]}},
|
|
"save": {"class_type": "SaveImage", "inputs": {"images": ["decode", 0], "filename_prefix": "studio_image"}}
|
|
}
|