3 people dbfbd598e7 feat(c3): ② Serve override editor — retune served-name/ctx/KV/spec/util (#636) ...

Phase 2 of the ② Serve rework (Phase 1 = #635 dead-end fix): a Route-C brought
model can be re-tuned before serve, without hand-editing a compose. Fields
(pre-filled from the resolved slug's defaults, dropdowns to avoid typos):
  • served name  → SERVED_NAME     (Input)
  • ctx          → MAX_MODEL_LEN   (Select — presets + the slug's own default)
  • KV cache     → KV_CACHE_DTYPE  (Select — fp8_e5m2/e4m3/turboquant_4bit_nc/…)
  • spec-decode  → SPEC on/off     (Select)
  • VRAM util    → GPU_MEMORY_UTILIZATION (Select)

Mechanism (all five ride env — no per-serve compose rewrite):
- data.py: ActionPlan gains `env`; dispatch merges {**os.environ, **plan.env}.
- services.py: serve_generated(compose_path, overrides) → plan.env (+ pins
  MODEL_DIR); serve_override_defaults() parses the sibling compose's
  ${VAR:-default} for pre-fill (stdlib regex, no PyYAML).
- swap_apply.py: emit parameterizes --served-model-name ${SERVED_NAME:-…} and
  gates --speculative-config behind ${SPEC:-on} via the SAME entrypoint the
  shipped nvfp4 compose uses — so SPEC=off drops the MTP drafter at up-time.
- app.py: LaneServePane fields, revealed + pre-filled only for a Route-C armed
  model (hidden otherwise); collect_overrides() → serve_generated. Loose
  validation (dropdowns constrain; the reconcile gate + boot are the real check).

Validated: emitted compose `docker compose config` resolves SERVED_NAME=X and
SPEC=off (entrypoint gate) cleanly; 169 byo/serve/override tests pass.


Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm

Co-authored-by: noonghunna <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>

2026-07-08 08:49:51 +05:00

..

calibration

Wire gemma-4-26b-a4b AWQ on stock vLLM v0.22.0; retire AutoRound (#326)

2026-06-06 00:46:13 +00:00

drafters

Add Carnice-V2-27B beellama single-card compose (beellama/carnice-v2-single-q5km-mtp) (#406)

2026-06-14 17:04:47 +05:00

engines

feat(qwen): NVFP4 community-validated slugs for Hopper/Blackwell (2 composes)

2026-07-06 21:32:15 +00:00

hardware

Phase 2: PRO 6000 + DGX Spark envelopes, fix Blackwell detector

2026-07-05 06:16:59 +00:00

models

docs(nvfp4): correct the KV-scales story — declared, NOT shipped, both models

2026-07-06 22:44:09 +00:00

workloads

…

__init__.py

…

arch_patches.yml

patches: de-register vllm-marlin-pad (merged upstream, native in v0.24.0)

2026-07-02 10:03:44 +00:00

baselines.yml

Re-derate 27B nvfp4 single default: SPEC=on + 65K (keep MTP's 2x)

2026-07-08 00:35:04 +00:00

booter.py

v0.8.0 [F] F3: G6-A 3-part additive [E] touch (pt1.predicted_b_breakdown, pt3.failure_log_excerpt+actual, container-log capture) + §6.1 Tier-1 (CONTRACT-2)

2026-05-17 04:19:53 +00:00

calibration_seed.yml

…

canonical_scenarios.py

…

capture.py

feat(pull): v0.8.2 STEP V1 — capture-on-hard-block pt1-gate emitter + BaseCaptureBundle protocol lift

2026-05-18 16:43:28 +00:00

classifier.py

feat(pull): v0.8.2 STEP V1 — capture-on-hard-block pt1-gate emitter + BaseCaptureBundle protocol lift

2026-05-18 16:43:28 +00:00

compat.py

feat(qwen): flip multi-max to fp8/e4m3 KV + weights-conditional Ampere compat

2026-07-06 14:55:01 +00:00

compose_registry.py

Re-derate 27B nvfp4 single default: SPEC=on + 65K (keep MTP's 2x)

2026-07-08 00:35:04 +00:00

dedup.py

feat(pull): v0.8.2 STEP V1 — capture-on-hard-block pt1-gate emitter + BaseCaptureBundle protocol lift

2026-05-18 16:43:28 +00:00

deriver.py

Detect MTP head in BYO Route-C swap instead of blanket spec-drop

2026-07-07 22:38:19 +00:00

diagnose_profile_cli.py

dgemma: adopt official vllm/vllm-openai:gemma image (drop 120-file sideload)

2026-06-11 05:31:36 +00:00

downloader.py

…

einput.py

…

envelopes.yml

Phase 2: PRO 6000 + DGX Spark envelopes, fix Blackwell detector

2026-07-05 06:16:59 +00:00

estate_cli.py

refactor(pods): rename cluster → pod (#610) + heterogeneous-rig guidance

2026-07-07 02:10:17 +00:00

failure_fingerprints.yml

feat(pull): v0.8.2 STEP V1 — capture-on-hard-block pt1-gate emitter + BaseCaptureBundle protocol lift

2026-05-18 16:43:28 +00:00

gates.py

Gate nvfp4 KV to datacenter Blackwell only (sm_100/103) — found on #571

2026-07-05 03:23:49 +00:00

hwdetect.py

feat(pull): v0.8.2 STEP V4 — optional whichllm hw-detect subprocess (CONTRACT-3, hw-detect-only)

2026-05-18 19:29:43 +00:00

launch_compat.py

fix(nvfp4): disable deepgemm on consumer cards

2026-07-07 22:05:09 +00:00

loop_input.py

feat(pull): v0.8.2 STEP V1 — capture-on-hard-block pt1-gate emitter + BaseCaptureBundle protocol lift

2026-05-18 16:43:28 +00:00

measurement_record.py

catalog-baselines slice 2c: canonical two-depth prefill/TTFT probe + anchor calibration

2026-07-04 16:22:48 +00:00

patch_attribution.py

feat(pull): v0.8.2 STEP V3 — arch-registry expansion + chat-template attribution/drift_guard

2026-05-18 18:29:58 +00:00

patches.yml

patches: de-register vllm-marlin-pad (merged upstream, native in v0.24.0)

2026-07-02 10:03:44 +00:00

profile_runtime.yml

Pull gate: actionable messages for uncurated derives of curated models (#424)

2026-06-18 01:10:01 +05:00

pull.py

feat(byo): apply-swap — download + serve a Route-C fine-tune (#628)

2026-07-08 05:50:16 +05:00

submit_pull.py

fix(pull): v0.8.2 STEP V2 — gh-less issue body must not carry the absolute capture path

2026-05-18 17:49:00 +00:00

swap_apply.py

feat(c3): ② Serve override editor — retune served-name/ctx/KV/spec/util (#636)

2026-07-08 08:49:51 +05:00

trust_pipeline.py

v0.8.0 [review] pre-tag fixes: scrub internal-path leaks from shipped source + make .pull-captures-corpus tests CI-safe (skip-when-absent)

2026-05-17 12:01:25 +00:00

weights.py

feat(qwen-moe): NVFP4 35B-A3B community slugs — the unified-memory pairing

2026-07-06 22:10:21 +00:00