noonghunnaandClaude Opus 4.8
19df532441
Previously a mixed-card rig no-op'd the concurrency injection ("no single
card-class row applies"). But it does have a right answer: vLLM already sizes
the KV pool to min(free blocks) across TP ranks (the cache is symmetric-
sharded), so the smallest card dictates the pool. Clamping the envelope lookup
to the smallest-VRAM card therefore MIRRORS the engine — it's the real ceiling,
not a conservative guess — and stays safe for TP=1 too (the ceiling fits
whichever single card vLLM lands on, all >= the smallest).
Behaviour:
5090 + 3090 -> smallest 3090 has no row -> compose default (unchanged)
5090 + H100 -> smallest 5090 is seeded -> inject its ceiling (was: no-op)
2x 5090 -> homogeneous -> unchanged
The common 5090+3090 case still lands on the compose default (now for the
principled reason: the 24 GB card caps the pool), so nothing regresses; the
gain is heterogeneous rigs whose SMALLEST card is itself a seeded >24 GB class.
Fixtures lock both the new inject-on-smallest and the still-no-op paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EfF565T9eSLaqGzidyJ1Pm
129 lines
5.7 KiB
Bash
129 lines
5.7 KiB
Bash
#!/usr/bin/env bash
|
|
# test-envelopes — guards for scripts/lib/profiles/envelopes.yml + its launcher
|
|
# injection (#246 Phase 2, concurrency-only first pass).
|
|
#
|
|
# REDs on: schema violations · unknown slugs · unknown card-class ids · a row
|
|
# with neither a `validated` (soak) nor a `computed` (kv-calc) provenance block,
|
|
# or a `computed` block missing its `basis` (born-from-a-basis discipline) · a
|
|
# broken injection contract. An EMPTY envelopes file is valid.
|
|
set -euo pipefail
|
|
|
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
cd "$ROOT_DIR"
|
|
HELPER="scripts/lib/profiles/launch_compat.py"
|
|
fail() { echo "FAIL: $1" >&2; exit 1; }
|
|
|
|
# --- 1. schema (pure python) --------------------------------------------------
|
|
python3 - <<'PY'
|
|
import sys
|
|
from pathlib import Path
|
|
import yaml
|
|
sys.path.insert(0, ".")
|
|
from scripts.lib.profiles.compose_registry import COMPOSE_REGISTRY # noqa: E402
|
|
from scripts.lib.profiles.compat import load_profiles # noqa: E402
|
|
|
|
doc = yaml.safe_load(Path("scripts/lib/profiles/envelopes.yml").read_text())
|
|
assert doc.get("schema_version") == 1, "schema_version must be 1"
|
|
rows = doc.get("envelopes") or {}
|
|
cards = set(load_profiles().hardware)
|
|
errors = []
|
|
for slug, byc in rows.items():
|
|
if slug not in COMPOSE_REGISTRY:
|
|
errors.append(f"envelopes[{slug}]: unknown registry slug"); continue
|
|
if not isinstance(byc, dict):
|
|
errors.append(f"envelopes[{slug}]: must be a card-class map"); continue
|
|
for card, row in byc.items():
|
|
w = f"envelopes[{slug}][{card}]"
|
|
if card not in cards:
|
|
errors.append(f"{w}: unknown hardware-profile id"); continue
|
|
if not isinstance(row.get("max_num_seqs"), int):
|
|
errors.append(f"{w}.max_num_seqs: required int")
|
|
if "compose_default" in row and not isinstance(row["compose_default"], int):
|
|
errors.append(f"{w}.compose_default: int")
|
|
# born-from-a-basis: a value MUST cite a `validated` soak OR a `computed`
|
|
# kv-calc ceiling — never a bare guess. (concurrency is a capacity
|
|
# question kv-calc answers deterministically; see envelopes.yml header.)
|
|
validated = row.get("validated")
|
|
computed = row.get("computed")
|
|
if not (isinstance(validated, dict) and validated) and \
|
|
not (isinstance(computed, dict) and computed):
|
|
errors.append(f"{w}: requires a `validated` (soak) OR `computed` "
|
|
"(kv-calc) provenance block — no guessed rows")
|
|
# a computed row must name its kv-calc basis (the invocation + boundary)
|
|
if isinstance(computed, dict) and computed and not computed.get("basis"):
|
|
errors.append(f"{w}.computed.basis: required — cite the kv-calc "
|
|
"invocation + PASS/cap boundary")
|
|
if errors:
|
|
print("test-envelopes: FAIL", file=sys.stderr)
|
|
for e in errors: print(f" ✗ {e}", file=sys.stderr)
|
|
sys.exit(1)
|
|
print(f" ✓ schema ({len(rows)} slug rows; empty is valid)")
|
|
PY
|
|
|
|
# --- 2. injection contract (fixture: temp envelopes with a 5090 row) ----------
|
|
TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
|
|
REAL="scripts/lib/profiles/envelopes.yml"
|
|
BACKUP="$TMP/backup.yml"; cp "$REAL" "$BACKUP"
|
|
REAL_SUM="$(sha256sum "$REAL" | cut -d' ' -f1)"
|
|
|
|
cat > "$REAL" <<'EOF'
|
|
schema_version: 1
|
|
envelopes:
|
|
vllm/dual:
|
|
rtx-5090:
|
|
max_num_seqs: 4
|
|
compose_default: 2
|
|
validated: { concurrency_soak: "4 @262K, 0-growth" }
|
|
EOF
|
|
|
|
pin() { python3 "$HELPER" resolve-variant-pin --variant "$1" --format shell --gpu-spec "$2" 2>/dev/null; }
|
|
S5090="0|RTX 5090|32607|12.0;1|RTX 5090|32607|12.0"
|
|
S3090="0|RTX 3090|24576|8.6;1|RTX 3090|24576|8.6"
|
|
HET_SMALL="0|RTX 5090|32607|12.0;1|RTX 3090|24576|8.6" # smallest = 3090 (no row)
|
|
HET_BIG="0|RTX 5090|32607|12.0;1|NVIDIA H100|81920|9.0" # smallest = 5090 (seeded)
|
|
|
|
grep -q "MAX_NUM_SEQS=4" <(pin vllm/dual "$S5090") || fail "5090 with a row must inject MAX_NUM_SEQS=4"
|
|
grep -q "MAX_NUM_SEQS" <(pin vllm/dual "$S3090") && fail "3090 (no row) must NOT inject"
|
|
# heterogeneous clamps to the SMALLEST-VRAM card (= the pool vLLM allocates):
|
|
# 5090+3090 -> smallest 3090 has no row -> compose default (no inject)
|
|
# 5090+H100 -> smallest 5090 is seeded -> inject its ceiling (4)
|
|
grep -q "MAX_NUM_SEQS" <(pin vllm/dual "$HET_SMALL") && fail "het rig w/ smallest=3090 (no row) must NOT inject"
|
|
grep -q "MAX_NUM_SEQS=4" <(pin vllm/dual "$HET_BIG") || fail "het rig must clamp to smallest-VRAM card (5090) and inject its ceiling"
|
|
|
|
# injection is provenance-agnostic: a `computed` row injects exactly like a
|
|
# `validated` one (the guard cares about provenance; the launcher does not).
|
|
cat > "$REAL" <<'EOF'
|
|
schema_version: 1
|
|
envelopes:
|
|
vllm/dual:
|
|
rtx-5090:
|
|
max_num_seqs: 4
|
|
compose_default: 2
|
|
computed: { basis: "kv-calc N=4 PASS, N=5 caps", target_ctx: 262144 }
|
|
EOF
|
|
grep -q "MAX_NUM_SEQS=4" <(pin vllm/dual "$S5090") || fail "computed row must inject like a validated one"
|
|
cat > "$REAL" <<'EOF'
|
|
schema_version: 1
|
|
envelopes:
|
|
vllm/dual:
|
|
rtx-5090:
|
|
max_num_seqs: 4
|
|
compose_default: 2
|
|
validated: { concurrency_soak: "4 @262K, 0-growth" }
|
|
EOF
|
|
grep -q "MAX_NUM_SEQS" <(MAX_NUM_SEQS=9 pin vllm/dual "$S5090" | grep "MAX_NUM_SEQS=4") && fail "user env must win"
|
|
# value at/below compose_default must not fire
|
|
cat > "$REAL" <<'EOF'
|
|
schema_version: 1
|
|
envelopes:
|
|
vllm/dual:
|
|
rtx-5090: { max_num_seqs: 2, compose_default: 2, validated: { concurrency_soak: "x" } }
|
|
EOF
|
|
grep -q "MAX_NUM_SEQS" <(pin vllm/dual "$S5090") && fail "value == compose_default must NOT inject (no gain)"
|
|
echo " ✓ injection contract (inject · no-row · het-clamp-to-smallest · computed-parity · user-env · no-gain)"
|
|
|
|
cp "$BACKUP" "$REAL"
|
|
[[ "$(sha256sum "$REAL" | cut -d' ' -f1)" == "$REAL_SUM" ]] || fail "real envelopes.yml not restored"
|
|
|
|
echo "test-envelopes: ok"
|