noonghunnaandClaude Opus 4.7
8a29b95da1
scripts/report.sh now has four optional sections: --verify verify-full.sh (~1-2 min) --stress verify-stress.sh (~5-10 min, 7/7 incl. Cliff 2 needles) --soak SOAK_MODE=continuous + summary.md embed (~25 min, catches Cliff 2b) --bench bench.sh (~3 min, canonical TPS) --full all four (~35 min total, the canonical cross-rig pass) Why soak as its own flag: verify-full + verify-stress + bench all PASS on configs that FAIL the multi-turn continuous soak (Cliff 2b at ~25K accumulated tokens). Until upstream lands a fix, soak is the only test that catches the agentic-workload failure mode that bit issues #41 / #42 / #43 / #45. Propagated --full as the recommended single-command path through: - README.md (bug/bench callout) - CONTRIBUTING.md (Numbers from your rig + new compose variant gate) - .github/PULL_REQUEST_TEMPLATE.md (one checkbox covers verify+stress+soak+bench) - .github/ISSUE_TEMPLATE/numbers-from-your-rig.yml (single paste includes soak) - BENCHMARKS.md ("How to add a row for your rig") Backward compatible: existing --verify and --bench flags unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2.2 KiB
2.2 KiB
Summary
Type of change
- New compose variant (
models/<model>/<engine>/compose/docker-compose.<name>.yml) - New patch / sidecar (
models/<model>/<engine>/patches/) - New script or tool (
scripts/,tools/) - New model (
models/<new-model>/) - Doc-only / typo
- Other (describe)
Verification
- Full rig + validation report attached — single command captures everything:
Runs hardware + stack + boot log capture plus verify-full + verify-stress 7/7 + SOAK_MODE=continuous + canonical bench in one ~35-min pass. Paste contents as a PR comment. See docs/CLIFFS.md for why the soak-continuous step is load-bearing (catches Cliff 2b, which verify-stress doesn't).
bash scripts/report.sh --full > my-rig.md - BENCHMARKS row added — under the appropriate model section, mirroring existing column shape (incl.
Rigcolumn). - CHANGELOG entry added in
models/<model>/CHANGELOG.md.
If you'd rather run the steps separately:
bash scripts/report.sh > my-rig.md(rig only, ~2 sec)bash scripts/verify-full.sh— fast functional smokebash scripts/verify-stress.sh— 7/7 boundary checks incl. Cliff 2 needlesSOAK_MODE=continuous SOAK_SESSIONS=5 SOAK_TURNS=5 bash scripts/soak-test.sh— required for new single-card composes (catches Cliff 2b)bash scripts/bench.sh— canonical TPS (3 warmups + 5 measured)
N/A justifications (if any boxes above are unchecked)
Cross-links
- Closes #
- Related upstream: