5 Commits
Author SHA1 Message Date
noonghunna eeb946b0a7 chore(changelog): subject-only rendering (drop commit body verbosity)
Release / release (push) Failing after 51s
CHANGELOG.md and GitHub Release notes now render commit subjects only.
Full commit message body (why/how/validation data) stays in 'git log',
one click away via the SHA link. Keeps both surfaces skim-readable.

v0.3.2 release pages were 100+ lines per commit; this brings them down
to ~1 line per commit.
2026-05-10 21:14:34 +00:00
github-actions[bot] 507a36f7f7 chore(changelog): regenerate for v0.3.2 [skip ci] 2026-05-10 21:08:51 +00:00
noonghunna 255c743dff chore: trigger v0.3.2 release workflow (GitHub deduped previous tag push)
Release / release (push) Failing after 48s
The v0.3.2 tag was originally pushed at commit 64b0474 but GitHub didn't
emit a CreateEvent (likely dedup after delete + re-push to same SHA), so
the release.yml workflow never fired. Empty commit gives the tag a fresh
SHA that GitHub will process cleanly.
2026-05-10 21:08:33 +00:00
noonghunna 64b0474a62 chore(changelog): automate CHANGELOG + release notes from commits via cliff (Option A)
CHANGELOG.md is now auto-generated from commit messages by git-cliff in
the release workflow. Hand-edits below the static header will be wiped on
the next tag.

Workflow (`.github/workflows/release.yml`):
  - On tag push (`v[0-9]+.[0-9]+.[0-9]+`):
    1. Render GitHub Release body: `git-cliff --latest --strip header`
       → just the per-version section, no SemVer preamble repeat
    2. Regenerate full CHANGELOG.md: `git-cliff` (default = all tags)
       → preserves header + all historical sections
    3. Commit CHANGELOG.md back to master with `[skip ci]` marker
    4. Publish GitHub Release with the latest-only body

Template (`cliff.toml`):
  - `[changelog].header` now holds the SemVer preamble + CalVer→SemVer
    mapping table (preserved across regens; stripped from GitHub Release
    bodies via `--strip header`).
  - `body` template now renders the **full commit message** (subject as
    bold bullet, body indented below) instead of just the first line.
    Rich narrative I write in commit message bodies (tables, validation
    numbers, before/after diffs) now flows into both CHANGELOG.md and the
    GitHub Release page from the same source.
  - Per-release Pin/Diff footer guarded with `{% if version %}` so the
    Unreleased section doesn't emit empty links.

CHANGELOG.md replaced with the auto-gen output. Past hand-written tables
and phase breakdowns are replaced by the corresponding commit messages
(those were already rich for commits that mattered — v0.3.1 soak-helper
fix has its Before/After table in the commit body and renders fine).

Going forward: just write rich commit messages and tag. Both surfaces
update automatically. No hand-edit of CHANGELOG.md required.
2026-05-10 20:53:51 +00:00
noonghunna 83bf73d3ec feat(quality-test): auto-set BENCHLOCAL_HERMES_RESOLVE_LOCALHOST=1 for localhost URLs
When the user runs quality-test.sh with a localhost-style URL
(default `http://localhost:8020`, or any `localhost`/`127.x`/`[::1]`
variant), auto-export `BENCHLOCAL_HERMES_RESOLVE_LOCALHOST=1` so
benchlocal-cli rewrites the hermes-agent's outbound model endpoint
from `localhost:<port>` to `host.docker.internal:<port>` inside the
Docker sandbox container.

Without this, the hermes-agent inside the sandbox can't reach the
host's vLLM (localhost resolves to the container itself) and every
scenario fails with `"API call failed after 3 retries: Connection
error."` — produced spurious 0/20 grades on this rig prior to the
benchlocal-cli runner.py 9c1566f fix.

Skips the auto-set when:
  - User already set the env var (explicit override)
  - URL points at a non-loopback host (real LAN IP, k8s service name,
    host.docker.internal already) — no rewrite needed

Emits a stderr breadcrumb when the auto-set fires so users can see
what changed.
2026-05-10 20:29:49 +00:00
4 changed files with 8836 additions and 466 deletions
+40 -3
View File
@@ -16,17 +16,54 @@ jobs:
uses: actions/checkout@v4
with:
fetch-depth: 0 # full history needed for git-cliff
token: ${{ secrets.GITHUB_TOKEN }}
- name: Generate release notes
id: cliff
# ---- (1) Generate the GitHub Release body (latest only, no header) ----
# `--strip header` drops the static SemVer preamble so the release page
# shows just the per-version section, while CHANGELOG.md (below) keeps
# the header at the top of the file.
- name: Generate release notes (latest only)
id: cliff-release
uses: orhun/git-cliff-action@v4
with:
config: cliff.toml
args: --latest --github-repo noonghunna/club-3090
args: --latest --strip header --github-repo noonghunna/club-3090
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
OUTPUT: RELEASE_NOTES.md
# ---- (2) Regenerate full CHANGELOG.md (all tags, with header) ----
- name: Regenerate CHANGELOG.md
uses: orhun/git-cliff-action@v4
with:
config: cliff.toml
args: --github-repo noonghunna/club-3090
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
OUTPUT: CHANGELOG.md
# ---- (3) Commit the regenerated CHANGELOG.md back to master ----
# The tag's commit doesn't include this auto-regen — but the GitHub
# Release page is correct (step 1), and master's CHANGELOG.md catches
# up ~1 min after tag push. Skipped if nothing changed.
- name: Commit CHANGELOG.md back to master
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git fetch origin master
git checkout -B master origin/master
# Re-run cliff against master HEAD so the regen reflects what master
# actually contains (the tag may not yet be on master if pushed from
# a feature branch; uncommon but handled).
git add CHANGELOG.md
if git diff --staged --quiet; then
echo "CHANGELOG.md unchanged — skipping commit."
else
git commit -m "chore(changelog): regenerate for ${{ github.ref_name }} [skip ci]"
git push origin master
fi
# ---- (4) Publish GitHub Release ----
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
with:
+8752 -444
View File
File diff suppressed because it is too large Load Diff
+31 -19
View File
@@ -8,15 +8,40 @@
# → release published.
[changelog]
# Header rendered once at top of every release body.
# Header rendered once at top of CHANGELOG.md (preserved across full regens).
# Use `--strip header` on `--latest` for GitHub Release bodies so they don't
# duplicate this intro on every release page.
header = """
# Changelog
Auto-generated from commit subjects by [git-cliff](https://git-cliff.org/) on tag
push. Click any commit SHA below to see the full message body (why / how /
validation data) — those live in `git log`, not here. Don't hand-edit; the file
is regenerated on every tag.
**Versioning:** SemVer in `0.x` — treat any minor bump as potentially breaking
until `1.0`. Past CalVer tags (`v2026.05.09`, `v2026.05.10`) are preserved for
history; SemVer takes over from `v0.3.0` onward.
| CalVer tag | SemVer equivalent | Date |
|---|---|---|
| `v2026.05.09` | (≈ v0.1.0) | 2026-05-09 — first tagged release |
| `v2026.05.10` | (≈ v0.2.0) | 2026-05-10 — stack reorg + Gemma 4 INT8 PTH unblock |
---
"""
# Body template — rendered per release (we use --latest so only one).
# Tera templating syntax. Each commit shows as a bullet with PR link if present.
# Body template — rendered per release. For `--latest` (GitHub Release) only
# the most recent block renders; for full regen of CHANGELOG.md, every tagged
# block renders in reverse-chronological order.
#
# **Subject-line only** — the commit's first line, with a link to the full
# commit. Full body (why/how/validation data) lives in `git log` and is one
# click away via the SHA link. Keeps CHANGELOG.md skim-readable.
body = """
{% if version %}\
## What's in {{ version }}
## {{ version }}{% if timestamp %} — {{ timestamp | date(format="%Y-%m-%d") }}{% endif %}
{% else %}\
## Unreleased
@@ -30,21 +55,8 @@ body = """
{% endfor %}
{% endfor %}
---
## Pinning to this release
**Versioning:** SemVer in `0.x` — treat any minor bump as potentially breaking until `1.0`. Past CalVer tags (`v2026.05.09`, `v2026.05.10`) are preserved for history; SemVer takes over from `v0.3.0` onward.
```bash
git checkout {{ version }}
```
When posting cross-rig benchmark numbers ([disc #86](https://github.com/noonghunna/club-3090/discussions/86)), please include this version tag (or commit SHA) so others can reproduce against the same script revision.
{% if previous.version %}\
**Full diff:** [{{ previous.version }}...{{ version }}](https://github.com/noonghunna/club-3090/compare/{{ previous.version }}...{{ version }})
{% endif %}\
{% if version %}[Pin: `git checkout {{ version }}`]{% if previous.version %} · [Full diff](https://github.com/noonghunna/club-3090/compare/{{ previous.version }}...{{ version }}){% endif %}
{% endif %}
"""
footer = ""
+13
View File
@@ -177,6 +177,19 @@ if [[ -n "$DETECTED_MODEL" && "$DETECTED_MODEL" != "$MODEL" ]]; then
MODEL="$DETECTED_MODEL"
fi
# hermesagent-20 runs its agent inside a Docker sandbox container. Localhost-style
# URLs (localhost/127.x/[::1]) inside the container resolve to the container itself,
# not the host's vLLM. Auto-set BENCHLOCAL_HERMES_RESOLVE_LOCALHOST=1 so benchlocal-cli
# (a) adds --add-host=host.docker.internal:host-gateway to the sandbox container, and
# (b) rewrites the model endpoint URL to use host.docker.internal:<port> for the
# hermes-agent's outbound API calls. Skip if already set (user override) or if URL
# already uses host.docker.internal / a non-loopback host (real LAN IP, k8s service).
if [[ -z "${BENCHLOCAL_HERMES_RESOLVE_LOCALHOST:-}" ]] \
&& [[ "$URL" =~ ^https?://(localhost|127\.|\[::1\]) ]]; then
export BENCHLOCAL_HERMES_RESOLVE_LOCALHOST=1
echo "[quality-test] localhost URL detected — auto-set BENCHLOCAL_HERMES_RESOLVE_LOCALHOST=1 for hermes sandbox endpoint rewrite" >&2
fi
# ---- run benchlocal-cli ------------------------------------------------------
RESULTS_DIR="${ROOT_DIR}/results/quality"