Local: L1=30 vs L1=60 A/B — the RAM IS working (~1.6-2x faster warm reads)

Reconciles the earlier L2=0 finding. With L2 on (production shape), a 60 GB L1 serves warm reads ~1.6x faster on average (~2x on the evicted half) than a 30 GB L1 for a ~50 GB working set: older sessions that spill past a 30 GB L1 fall to L2 disk (~3-5s) but stay in RAM at L1=60 (~1.3-2.4s). Two-part model: L2 disk decides *whether* a read is warm (retention); L1 RAM decides *how fast* (speed). L1 is a fast write-through front for L2, not a standalone store — hence L2=0 gave 0/8 yet L2=1+L1=60 is fastest. Decision: keep L1=60, it is not reclaimable headroom. - Add SALT env to lmcache-retention-test.sh (fresh keys per A/B leg, no L2 collision) - Add _local-lmcache-tuning/l1-ab-run.sh (two-leg orchestrator + comparison) - Record result + reconciliation in HANDOFF.md Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 21:44:15 +02:00
parent 8269e79e54
commit b03a134925
6 changed files with 346 additions and 7 deletions
--- a/_local-lmcache-tuning/HANDOFF.md
+++ b/_local-lmcache-tuning/HANDOFF.md
@@ -92,6 +92,44 @@ pgrep -af lmcache-retention-test.sh      # still running?
  `<scratchpad>/tasks/bacnmj267.output`. Warm count = sessions genuinely in fast
  60 GB L1 RAM. Endpoint down during the 2 restarts; restored to L2=1 at end.

+## L1=30 vs L1=60 A/B (COMPLETE, 2026-07-13 21:42) — RECONCILES the L2=0 result
+- Question: with L2 ON (production shape), does 60 GB L1 give FASTER warm reads
+  than 30 GB L1 on a ~50 GB working set — or is the extra 30 GB pure headroom?
+- Method: `_local-lmcache-tuning/l1-ab-run.sh` — recreate at L1=30 then L1=60 (L2
+  stays 1), each leg a fresh SALT (new `SALT` env in the retention test → byte-
+  different prefixes, cannot hit the 49 GB already on L2 disk or the other leg).
+  L1=60 leg doubles as the prod restore. Compares raw Round-2 (warm) TTFTs, NOT
+  pass/fail (both legs pass 8/8 with L2 on). Logs: `l1ab-L130.log`, `l1ab-L160.log`,
+  `l1ab-run.log`.
+- RESULT — **L1=60 warm reads ~1.6× faster on average; the RAM IS doing work:**
+  ```
+  sess  L1=30 s  L1=60 s  faster
+    1     3.32     2.39    L1=60
+    2     4.35     2.05    L1=60
+    3     4.16     1.42    L1=60
+    4     3.58     1.71    L1=60
+    5     5.09     1.27    L1=60
+    6     1.09     3.04    L1=30   (noise; MRU-resident either way)
+    7     1.87     1.05    L1=60
+    8     1.48     2.50    L1=30   (noise; MRU-resident either way)
+  L1=30 warm: mean 3.12s median 3.45s max 5.09s
+  L1=60 warm: mean 1.93s median 1.88s max 3.04s
+  ```
+  Pattern fits theory: the OLDER half (sess 1–5) spills past a 30 GB L1 to L2 disk
+  (~3–5 s) but stays in RAM at L1=60 (~1.3–2.4 s) — ~2× on exactly the sessions that
+  get evicted. The newest ~3 (6–8) are L1-resident in BOTH legs → jitter-level flips.
+- **RECONCILIATION of the two findings (both true, different questions):**
+  - L2 disk decides *whether* a read is warm at all (L2=0 → 0/8; L2=1 → 8/8). RETENTION = L2.
+  - L1 RAM decides *how fast* a warm read is: bigger L1 keeps more sessions in RAM,
+    ~2× faster than falling to L2 disk. WARM-READ SPEED for large working sets = L1.
+  - So L1 works as a **fast front for L2**, not a standalone retention store — which
+    is exactly why L2=0 gave 0/8 (no backend to persist to) yet L2=1+L1=60 is fastest.
+- DECISION: **keep L1=60** — it is NOT reclaimable headroom; it earns ~1.6–2× on warm
+  reads once the working set exceeds ~30 GB (multi-session chat). Production restored
+  and verified: LMCACHE_L1_GB=60, LMCACHE_L2=1, container up. Config is prod-ready.
+- Caveat: n=1 per leg; warm TTFT is jittery (see sess 6/8). Signal is the consistent
+  ~2× on the evicted half, not the modest 1.6× aggregate. Good enough to decide "keep RAM".
+
 ## NEXT PHASE (queued, gated on prod-ready)
 - User wants a **big Hermes test** against qwen3.6-27b :8017 using the **"coldcase"
  project on .96** (192.168.1.96, pings OK; NO coldcase refs in club-3090 repo).
--- a/_local-lmcache-tuning/l1-ab-run.sh
+++ b/_local-lmcache-tuning/l1-ab-run.sh
@@ -0,0 +1,116 @@
+#!/usr/bin/env bash
+# ===========================================================================
+# l1-ab-run.sh — does the 60 GB of L1 RAM actually buy anything, or is it
+# pure headroom the L2 disk makes moot?
+# ---------------------------------------------------------------------------
+# The L2=0 experiment already showed L2 disk (not L1 RAM) delivers RETENTION.
+# This asks the sharper question with L2 left ON (production shape): at a
+# 50 GB working set, does a 60 GB L1 give FASTER warm reads than a 30 GB L1?
+#
+#   - If L1 RAM serves reads:  L1=60 holds all 8 sessions -> every Round-2 read
+#     is L1-fast; L1=30 spills the oldest ~3 to L2 -> those read slower.
+#   - If L1 read is inert (the L2=0=0/8 hypothesis): both legs read everything
+#     from L2 -> identical Round-2 TTFTs -> the extra 30 GB is reclaimable.
+#
+# Method: for L1 in {30, 60}, recreate the container (L2 stays 1), run the
+# retention test with a UNIQUE salt (fresh cache keys — cannot hit the 49 GB
+# already on L2 disk, nor the other leg's blocks), capture the Round-2 curve.
+# The L1=60 leg doubles as the restore to production. Compares raw Round-2
+# TTFTs (NOT pass/fail — with L2 on both legs "pass" the 8 s threshold).
+# ===========================================================================
+set -uo pipefail
+cd /home/alex/club-3090
+C=vllm-qwen36-27b-lmcache
+SLUG=vllm/qwen-27b-dual-lmcache
+DIR=_local-lmcache-tuning
+STAMP=$(date +%s)
+
+run_leg() {
+  local L1="$1"
+  local log="$DIR/l1ab-L1${L1}.log"
+  echo "===================================================================="
+  echo "LEG L1=${L1}GB  (L2 stays 1) — $(date '+%H:%M:%S')"
+  echo "===================================================================="
+  # inline prefix: only THIS switch.sh sees the override; L2 comes from .env (=1).
+  LMCACHE_L1_GB="$L1" bash scripts/switch.sh "$SLUG" --force 2>&1 | tail -8
+  local got_l1 got_l2
+  got_l1=$(docker exec "$C" printenv LMCACHE_L1_GB 2>/dev/null || echo MISSING)
+  got_l2=$(docker exec "$C" printenv LMCACHE_L2   2>/dev/null || echo MISSING)
+  echo ">> container LMCACHE_L1_GB=$got_l1  LMCACHE_L2=$got_l2  (want L1=$L1 L2=1)"
+  if [ "$got_l1" != "$L1" ] || [ "$got_l2" != "1" ]; then
+    echo "!! ABORT leg: env did not match. Skipping test for L1=$L1." | tee "$log"
+    return 1
+  fi
+  # Unique salt per leg => byte-different prefixes => guaranteed-cold Round 1,
+  # no contamination from L2 disk or the other leg.
+  SALT="abL1${L1}-${STAMP}" stdbuf -oL -eL \
+    bash scripts/lmcache-retention-test.sh 2>&1 | tee "$log"
+  echo ">> leg L1=$L1 done, log=$log"
+}
+
+echo "== L1 A/B START $(date '+%H:%M:%S')  salt-stamp=$STAMP =="
+run_leg 30
+run_leg 60   # <- also the restore to production (.env L1=60)
+
+# Safety net: the RAM gate reads MemAvailable once (no settle-retry); if a slow
+# reclaim after the L1=30 teardown tripped it, production could be left DOWN.
+# Retry the plain (.env L1=60) restore up to twice before giving up.
+for attempt in 1 2; do
+  FL1=$(docker exec "$C" printenv LMCACHE_L1_GB 2>/dev/null || echo MISSING)
+  up=$(docker ps --filter "name=$C" --format '{{.Names}}')
+  [ "$FL1" = "60" ] && [ -n "$up" ] && break
+  echo "!! production not at L1=60/up (got L1=$FL1, up='$up') — restore retry $attempt after 20s"
+  sleep 20
+  bash scripts/switch.sh "$SLUG" --force 2>&1 | tail -6
+done
+
+# Final production sanity.
+FL1=$(docker exec "$C" printenv LMCACHE_L1_GB 2>/dev/null || echo MISSING)
+FL2=$(docker exec "$C" printenv LMCACHE_L2   2>/dev/null || echo MISSING)
+echo ">> FINAL production state: LMCACHE_L1_GB=$FL1  LMCACHE_L2=$FL2  (want 60 / 1)"
+
+# ---- Compare the two Round-2 curves --------------------------------------
+echo "===================================================================="
+echo "COMPARISON — Round-2 (warm) TTFT per session, L1=30 vs L1=60"
+echo "===================================================================="
+python3 - "$DIR/l1ab-L130.log" "$DIR/l1ab-L160.log" <<'PY'
+import re, sys, statistics as st
+def curve(path):
+    # Round-2 lines look like: "  session  1: TTFT   2.13s  WARM  retained"
+    out=[]
+    try:
+        txt=open(path,encoding="utf-8").read()
+    except FileNotFoundError:
+        return out
+    seg=txt.split("Round 2",1)
+    body=seg[1] if len(seg)>1 else ""
+    for m in re.finditer(r"session\s+(\d+):\s+TTFT\s+([\d.]+)s", body):
+        out.append((int(m.group(1)), float(m.group(2))))
+    return out
+a=curve(sys.argv[1]); b=curve(sys.argv[2])
+da=dict(a); db=dict(b)
+print(f"{'sess':>4} {'L1=30 s':>9} {'L1=60 s':>9}   {'faster':>8}")
+for s in sorted(set(da)|set(db)):
+    x=da.get(s); y=db.get(s)
+    xs=f"{x:.2f}" if x is not None else "  --"
+    ys=f"{y:.2f}" if y is not None else "  --"
+    fast = ""
+    if x is not None and y is not None:
+        fast = "L1=60" if y < x*0.85 else ("L1=30" if x < y*0.85 else "~tie")
+    print(f"{s:>4} {xs:>9} {ys:>9}   {fast:>8}")
+def summ(v):
+    xs=[t for _,t in v]
+    return f"n={len(xs)} mean={st.mean(xs):.2f}s median={st.median(xs):.2f}s max={max(xs):.2f}s" if xs else "no data"
+print("\nL1=30 warm:", summ(a))
+print("L1=60 warm:", summ(b))
+if a and b:
+    ma,mb=st.mean(t for _,t in a),st.mean(t for _,t in b)
+    if mb < ma*0.85:
+        print(f"\nVERDICT: L1=60 warm reads ~{ma/mb:.1f}x faster => the RAM IS doing work.")
+    elif ma < mb*0.85:
+        print(f"\nVERDICT: L1=30 faster (unexpected) — investigate.")
+    else:
+        print("\nVERDICT: warm reads ~identical => extra 30 GB L1 is HEADROOM (reclaimable);"
+              " L2 disk is serving the reads either way.")
+PY
+echo "== L1 A/B DONE $(date '+%H:%M:%S') =="
--- a/_local-lmcache-tuning/l1ab-L130.log
+++ b/_local-lmcache-tuning/l1ab-L130.log
@@ -0,0 +1,32 @@
+LMCache L1 retention probe -> http://localhost:8017   model=qwen3.6-27b
+  sessions=8  target_tokens/session~=48000  warm_threshold=8.0s  kv=131KB/tok
+
+Building calibrated haystacks (one /tokenize probe each)...
+
+Round 1 - cold insert (fills L1):
+  session  1: cold TTFT   53.20s  (prompt_tokens=47559)
+  session  2: cold TTFT   52.73s  (prompt_tokens=47991)
+  session  3: cold TTFT   53.78s  (prompt_tokens=48207)
+  session  4: cold TTFT   51.33s  (prompt_tokens=46318)
+  session  5: cold TTFT   52.83s  (prompt_tokens=47015)
+  session  6: cold TTFT   54.30s  (prompt_tokens=47676)
+  session  7: cold TTFT   55.95s  (prompt_tokens=48343)
+  session  8: cold TTFT   56.34s  (prompt_tokens=48098)
+
+  inserted ~381207 tokens ~= 49.9 GB of L1
+  capacity lines: L1=30 -> ~229K tok (~30 GB), L1=60 -> ~458K tok (~60 GB)
+
+Round 2 - retention read (same prefixes; warm = retained):
+  session  1: TTFT    3.32s  WARM  retained
+  session  2: TTFT    4.35s  WARM  retained
+  session  3: TTFT    4.16s  WARM  retained
+  session  4: TTFT    3.58s  WARM  retained
+  session  5: TTFT    5.09s  WARM  retained
+  session  6: TTFT    1.09s  WARM  retained
+  session  7: TTFT    1.87s  WARM  retained
+  session  8: TTFT    1.48s  WARM  retained
+
+  retained warm: 8/8
+
+  PASS - all sessions retained. L1 held the full working set;
+         this set would have evicted its earliest sessions at L1=30.
--- a/_local-lmcache-tuning/l1ab-L160.log
+++ b/_local-lmcache-tuning/l1ab-L160.log
@@ -0,0 +1,32 @@
+LMCache L1 retention probe -> http://localhost:8017   model=qwen3.6-27b
+  sessions=8  target_tokens/session~=48000  warm_threshold=8.0s  kv=131KB/tok
+
+Building calibrated haystacks (one /tokenize probe each)...
+
+Round 1 - cold insert (fills L1):
+  session  1: cold TTFT   55.80s  (prompt_tokens=48631)
+  session  2: cold TTFT   51.99s  (prompt_tokens=46869)
+  session  3: cold TTFT   54.48s  (prompt_tokens=48031)
+  session  4: cold TTFT   55.22s  (prompt_tokens=48240)
+  session  5: cold TTFT   55.62s  (prompt_tokens=47982)
+  session  6: cold TTFT   58.44s  (prompt_tokens=49035)
+  session  7: cold TTFT   56.92s  (prompt_tokens=47843)
+  session  8: cold TTFT   58.67s  (prompt_tokens=48719)
+
+  inserted ~385350 tokens ~= 50.5 GB of L1
+  capacity lines: L1=30 -> ~229K tok (~30 GB), L1=60 -> ~458K tok (~60 GB)
+
+Round 2 - retention read (same prefixes; warm = retained):
+  session  1: TTFT    2.39s  WARM  retained
+  session  2: TTFT    2.05s  WARM  retained
+  session  3: TTFT    1.42s  WARM  retained
+  session  4: TTFT    1.71s  WARM  retained
+  session  5: TTFT    1.27s  WARM  retained
+  session  6: TTFT    3.04s  WARM  retained
+  session  7: TTFT    1.05s  WARM  retained
+  session  8: TTFT    2.50s  WARM  retained
+
+  retained warm: 8/8
+
+  PASS - all sessions retained. L1 held the full working set;
+         this set would have evicted its earliest sessions at L1=30.
--- a/_local-lmcache-tuning/l1ab-run.log
+++ b/_local-lmcache-tuning/l1ab-run.log
@@ -0,0 +1,110 @@
+== L1 A/B START 21:19:29  salt-stamp=1783970369 ==
+====================================================================
+LEG L1=30GB  (L2 stays 1) — 21:19:29
+====================================================================
+[switch]   64s — Loading weights
+[switch]   120s elapsed, still waiting...
+[switch]   180s elapsed, still waiting...
+[switch]   200s — Capturing CUDA graphs
+[switch]   216s — Application startup complete
+[switch] ✓ ready (216s)
+[switch] ▶ API:  http://192.168.1.98:8017/v1   (model: qwen3.6-27b · OpenAI-compatible · no auth)
+[switch] done. Try:  curl -s http://localhost:8017/v1/models | jq .
+>> container LMCACHE_L1_GB=30  LMCACHE_L2=1  (want L1=30 L2=1)
+LMCache L1 retention probe -> http://localhost:8017   model=qwen3.6-27b
+  sessions=8  target_tokens/session~=48000  warm_threshold=8.0s  kv=131KB/tok
+
+Building calibrated haystacks (one /tokenize probe each)...
+
+Round 1 - cold insert (fills L1):
+  session  1: cold TTFT   53.20s  (prompt_tokens=47559)
+  session  2: cold TTFT   52.73s  (prompt_tokens=47991)
+  session  3: cold TTFT   53.78s  (prompt_tokens=48207)
+  session  4: cold TTFT   51.33s  (prompt_tokens=46318)
+  session  5: cold TTFT   52.83s  (prompt_tokens=47015)
+  session  6: cold TTFT   54.30s  (prompt_tokens=47676)
+  session  7: cold TTFT   55.95s  (prompt_tokens=48343)
+  session  8: cold TTFT   56.34s  (prompt_tokens=48098)
+
+  inserted ~381207 tokens ~= 49.9 GB of L1
+  capacity lines: L1=30 -> ~229K tok (~30 GB), L1=60 -> ~458K tok (~60 GB)
+
+Round 2 - retention read (same prefixes; warm = retained):
+  session  1: TTFT    3.32s  WARM  retained
+  session  2: TTFT    4.35s  WARM  retained
+  session  3: TTFT    4.16s  WARM  retained
+  session  4: TTFT    3.58s  WARM  retained
+  session  5: TTFT    5.09s  WARM  retained
+  session  6: TTFT    1.09s  WARM  retained
+  session  7: TTFT    1.87s  WARM  retained
+  session  8: TTFT    1.48s  WARM  retained
+
+  retained warm: 8/8
+
+  PASS - all sessions retained. L1 held the full working set;
+         this set would have evicted its earliest sessions at L1=30.
+>> leg L1=30 done, log=_local-lmcache-tuning/l1ab-L130.log
+====================================================================
+LEG L1=60GB  (L2 stays 1) — 21:30:54
+====================================================================
+[switch]   68s — Loading weights
+[switch]   120s elapsed, still waiting...
+[switch]   180s elapsed, still waiting...
+[switch]   208s — Capturing CUDA graphs
+[switch]   224s — Application startup complete
+[switch] ✓ ready (224s)
+[switch] ▶ API:  http://192.168.1.98:8017/v1   (model: qwen3.6-27b · OpenAI-compatible · no auth)
+[switch] done. Try:  curl -s http://localhost:8017/v1/models | jq .
+>> container LMCACHE_L1_GB=60  LMCACHE_L2=1  (want L1=60 L2=1)
+LMCache L1 retention probe -> http://localhost:8017   model=qwen3.6-27b
+  sessions=8  target_tokens/session~=48000  warm_threshold=8.0s  kv=131KB/tok
+
+Building calibrated haystacks (one /tokenize probe each)...
+
+Round 1 - cold insert (fills L1):
+  session  1: cold TTFT   55.80s  (prompt_tokens=48631)
+  session  2: cold TTFT   51.99s  (prompt_tokens=46869)
+  session  3: cold TTFT   54.48s  (prompt_tokens=48031)
+  session  4: cold TTFT   55.22s  (prompt_tokens=48240)
+  session  5: cold TTFT   55.62s  (prompt_tokens=47982)
+  session  6: cold TTFT   58.44s  (prompt_tokens=49035)
+  session  7: cold TTFT   56.92s  (prompt_tokens=47843)
+  session  8: cold TTFT   58.67s  (prompt_tokens=48719)
+
+  inserted ~385350 tokens ~= 50.5 GB of L1
+  capacity lines: L1=30 -> ~229K tok (~30 GB), L1=60 -> ~458K tok (~60 GB)
+
+Round 2 - retention read (same prefixes; warm = retained):
+  session  1: TTFT    2.39s  WARM  retained
+  session  2: TTFT    2.05s  WARM  retained
+  session  3: TTFT    1.42s  WARM  retained
+  session  4: TTFT    1.71s  WARM  retained
+  session  5: TTFT    1.27s  WARM  retained
+  session  6: TTFT    3.04s  WARM  retained
+  session  7: TTFT    1.05s  WARM  retained
+  session  8: TTFT    2.50s  WARM  retained
+
+  retained warm: 8/8
+
+  PASS - all sessions retained. L1 held the full working set;
+         this set would have evicted its earliest sessions at L1=30.
+>> leg L1=60 done, log=_local-lmcache-tuning/l1ab-L160.log
+>> FINAL production state: LMCACHE_L1_GB=60  LMCACHE_L2=1  (want 60 / 1)
+====================================================================
+COMPARISON — Round-2 (warm) TTFT per session, L1=30 vs L1=60
+====================================================================
+sess   L1=30 s   L1=60 s     faster
+   1      3.32      2.39      L1=60
+   2      4.35      2.05      L1=60
+   3      4.16      1.42      L1=60
+   4      3.58      1.71      L1=60
+   5      5.09      1.27      L1=60
+   6      1.09      3.04      L1=30
+   7      1.87      1.05      L1=60
+   8      1.48      2.50      L1=30
+
+L1=30 warm: n=8 mean=3.12s median=3.45s max=5.09s
+L1=60 warm: n=8 mean=1.93s median=1.88s max=3.04s
+
+VERDICT: L1=60 warm reads ~1.6x faster => the RAM IS doing work.
+== L1 A/B DONE 21:42:31 ==
--- a/scripts/lmcache-retention-test.sh
+++ b/scripts/lmcache-retention-test.sh
@@ -41,6 +41,11 @@
 #                       re-prefill is ~35-45 s — 8 s cleanly separates them)
 #   KV_KB_PER_TOKEN    LMCache cache rate for the GB math. Default: 131 (measured)
 #   PROBE_MAX_TOKENS   Output cap per request (just enough to complete). Default: 8
+#   SALT               Mixes into every prefix's seed + anchor so a re-run inserts
+#                      BYTE-DIFFERENT prefixes -> fresh cache keys that cannot hit
+#                      KV left on L2 disk by a prior run. Default: "" (the original
+#                      deterministic prefixes). Give each A/B leg a distinct SALT so
+#                      the two legs never read each other's (or old) cached blocks.
 #
 # Exit 0 = all sessions retained (PASS). Exit 1 = one or more evicted.
 # ===========================================================================
@@ -54,18 +59,22 @@ SESSION_TOKENS="${SESSION_TOKENS:-48000}"
 WARM_THRESHOLD_S="${WARM_THRESHOLD_S:-8.0}"
 KV_KB_PER_TOKEN="${KV_KB_PER_TOKEN:-131}"
 PROBE_MAX_TOKENS="${PROBE_MAX_TOKENS:-8}"
+SALT="${SALT:-}"

 command -v python3 >/dev/null || { echo "Fix: python3 not found on PATH." >&2; exit 2; }

 python3 - "$URL" "$MODEL" "$NUM_SESSIONS" "$SESSION_TOKENS" \
-           "$WARM_THRESHOLD_S" "$KV_KB_PER_TOKEN" "$PROBE_MAX_TOKENS" <<'PY'
-import json, random, string, sys, time, urllib.request
+           "$WARM_THRESHOLD_S" "$KV_KB_PER_TOKEN" "$PROBE_MAX_TOKENS" "$SALT" <<'PY'
+import json, random, string, sys, time, urllib.request, zlib
 # Community rigs run non-UTF-8 locales; a piped stdout defaults to ASCII (repo
 # convention: pin utf-8 on both read and write).
 sys.stdout.reconfigure(encoding="utf-8")

-URL, MODEL, N, STOK, WARM, KVKB, PMAX = sys.argv[1:]
+URL, MODEL, N, STOK, WARM, KVKB, PMAX, SALT = sys.argv[1:]
 N = int(N); STOK = int(STOK); WARM = float(WARM); KVKB = float(KVKB); PMAX = int(PMAX)
+# Stable integer offset derived from SALT (crc32 is process-independent, unlike
+# builtin hash() under PYTHONHASHSEED). Empty SALT -> 0 -> original prefixes.
+SALT_OFF = (zlib.crc32(SALT.encode()) % 1_000_000) if SALT else 0


 def tokenize_count(text):
@@ -90,14 +99,16 @@ def rand_words(rng, n):


 def make_prefix(idx, target_tokens):
-    """Deterministic per idx -> byte-identical across rounds -> real cache hit.
+    """Deterministic per (idx, SALT) -> byte-identical across rounds within a run
+       (real cache hit) but byte-DIFFERENT across salts (fresh key vs prior runs).
       Calibrated against the live tokenizer so token count is ~on target."""
-    rng = random.Random(idx * 7919 + 13)
-    sample = " ".join(rand_words(random.Random(idx), 400))
+    key = idx + SALT_OFF
+    rng = random.Random(key * 7919 + 13)
+    sample = " ".join(rand_words(random.Random(key), 400))
    c = tokenize_count(sample)
    tpw = (c / 400.0) if c else 1.4          # tokens-per-word for this stream
    n_words = max(64, int(target_tokens / tpw))
-    header = f"SESSION-{idx:03d}-anchor-{idx * 7919}\n"
+    header = f"SESSION-{idx:03d}-salt-{SALT_OFF}-anchor-{key * 7919}\n"
    body = header + " ".join(rand_words(rng, n_words))
    return body + "\n\nReply with only: OK"