Files
club-3090/models
noonghunna 7b0c4d4abc docs(lmcache): fold cross-rig findings — warm-L2 compute-bound + aborted-prefill caveat
From alexpolo1's bare-metal 2x 3090 + NVMe re-run (#423, 2026-06-24, after #454):

1. Warm-L2 rehydrate is COMPUTE-bound, not disk-bound (5.30 s = 405 ms disk +
   37 ms H2D + ~4.8 s Mamba/GDN-state recompute) — a faster SSD won't speed
   warm-load on this hybrid; the L2 tier buys persistence/capacity, not warm
   speed. Closes the lmcache.yml "L2 latency not yet measured on-rig" caveat.
2. An aborted prefill banks only the completed prefix (46/320) — LMCache
   commits a session's blocks only on request completion, so the first long
   prefill must run to completion or it re-prefills from scratch.

INTERNALS.md LMCache section + lmcache.yml header. Comment/prose only.
2026-06-24 17:59:17 +00:00
..