Ran the repo's own suite on both arms, same day / same harness, to test the
'possibly contaminated numbers' speculation in club-3090 #710.
CORRECTION — hermesagent-20 is NOT affected by MTP:
MTP-on 11/20 (55%) MTP-off 10/20 (50%)
1 scenario flipped (HA-05, favouring MTP-ON); 9 failed in BOTH arms.
A 1-scenario delta at n=20 is noise -> the banked 60% is within noise of
both arms and is probably fine. Posted as a correction on #710.
REINFORCED — MTP-off costs no quality:
det 57/75 on BOTH arms, identical pack-for-pack (toolcall 10/15,
instructfollow 13/15, structoutput 13/15, dataextract 15/15,
reasonmath 6/15). Verified two independent runs: 0/75 byte-identical
responses, different file hashes.
Both arms score det 57/75 today vs the banked det 64/75 (--full, 2026-06-07
harness) -> that 7-point gap is harness/time drift, not MTP. Cross-time
scores on this suite are not an A/B; the same-day control is what caught it.
Refines the #710 blast radius: the damage scales with PREFIX REUSE, not with
agent-ness. The targeted repro reuses one system prompt 3x (probe 0/6);
hermesagent-20 runs 20 different scenarios so it washes out. Worst real-world
exposure = a long-lived agent session on one stable system prompt.
MTP-off stays: the case rests on TPS (+79%) + the targeted repro, not the packs.
6e30a10b29