The results/ directory accumulated 21 MB of intermediate investigation
output during today's cliff/grammar work. Most of that (residency probes,
soak iteration matrices) lived its useful life in issue threads and the
gitignored docs/diagnostics/ memos, and isn't worth eternal evidence.
What's evidence (committed): 1.5 MB of raw bench output backing published
numbers in STRUCTURED_COT.md / BENCHMARKS.md / CHANGELOG:
- results/grammar-ab-20260503-224235/ — Phase 2 30-problem subset bench
(was the basis for the n=30 PROMPT_TERSE-wins finding that Phase 3
later disproved at scale)
- results/grammar-full-20260504-003118-gpu0/ — Phase 3 HE+ shard 1 (82 rows)
- results/grammar-full-20260504-003118-gpu1/ — Phase 3 HE+ shard 2 (82 rows)
- results/grammar-full-lcb-20260504-021003/ — Phase 3 LCB v6 (50 rows + shard
metadata; this is the post-bug-fix re-run after the harness LCB issue
surfaced and was patched)
- tools/grammar-eval/codex-check-scratchpad.gbnf — fourth grammar candidate
exploring DeepSeek + STATE/CHECK pre-VERDICT scaffolding (untested in
Phase 3; available for future bench)
What's investigation artifact (now gitignored): residency-* (Codex Cliff 2b
investigation probes, ~5 MB) + soak-* (~16 MB of cliff probe iterations,
final validation matrix already in #41 + cliff2_accumulated_ctx_finding.md
memory).
.gitignore pattern: `results/*` with explicit negations for `grammar-ab-*`,
`grammar-full-*`, and the existing `v0.20-migration` directory. Future
grammar bench runs auto-track when committed; future residency/soak runs
auto-ignore. The pattern works because git's directory-ignore precedence
applies to immediate children, not the parent itself, so negations
re-include named subdirs.
Total committed: 522 lines of bench output (jsonl + json + summary md).
The data is small (per-row generations cap at ~4096 tokens × 5 conditions
× 214 problems = ~4 MB raw, much of which compresses well in jsonl).
Held back for user decision: tools/residency-instrument/ (Codex's Cliff 2b
instrumentation harness — useful tool, no docs yet, may want a README
before publication).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
d82e89807a