Three fixes from running it against the live game:
1. It blind-pressed the open key. The grid was ALREADY open, so the press
CLOSED it and the scan read grass. Now it detects the grid via its bottom
hint and toggles only when needed, restoring the state it found. The hint
match is fuzzy because Tesseract renders it "PRESS FI-F8 TO BIRD A SKIN"
(BIND->BIRD, SKILL->SKIN).
2. The tooltip ROI was fixed, but the tooltip renders ABOVE the hovered cell
and moves with the row, so it was reading the game world. Now taken relative
to the cell. Skill identification is a fuzzy match against a known list
rather than a demand for clean OCR: real reads included "XI BLESSED HAMMER"
and one frame OCR'd Fist of the Heavens as "LNMEPULE".
3. Hotkey labels are read by TEMPLATE MATCH, not OCR. They are ~20x12px of
white glyph over whatever icon is behind them; brightness thresholding
cannot separate the two when the icon is also bright (F4's swirl, the row-4
weapons) and OCR managed 4/8. A top-hat isolates small bright features
regardless of background: 8/8 with zero false positives.
Matching a same-SIZED crop is 1px-brittle — int() rounding in the grid
geometry lands a pixel off the measured centre and the score collapses from
~1.0 to ~0.4. Exactly the three cells where truncation differed failed. A
4px search slack fixes it; 8/8 held at every threshold 0.60-0.78.
Adds assets/templates/skill_binds/f1..f8.png, cut from a frame with all eight
labels visible.
Verified live: correctly reported conviction=f5 as casting TELEPORT and
concentration=f8 as casting CONVICTION — both confirmed by hand beforehand —
and found Concentration sitting unbound.
Co-Authored-By: Claude Opus 5 <[email protected]>
Answers "which key is each skill actually on?" with screenshots and clicks, no
external service. For each cell of the bind grid it hovers, OCRs the tooltip
for the skill name, OCRs the icon's upper-right corner for the bound F-key, and
optionally saves the icon as a template with that corner blanked.
The corner is excluded from the saved template on purpose. D2R draws the hotkey
label there, so an icon captured with it only matches while the skill stays on
that key — rebind it and the template silently stops matching, looking like
template rot rather than a bind change. Same pixels, read separately as data.
Why it exists: on 2026-08-28 an Enigma put Teleport on F5, displacing
Conviction, while config still said conviction=f5 — every attack-aura cast
would have teleported the character mid-fight. F7 was Vengeance, not Holy Bolt;
F8 was Conviction, not Concentration. The startup preflight reported this
correctly and it was dismissed as a marginal template.
Two things learned building it, both encoded here:
- The tooltip renders ABOVE the hovered cell and moves with the row, so a fixed
ROI reads the game world. The first version returned "YEW Y" and "PET". The
band is now taken relative to the cell.
- Identification is a fuzzy match against a known-skill list rather than a
demand for clean OCR. Real reads included "XI BLESSED HAMMER" and "HOLY
SHIELD L", and one frame OCR'd Fist of the Heavens as "LNMEPULE" while the
full text still contained the name. Validated 7/7 offline against saved
frames.
It refuses to compare when it clearly could not read the grid (<3 skills or 0
bound keys) and exits 2. The first version scanned a closed grid, identified
nothing, and then reported all seven configured keys as unbound — presenting
its own blindness as findings.
Exits 1 on a real mismatch so it can gate a run. Menu entry:
python tools/testbed.py spellbook --assets
Co-Authored-By: Claude Opus 5 <[email protected]>
