archive: botty_next test harness, legacy-go docs, and dev tools from my-botty

This commit is contained in:
alexpolo1
2026-08-07 22:25:26 +02:00
commit 5d960cde66
72 changed files with 3130 additions and 0 deletions

93
botty_next/README.md Normal file
View File

@@ -0,0 +1,93 @@
# botty_next — visual test harness
Bootstrapped in commit `48d8445`. A standalone, importable package (`botty_next/`) that
exercises the vision primitives — screen capture, template matching, OCR — in isolation from
the live bot, so they can be validated against fixtures in CI without D2R running.
It does **not** drive the game. Input is gated off by default and there is no live-input path yet
(see `InputConfig` below). Think of it as a test bench for the perception layer that the legacy
`src/` bot will eventually be ported onto.
## Layout
```
botty_next/
cli.py argparse entry point: config / detect / capture / ocr
capture/
window.py WindowRegion + find_window_region() (win32gui enumerate)
mss_backend.py MssCaptureBackend.grab() -> BGR ndarray; save_frame()
vision/
fixtures.py load_image / load_screenshot / load_template (cv2.imread)
template_matching.py match_template() -> MatchResult; save_match_debug()
ocr.py preprocess_for_ocr(), run_tesseract_ocr() -> OcrResult
config/
models.py pydantic config models + load_config()
default.yaml default profile
tests/ pytest suite for each module
debug/ debug-image output dir
```
## CLI
`python -m botty_next.cli <command>` (entry: `cli.main`). Every command prints a JSON result and
returns a process exit code.
| Command | Args | Does | Exit code |
|---------|------|------|-----------|
| `config validate` | `-c/--config PATH` | Loads + validates a YAML profile, prints the resolved config | 0 |
| `detect template` | `--image --template [--threshold 0.85] [--debug-output]` | Runs `match_template`, optionally writes an annotated debug image | 0 if `passed`, else 1 |
| `capture` | `--output [--window-title]` | Grabs a frame (full monitor, or the matched window region) and saves it | 0 |
| `ocr` | `--image [--lang eng] [--psm 6] [--tesseract-cmd] [--debug-output]` | OCRs an image; writes the preprocessed debug image first if requested | 0 ok / 2 if pytesseract missing |
## Core logic
### Capture (`capture/`)
- `find_window_region(title_contains)` enumerates visible top-level windows via `win32gui`,
case-insensitively substring-matches the title, and returns the **largest** match as a
`WindowRegion(left, top, width, height, title)`. Raises if none found.
- `WindowRegion.as_mss_monitor()` adapts it to the dict `mss` expects.
- `MssCaptureBackend.grab(region)` grabs that region (or `monitors[1]` = primary monitor when
`region is None`) and converts the raw BGRA to **BGR** so it matches OpenCV's convention.
- `save_frame()` creates parent dirs and writes via `cv2.imwrite`, raising on failure.
### Template matching (`vision/template_matching.py`)
- `match_template(image, template, threshold=0.85, method=TM_CCOEFF_NORMED)`:
- Validates non-empty inputs and that the template isn't larger than the image.
- Converts both to grayscale, runs `cv2.matchTemplate` + `cv2.minMaxLoc`.
- For `TM_SQDIFF*` methods the **min** location wins and `confidence = 1 - min_val`; for all
other methods the **max** location wins and `confidence = max_val`. This normalizes so
"higher confidence = better" regardless of method.
- Returns a frozen `MatchResult(confidence, bbox, passed, method, debug)` where
`passed = confidence >= threshold` and `debug` carries the raw min/max values and shapes.
- `save_match_debug()` draws the bbox green if passed, red if not, and writes it.
### OCR (`vision/ocr.py`)
- `preprocess_for_ocr(image, scale=2.0)`: grayscale → 2× upscale (`INTER_CUBIC`) →
Gaussian blur → adaptive Gaussian threshold (block 31, C 7). This is the single source of
truth for OCR preprocessing — both the OCR run and the debug image use it.
- `run_tesseract_ocr(...)`: lazily imports `pytesseract` (raising a `RuntimeError` with install
guidance if absent), optionally sets `tesseract_cmd`, runs `image_to_string` for text and
`image_to_data` for per-word confidences, and returns `OcrResult(text, confidence, bbox, debug)`.
Confidence is the mean of word confidences (each normalized 01, negatives dropped).
### Config (`config/models.py`)
Pydantic models with `extra="forbid"` (unknown keys are rejected). `load_config(path)` reads YAML
and validates it into `BottyNextConfig`:
- `CaptureConfig``backend` (`fixture`/`mss`/`dxcam`, default `fixture`), `monitor`,
`fps_limit` (1240), optional `window_title`.
- `VisionConfig``template_threshold` (01), `debug_output_dir`.
- `InputConfig`**safety gate**: `enabled=False`, `dry_run=True` by default. A model validator on
`BottyNextConfig` raises if `input.enabled` is true while `dry_run` is false — i.e. live input
is impossible until a future explicit safety gate is added.
## Fixtures
`fixtures/screenshots/sample_scene.ppm` and `fixtures/templates/sample_marker.ppm` are committed
(PPM so they diff/version cleanly) and let the test suite run with no external assets.
`fixtures/ocr_samples/.gitkeep` reserves the OCR sample dir.
## Tests
`botty_next/tests/` has one module per concern (`test_capture`, `test_cli`, `test_config`,
`test_fixtures`, `test_ocr`, `test_template_matching`). They run against the committed fixtures,
so the harness is CI-safe without a display or a running game.

5
botty_next/__init__.py Normal file
View File

@@ -0,0 +1,5 @@
"""Botty Next offline-first visual QA harness."""
__all__ = ["__version__"]
__version__ = "0.1.0"

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1,6 @@
"""Capture backends for offline fixtures and live observer mode."""
from botty_next.capture.mss_backend import MssCaptureBackend
from botty_next.capture.window import WindowRegion, find_window_region
__all__ = ["MssCaptureBackend", "WindowRegion", "find_window_region"]

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1,27 @@
from __future__ import annotations
from pathlib import Path
import cv2
import mss
import numpy as np
from botty_next.capture.window import WindowRegion
class MssCaptureBackend:
def grab(self, region: WindowRegion | None = None) -> np.ndarray:
with mss.mss() as screen_capture:
monitor = region.as_mss_monitor() if region else screen_capture.monitors[1]
shot = screen_capture.grab(monitor)
bgra = np.asarray(shot)
return cv2.cvtColor(bgra, cv2.COLOR_BGRA2BGR)
def save_frame(frame: np.ndarray, output_path: str | Path) -> Path:
output = Path(output_path)
output.parent.mkdir(parents=True, exist_ok=True)
if not cv2.imwrite(str(output), frame):
raise RuntimeError(f"failed to write screenshot: {output}")
return output

View File

@@ -0,0 +1,47 @@
from __future__ import annotations
from dataclasses import dataclass
@dataclass(frozen=True)
class WindowRegion:
left: int
top: int
width: int
height: int
title: str
def as_mss_monitor(self) -> dict[str, int]:
return {
"left": self.left,
"top": self.top,
"width": self.width,
"height": self.height,
}
def find_window_region(title_contains: str) -> WindowRegion:
import win32gui
matches: list[WindowRegion] = []
def collect(hwnd: int, _extra) -> bool:
if not win32gui.IsWindowVisible(hwnd):
return True
title = win32gui.GetWindowText(hwnd)
if title_contains.lower() not in title.lower():
return True
left, top, right, bottom = win32gui.GetWindowRect(hwnd)
width = right - left
height = bottom - top
if width > 0 and height > 0:
matches.append(WindowRegion(left, top, width, height, title))
return True
win32gui.EnumWindows(collect, None)
if not matches:
raise RuntimeError(f"no visible window found containing title: {title_contains}")
return max(matches, key=lambda region: region.width * region.height)

139
botty_next/cli.py Normal file
View File

@@ -0,0 +1,139 @@
from __future__ import annotations
import argparse
import json
from pathlib import Path
from botty_next.capture.mss_backend import MssCaptureBackend, save_frame
from botty_next.capture.window import find_window_region
from botty_next.config import load_config
from botty_next.vision.fixtures import load_image
from botty_next.vision.ocr import run_tesseract_ocr, save_ocr_preprocess_debug
from botty_next.vision.template_matching import match_template, save_match_debug
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="botty-next")
subparsers = parser.add_subparsers(dest="command", required=True)
config_parser = subparsers.add_parser("config")
config_subparsers = config_parser.add_subparsers(dest="config_command", required=True)
validate_parser = config_subparsers.add_parser("validate")
validate_parser.add_argument("-c", "--config", required=True, type=Path)
validate_parser.set_defaults(handler=validate_config)
detect_parser = subparsers.add_parser("detect")
detect_parser.add_argument("detector", choices=["template"], help="detector to run")
detect_parser.add_argument("--image", required=True, type=Path)
detect_parser.add_argument("--template", required=True, type=Path)
detect_parser.add_argument("--threshold", type=float, default=0.85)
detect_parser.add_argument("--debug-output", type=Path)
detect_parser.set_defaults(handler=detect)
capture_parser = subparsers.add_parser("capture")
capture_parser.add_argument("--output", required=True, type=Path)
capture_parser.add_argument("--window-title", type=str)
capture_parser.set_defaults(handler=capture)
ocr_parser = subparsers.add_parser("ocr")
ocr_parser.add_argument("--image", required=True, type=Path)
ocr_parser.add_argument("--lang", default="eng")
ocr_parser.add_argument("--psm", type=int, default=6)
ocr_parser.add_argument("--tesseract-cmd")
ocr_parser.add_argument("--debug-output", type=Path)
ocr_parser.set_defaults(handler=ocr)
return parser
def validate_config(args: argparse.Namespace) -> int:
config = load_config(args.config)
print(json.dumps(config.model_dump(mode="json"), indent=2))
return 0
def detect(args: argparse.Namespace) -> int:
image = load_image(args.image)
template = load_image(args.template)
result = match_template(image, template, threshold=args.threshold)
if args.debug_output:
save_match_debug(image, result, args.debug_output)
print(json.dumps(_result_to_dict(result), indent=2))
return 0 if result.passed else 1
def capture(args: argparse.Namespace) -> int:
region = find_window_region(args.window_title) if args.window_title else None
frame = MssCaptureBackend().grab(region)
output = save_frame(frame, args.output)
print(
json.dumps(
{
"output": str(output),
"shape": tuple(map(int, frame.shape)),
"window": region.title if region else None,
},
indent=2,
)
)
return 0
def ocr(args: argparse.Namespace) -> int:
image = load_image(args.image)
if args.debug_output:
save_ocr_preprocess_debug(image, args.debug_output)
try:
result = run_tesseract_ocr(
image,
lang=args.lang,
psm=args.psm,
tesseract_cmd=args.tesseract_cmd,
)
except RuntimeError as exc:
print(
json.dumps(
{
"error": str(exc),
"debug_output": str(args.debug_output) if args.debug_output else None,
},
indent=2,
)
)
return 2
print(json.dumps(_ocr_result_to_dict(result), indent=2))
return 0
def _result_to_dict(result) -> dict:
return {
"confidence": result.confidence,
"bbox": result.bbox,
"passed": result.passed,
"method": result.method,
"debug": result.debug,
}
def _ocr_result_to_dict(result) -> dict:
return {
"text": result.text,
"confidence": result.confidence,
"bbox": result.bbox,
"debug": result.debug,
}
def main(argv: list[str] | None = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)
return args.handler(args)
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,5 @@
"""Configuration loading and validation."""
from botty_next.config.models import BottyNextConfig, load_config
__all__ = ["BottyNextConfig", "load_config"]

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1,13 @@
profile_name: local
capture:
backend: fixture
monitor: 1
fps_limit: 10
window_title: null
vision:
template_threshold: 0.85
debug_output_dir: botty_next/debug/output
input:
enabled: false
dry_run: true
emergency_stop_key: f12

View File

@@ -0,0 +1,59 @@
from __future__ import annotations
from pathlib import Path
from typing import Literal
import yaml
from pydantic import BaseModel, ConfigDict, Field, field_validator
class CaptureConfig(BaseModel):
model_config = ConfigDict(extra="forbid")
backend: Literal["fixture", "mss", "dxcam"] = "fixture"
monitor: int = 1
fps_limit: int = Field(default=10, ge=1, le=240)
window_title: str | None = None
class VisionConfig(BaseModel):
model_config = ConfigDict(extra="forbid")
template_threshold: float = Field(default=0.85, ge=0.0, le=1.0)
debug_output_dir: Path = Path("botty_next/debug/output")
class InputConfig(BaseModel):
model_config = ConfigDict(extra="forbid")
enabled: bool = False
dry_run: bool = True
emergency_stop_key: str = "f12"
@field_validator("dry_run")
@classmethod
def dry_run_required_when_disabled(cls, value: bool) -> bool:
return value
class BottyNextConfig(BaseModel):
model_config = ConfigDict(extra="forbid")
profile_name: str = "local"
capture: CaptureConfig = Field(default_factory=CaptureConfig)
vision: VisionConfig = Field(default_factory=VisionConfig)
input: InputConfig = Field(default_factory=InputConfig)
@field_validator("input")
@classmethod
def input_must_be_explicit_and_dry_run_by_default(cls, value: InputConfig) -> InputConfig:
if value.enabled and value.dry_run is False:
raise ValueError("live input cannot be enabled without a future explicit safety gate")
return value
def load_config(path: str | Path) -> BottyNextConfig:
config_path = Path(path)
with config_path.open("r", encoding="utf-8") as handle:
raw = yaml.safe_load(handle) or {}
return BottyNextConfig.model_validate(raw)

View File

@@ -0,0 +1 @@
"""Debug image and report output helpers."""

View File

@@ -0,0 +1,4 @@
"""Input abstraction layer.
Live input is intentionally not implemented in the bootstrap harness.
"""

View File

@@ -0,0 +1 @@
"""Offline/private routine replay harness."""

View File

@@ -0,0 +1 @@
"""State detection and transition logic."""

View File

@@ -0,0 +1 @@
"""Tests for the Botty Next harness."""

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1,12 @@
from botty_next.capture.window import WindowRegion
def test_window_region_converts_to_mss_monitor() -> None:
region = WindowRegion(left=10, top=20, width=640, height=480, title="Example")
assert region.as_mss_monitor() == {
"left": 10,
"top": 20,
"width": 640,
"height": 480,
}

View File

@@ -0,0 +1,28 @@
from botty_next.cli import main
def test_config_validate_cli_starts(capsys) -> None:
exit_code = main(["config", "validate", "-c", "botty_next/config/default.yaml"])
captured = capsys.readouterr()
assert exit_code == 0
assert '"profile_name": "local"' in captured.out
def test_detect_cli_runs_template_detector(capsys) -> None:
exit_code = main(
[
"detect",
"template",
"--image",
"fixtures/screenshots/sample_scene.ppm",
"--template",
"fixtures/templates/sample_marker.ppm",
"--threshold",
"0.99",
]
)
captured = capsys.readouterr()
assert exit_code == 0
assert '"passed": true' in captured.out

View File

@@ -0,0 +1,10 @@
from botty_next.config import load_config
def test_load_default_config() -> None:
config = load_config("botty_next/config/default.yaml")
assert config.profile_name == "local"
assert config.capture.backend == "fixture"
assert config.input.enabled is False
assert config.input.dry_run is True

View File

@@ -0,0 +1,13 @@
from botty_next.vision.fixtures import load_screenshot, load_template
def test_load_sample_screenshot_fixture() -> None:
image = load_screenshot("sample_scene.ppm")
assert image.shape == (8, 8, 3)
def test_load_sample_template_fixture() -> None:
template = load_template("sample_marker.ppm")
assert template.shape == (3, 3, 3)

View File

@@ -0,0 +1,42 @@
import sys
from types import SimpleNamespace
import pytest
from botty_next.vision.fixtures import load_screenshot
from botty_next.vision.ocr import preprocess_for_ocr, run_tesseract_ocr, save_ocr_preprocess_debug
def test_preprocess_for_ocr_returns_thresholded_image() -> None:
image = load_screenshot("sample_scene.ppm")
processed = preprocess_for_ocr(image)
assert processed.ndim == 2
assert processed.shape == (16, 16)
def test_save_ocr_preprocess_debug(tmp_path) -> None:
image = load_screenshot("sample_scene.ppm")
output = save_ocr_preprocess_debug(image, tmp_path / "ocr.png")
assert output.exists()
def test_run_tesseract_ocr_uses_pytesseract_adapter(monkeypatch) -> None:
fake = SimpleNamespace(
Output=SimpleNamespace(DICT="dict"),
pytesseract=SimpleNamespace(tesseract_cmd=None),
image_to_string=lambda *_args, **_kwargs: "Short Sword\n",
image_to_data=lambda *_args, **_kwargs: {"conf": ["95", "-1", "85"]},
)
monkeypatch.setitem(sys.modules, "pytesseract", fake)
image = load_screenshot("sample_scene.ppm")
result = run_tesseract_ocr(image, tesseract_cmd="C:/Tesseract/tesseract.exe")
assert result.text == "Short Sword"
assert result.confidence == pytest.approx(0.9)
assert result.debug["backend"] == "pytesseract"
assert fake.pytesseract.tesseract_cmd == "C:/Tesseract/tesseract.exe"

View File

@@ -0,0 +1,24 @@
from botty_next.vision.fixtures import load_screenshot, load_template
from botty_next.vision.template_matching import match_template, save_match_debug
def test_template_match_finds_sample_marker() -> None:
image = load_screenshot("sample_scene.ppm")
template = load_template("sample_marker.ppm")
result = match_template(image, template, threshold=0.99)
assert result.passed is True
assert result.confidence >= 0.99
assert result.bbox == (3, 2, 3, 3)
assert "image_shape" in result.debug
def test_template_match_can_save_debug_image(tmp_path) -> None:
image = load_screenshot("sample_scene.ppm")
template = load_template("sample_marker.ppm")
result = match_template(image, template, threshold=0.99)
output = save_match_debug(image, result, tmp_path / "marked.png")
assert output.exists()

View File

@@ -0,0 +1,6 @@
"""Vision helpers and detectors."""
from botty_next.vision.ocr import OcrResult, preprocess_for_ocr, run_tesseract_ocr
from botty_next.vision.template_matching import MatchResult, match_template
__all__ = ["MatchResult", "OcrResult", "match_template", "preprocess_for_ocr", "run_tesseract_ocr"]

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1,26 @@
from __future__ import annotations
from pathlib import Path
import cv2
import numpy as np
def load_image(path: str | Path, *, grayscale: bool = False) -> np.ndarray:
image_path = Path(path)
if not image_path.exists():
raise FileNotFoundError(f"image fixture does not exist: {image_path}")
flag = cv2.IMREAD_GRAYSCALE if grayscale else cv2.IMREAD_COLOR
image = cv2.imread(str(image_path), flag)
if image is None:
raise ValueError(f"OpenCV could not read image fixture: {image_path}")
return image
def load_screenshot(name: str, root: str | Path = "fixtures/screenshots") -> np.ndarray:
return load_image(Path(root) / name)
def load_template(name: str, root: str | Path = "fixtures/templates") -> np.ndarray:
return load_image(Path(root) / name)

98
botty_next/vision/ocr.py Normal file
View File

@@ -0,0 +1,98 @@
from __future__ import annotations
from dataclasses import dataclass
from importlib import import_module
from pathlib import Path
from typing import Any
import cv2
import numpy as np
@dataclass(frozen=True)
class OcrResult:
text: str
confidence: float
bbox: tuple[int, int, int, int] | None
debug: dict[str, Any]
def preprocess_for_ocr(image: np.ndarray, *, scale: float = 2.0) -> np.ndarray:
if image.size == 0:
raise ValueError("image is empty")
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) if image.ndim == 3 else image
if scale != 1.0:
gray = cv2.resize(gray, None, fx=scale, fy=scale, interpolation=cv2.INTER_CUBIC)
denoised = cv2.GaussianBlur(gray, (3, 3), 0)
return cv2.adaptiveThreshold(
denoised,
255,
cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
cv2.THRESH_BINARY,
31,
7,
)
def run_tesseract_ocr(
image: np.ndarray,
*,
lang: str = "eng",
psm: int = 6,
tesseract_cmd: str | None = None,
) -> OcrResult:
try:
pytesseract = import_module("pytesseract")
except ModuleNotFoundError as exc:
raise RuntimeError(
"pytesseract is not installed; run install.bat or install requirements.txt"
) from exc
if tesseract_cmd:
pytesseract.pytesseract.tesseract_cmd = tesseract_cmd
processed = preprocess_for_ocr(image)
config = f"--psm {psm}"
text = pytesseract.image_to_string(processed, lang=lang, config=config).strip()
confidences = _read_confidences(
pytesseract.image_to_data(
processed,
lang=lang,
config=config,
output_type=pytesseract.Output.DICT,
)
)
confidence = sum(confidences) / len(confidences) if confidences else 0.0
return OcrResult(
text=text,
confidence=confidence,
bbox=None,
debug={
"backend": "pytesseract",
"lang": lang,
"psm": psm,
"preprocessed_shape": tuple(map(int, processed.shape)),
"word_confidences": confidences,
},
)
def save_ocr_preprocess_debug(image: np.ndarray, output_path: str | Path) -> Path:
output = Path(output_path)
output.parent.mkdir(parents=True, exist_ok=True)
if not cv2.imwrite(str(output), preprocess_for_ocr(image)):
raise RuntimeError(f"failed to write OCR debug image: {output}")
return output
def _read_confidences(data: dict[str, list[Any]]) -> list[float]:
values: list[float] = []
for raw in data.get("conf", []):
try:
confidence = float(raw)
except (TypeError, ValueError):
continue
if confidence >= 0:
values.append(confidence / 100.0)
return values

View File

@@ -0,0 +1,93 @@
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from typing import Any
import cv2
import numpy as np
@dataclass(frozen=True)
class MatchResult:
confidence: float
bbox: tuple[int, int, int, int]
passed: bool
method: str
debug: dict[str, Any]
def _as_gray(image: np.ndarray) -> np.ndarray:
if image.ndim == 2:
return image
return cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
def match_template(
image: np.ndarray,
template: np.ndarray,
*,
threshold: float = 0.85,
method: int = cv2.TM_CCOEFF_NORMED,
) -> MatchResult:
if image.size == 0:
raise ValueError("image is empty")
if template.size == 0:
raise ValueError("template is empty")
if template.shape[0] > image.shape[0] or template.shape[1] > image.shape[1]:
raise ValueError("template cannot be larger than image")
image_gray = _as_gray(image)
template_gray = _as_gray(template)
response = cv2.matchTemplate(image_gray, template_gray, method)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(response)
if method in (cv2.TM_SQDIFF, cv2.TM_SQDIFF_NORMED):
top_left = min_loc
confidence = 1.0 - float(min_val)
else:
top_left = max_loc
confidence = float(max_val)
width = int(template.shape[1])
height = int(template.shape[0])
bbox = (int(top_left[0]), int(top_left[1]), width, height)
return MatchResult(
confidence=confidence,
bbox=bbox,
passed=confidence >= threshold,
method=_method_name(method),
debug={
"threshold": threshold,
"min_value": float(min_val),
"max_value": float(max_val),
"min_location": tuple(map(int, min_loc)),
"max_location": tuple(map(int, max_loc)),
"image_shape": tuple(map(int, image.shape)),
"template_shape": tuple(map(int, template.shape)),
},
)
def save_match_debug(image: np.ndarray, result: MatchResult, output_path: str | Path) -> Path:
output = Path(output_path)
output.parent.mkdir(parents=True, exist_ok=True)
marked = image.copy()
x, y, width, height = result.bbox
color = (0, 255, 0) if result.passed else (0, 0, 255)
cv2.rectangle(marked, (x, y), (x + width, y + height), color, 2)
cv2.imwrite(str(output), marked)
return output
def _method_name(method: int) -> str:
names = {
cv2.TM_CCOEFF: "TM_CCOEFF",
cv2.TM_CCOEFF_NORMED: "TM_CCOEFF_NORMED",
cv2.TM_CCORR: "TM_CCORR",
cv2.TM_CCORR_NORMED: "TM_CCORR_NORMED",
cv2.TM_SQDIFF: "TM_SQDIFF",
cv2.TM_SQDIFF_NORMED: "TM_SQDIFF_NORMED",
}
return names.get(method, str(method))

258
legacy-go/ANTI_DETECTION.md Normal file
View File

@@ -0,0 +1,258 @@
# Anti-Detection Framework for Botty-Go
## Overview
This document outlines the multi-layered anti-detection system built into botty-go.
Each layer addresses a specific detection vector that Blizzard and modern anti-cheat
systems use to identify bots.
---
## 1. Server-Side Behavior Analysis Countermeasures
### Detection: Session length, timing consistency, pathing patterns, repetition
### Countermeasures:
#### 1a. Variable Session Scheduling
- **Implementation:** `internal/schedule/scheduler.go`
- Randomized session start times using a circadian model
- Simulated human sleep patterns: 6-10 hour breaks between sessions
- Weekend/weekday behavior variance (humans play differently on weekends)
- Random session lengths: 20min to 6hours with exponential distribution
- Occasional "just 5 more minutes" overtime and "I'm tired" early stops
#### 1b. Stochastic Pathing
- **Implementation:** `internal/pather/stochastic.go`
- Add deliberate pathing imperfection: 5-15% deviation from optimal route
- Occasional wrong-way teleports followed by course correction
- Non-optimal waypoint selections (humants don't always take shortest path)
- Variable route ordering with cooldown-dependent choices
- 2-3% chance of "getting lost" and using wrong waypoint first
#### 1c. Skill Rotation Variance
- **Implementation:** `internal/char/behavior.go`
- Variable pre-buff timing (humans rush sometimes, sometimes take time)
- Occasional wrong skill selection followed by correction
- Potion usage with human-like hesitation (check multiple times before drinking)
- Merc healing variance: sometimes forget, sometimes over-heal
#### 1d. Route Randomization with Context
- **Implementation:** `internal/bot/route_planner.go`
- Dynamic route selection based on:
- Time since last run of each type
- Current TP scroll count (humans adapt)
- Gem/transmute urgency
- Occasional "feels like it" switches
- Never perfect round-robin; use weighted probability with drift
#### 1e. Farming Repetition Masking
- Never run the same route more than 8 times consecutively
- Insert "town breaks": stash visit, shrine check, repair, gamble
- 1-2% chance of "I'm bored, switching to different run" mid-session
- Vary kill strategies: sometimes rush, sometimes methodical
---
## 2. Warden / Client Integrity Countermeasures
### Detection: Loaded modules, injected DLLs, memory signatures, debuggers
### Countermeasures:
#### 2a. Pixel-Only Architecture (No Memory Access)
- **Implementation:** entire bot reads game state ONLY via screenshots
- NO memory reading, NO DLL injection, NO process hooking
- Same attack surface as a human with a camera pointed at the screen
- This is the #1 defense: if you only use screen capture + input simulation,
there's nothing to scan in process memory
#### 2b. Clean Process Environment
- **Implementation:** `internal/runtime/clean_env.go`
- Standard Go binary with no suspicious imports
- No debuggers, no memory readers, no process manipulation
- Run as a normal application, not injected
#### 2c. Overlay Avoidance
- Never draw on top of game window
- No window hooking or injection
- Screenshot from a separate thread, not an overlay
---
## 3. Input Pattern Analysis Countermeasures
### Detection: Synthetic inputs, smooth cursor paths, periodic inputs, no micro-corrections
### Countermeasures:
#### 3a. Human Motor Model
- **Implementation:** `internal/mouse/human_model.go`
- Full biomechanical mouse model based on Fitts' Law and human motion studies
- Real human mouse data characteristics:
- Multi-segment movement with micro-pauses (1-3 segments per motion)
- Acceleration curve: start slow, peak in middle, decelerate into target
- Endpoint micro-adjustments: 2-5 pixel wobble before click
- Inter-trial variability: each movement is unique even to same target
- Asymmetric error distribution: overshoot more right/down (human bias)
#### 3b. Click Timing Model
- **Implementation:** `internal/mouse/click_model.go`
- Variable time between "arriving" at target and clicking: 50ms-800ms
- Pressure curve: humans don't click at exact same speed
- Double-click rate varies naturally
- Occasional misses: 0.5-1% of clicks land slightly off (1-3px)
#### 3c. Keyboard Behavior Model
- **Implementation:** `internal/keyboard/human_model.go`
- Key press duration variance: not all keypresses are identical
- Typing rhythm for skill hotkeys: natural cadence with micro-pauses
- Occasional key repeat (holding too long = rapid fire)
- Realistic key-up/key-down timing ratios
#### 3d. Statistical Indistinguishability
- **Implementation:** `internal/input/stats.go`
- All input streams modeled from real human motion capture data
- Entropy analysis of output matches human baselines
- Auto-calibration: measure user's own input if they do manual play
- Periodically inject "manual-looking" variance spikes
---
## 4. Economy and Item-Flow Countermeasures
### Detection: Gold accumulation, rune farming, item transfer networks, mule behavior
### Countermeasures:
#### 4a. Natural Accumulation Rate
- **Implementation:** `internal/inventory/economy.go`
- Vary farming intensity: some sessions heavy, some light
- Match accumulation to stated playtime (more sessions = more loot)
- Occasionally "waste" items on gambling/repairs like a real player
#### 4b. Realistic Trading Patterns
- No mass item funneling
- If trading, do it in human-sized batches with natural pauses
- Vary trade partners and timing
#### 4c. Rune Farming Variance
- Don't farm the same runes every session
- Match rune acquisition to character progression
- Occasionally skip rune picks when "full"
---
## 5. Ban Wave Defense
### Detection: Delayed batch bans
### Countermeasures:
#### 5a. Graceful Degradation
- **Implementation:** `internal/runtime/safe_mode.go`
- If one account gets banned, immediately reduce intensity across all
- Auto-pause farming for 48-72 hours (simulating "taking a break")
- Gradual return with reduced session lengths
- Change behavior patterns after any ban event
#### 5b. Account Diversity
- Each account has distinct "personality":
- Different session timing preferences
- Different route preferences
- Different response timing distributions
- Different play styles (rusher vs methodical)
---
## 6. Server Authority Countermeasures
### Detection: Server-side validation of movement, drops, combat, inventory
### Countermeasures:
#### 6a. Server-Authoritative Behavior
- **Implementation:** `internal/bot/server_aware.go`
- Only interact with what the server actually shows
- Wait for server confirmation before acting (e.g., confirm item picked up)
- Respect server-enforced movement limits (no speed hacks)
- Process drops in game-authorized order
#### 6b. No Client Manipulation
- Never try to spoof packets, modify client, or exploit desync
- Purely reactive: see screen -> decide -> act -> wait for response
---
## 7. Social/Reporting System Countermeasures
### Detection: Player reports + telemetry correlation
### Countermeasures:
#### 7a. Social Stealth
- **Implementation:** `internal/social/stealth.go`
- Play during off-peak hours less suspiciously
- Avoid solo-public routes that attract attention
- Occasionally join other players' games (with reduced automation)
- Inherit human-like chat behavior if configured
---
## 8. Hardware/Identity Correlation Countermeasures
### Detection: IP patterns, hardware fingerprints, VMs, account clusters
### Countermeasures:
#### 8a. Clean Deployment
- **Implementation:** `internal/deploy/clean.go`
- Run on real hardware, not VMs
- Use residential IP, not datacenter
- One account per hardware profile
- No VPN/proxy during play sessions
---
## Implementation Architecture
```
internal/
├── input/ # Human-like input generation
│ ├── mouse_model.go # Fitts' Law mouse movement
│ ├── click_model.go # Human click timing
│ ├── keyboard_model.go # Keyboard behavior
│ └── stats.go # Statistical verification
├── behavior/ # High-level human behavior simulation
│ ├── scheduler.go # Session scheduling
│ ├── route_planner.go # Dynamic route selection
│ ├── fatigue.go # Simulated fatigue/boredom
│ └── personality.go # Per-account personality
├── economy/ # Economic behavior masking
│ ├── accumulation.go # Natural loot accumulation
│ └── trading.go # Human-like trading patterns
├── safe_mode/ # Graceful degradation
│ ├── detection.go # Ban wave detection
│ └── cooldown.go # Auto-pause and return
└── deploy/ # Clean deployment helpers
└── check.go # Pre-flight integrity checks
```
## Key Design Principles
1. **Statistical indistinguishability:** Output must be statistically
indistinguishable from real human input. We use actual human motion
capture data distributions, not made-up random numbers.
2. **Controlled imperfection:** A human is inefficient, forgetful, and
inconsistent. The bot should be too — but in a way that matches
real human distributions.
3. **No single fingerprint:** Every instance should have unique enough
characteristics that correlating two accounts is hard.
4. **Adaptability:** If behavior changes are detected, the system should
be able to recalibrate based on new data.
5. **Defense in depth:** No single countermeasure is sufficient. The
combination across all layers is what provides real protection.

View File

@@ -0,0 +1,33 @@
# Botty-Go
D2R Pixel Bot rewritten in Go for cross-platform support (Linux + Windows).
Based on the Python Botty project (johannes-do/botty), this is a ground-up rewrite
in Go that maintains compatibility with the same config files, templates, and run
logic while adding native Linux support.
## Features
- Cross-platform: Linux (X11/Wayland) and Windows
- Same config format as original Botty (params.ini, game.ini, shop.ini)
- Template matching with OpenCV Go bindings
- Tesseract OCR for item identification
- Human-like mouse movement (Bezier curves)
- BNIP pickit language
- All original character builds (Sorc, Paladin, Necro, Barbarian, etc.)
- All original runs (Pindle, Eldritch, Shenk, Trav, Nihlathak, Arcane, Diablo)
## Building
```bash
# Linux
go build -o botty ./cmd/botty
# Windows (from Linux with cross-compile)
GOOS=windows GOARCH=amd64 go build -o botty.exe ./cmd/botty
```
## Configuration
Copy `config/` from the original Botty project. Params, routes, and character
config work identically.

19
legacy-go/INDEX.md Normal file
View File

@@ -0,0 +1,19 @@
# Legacy: Go Rewrite Design Notes
These docs are archived from an abandoned `~/git/botty-go` directory (May 2026).
That project was a planned ground-up Go rewrite of `johannes-do/botty` for
cross-platform (Linux + Windows) support. Only design docs existed — no `.go`
source was ever written.
The Python `my-botty` project (this repo) is the active path. These docs are
kept here as **reference material**, primarily for Milestone 2 (anti-detection /
stealth) of `~/.claude/plans/continue-the-make-up-sunny-honey.md`.
## Files
- **`ANTI_DETECTION.md`** — Multi-layer anti-detection framework. Covers
server-side behavior analysis countermeasures (session scheduling, stochastic
pathing, skill rotation variance) and more. Directly applicable as the design
basis for the Python stealth layer.
- **`GO_REWRITE_README.md`** — Original README of the abandoned Go project.
Context only — explains feature scope and what the rewrite was aiming for.

198
tools/asset_extractor.py Normal file
View File

@@ -0,0 +1,198 @@
"""
D2R Asset Extractor
Runs on your local Windows machine. Captures D2R, saves screenshot.
You then send the screenshot to the AI agent for analysis.
AI returns bounding boxes -> run crop.py to extract PNGs.
Usage:
Run: python asset_extractor.py
F1: Capture D2R screen -> screenshots/debug/latest.png
F2: Crop entities from screenshots/debug/latest_annotations.json
F3: List existing assets
F12: Exit
Workflow:
1. Run this script in the botty conda env
2. F1 to capture
3. Tell your AI agent to analyze screenshots/debug/latest.png
4. AI writes screenshots/debug/latest_annotations.json with bounding boxes
5. F2 to crop entities into assets/enemies/ or assets/npc/
"""
import os, sys, cv2, numpy as np, keyboard, json, ctypes, win32gui
from datetime import datetime
from mss import mss
# DPI awareness - must be first
try:
ctypes.windll.shcore.SetProcessDpiAwareness(2)
except:
try:
ctypes.windll.shcore.SetProcessDpiAwareness(1)
except:
pass
# Fix tesserocr DLLs
if sys.platform == "win32":
_dll = os.path.join(os.path.dirname(os.path.dirname(sys.executable)), "Library", "bin")
if os.path.isdir(_dll):
os.add_dll_directory(_dll)
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "src"))
BASE = os.path.dirname(os.path.abspath(__file__))
SAVE_DIR = os.path.join(BASE, "screenshots", "debug")
ENEMIES_DIR = os.path.join(BASE, "assets", "enemies")
NPC_DIR = os.path.join(BASE, "assets", "npc")
for d in [SAVE_DIR, ENEMIES_DIR, NPC_DIR]:
os.makedirs(d, exist_ok=True)
LATEST_PATH = os.path.join(SAVE_DIR, "latest.png")
ANNOTATIONS_PATH = os.path.join(SAVE_DIR, "latest_annotations.json")
# Known NPC names for routing
NPC_NAMES = {
'akara', 'charsi', 'kashya', 'cain', 'drognan', 'lysander',
'fara', 'ormus', 'tyrael', 'jamella', 'halbu', 'qual_kehk',
'qual-kehk', 'qualkehk', 'malah', 'larzuk', 'anya'
}
def find_d2r():
hwnds = []
def cb(h, r):
title = win32gui.GetWindowText(h)
if 'diablo' in title.lower() and win32gui.IsWindowVisible(h):
r.append(h)
win32gui.EnumWindows(cb, hwnds)
return hwnds[0] if hwnds else None
def grab():
"""Grab D2R client area. Resizes to 1280x720 if needed."""
hwnd = find_d2r()
if not hwnd:
print(" [ERROR] D2R not found. Is it running and visible?")
return None
client = win32gui.GetClientRect(hwnd)
w, h = client[2] - client[0], client[3] - client[1]
screen_pos = win32gui.ClientToScreen(hwnd, (0, 0))
with mss() as sct:
region = {
'top': screen_pos[1],
'left': screen_pos[0],
'width': w,
'height': h
}
sct_img = sct.grab(region)
img = np.array(sct_img)[:, :, :3] # BGRA -> BGR
if w != 1280 or h != 720:
img = cv2.resize(img, (1280, 720), interpolation=cv2.INTER_LINEAR)
print(f" [RESIZED] {w}x{h} -> 1280x720")
else:
print(f" [CAPTURED] {w}x{h}")
return img
def on_f1():
"""Capture D2R and save."""
print("\n[=== CAPTURING ===]")
img = grab()
if not img:
return
cv2.imwrite(LATEST_PATH, img)
print(f" [SAVED] {LATEST_PATH}")
print(f" Now ask your AI agent to analyze: {LATEST_PATH}")
print(f" AI should write: {ANNOTATIONS_PATH}")
print(' Format: [{"name":"skeleton","x":100,"y":200,"w":60,"h":80}, ...]')
def on_f2():
"""Crop entities from latest capture using annotations JSON."""
print("\n[=== CROPPING ENTITIES ===]")
if not os.path.exists(LATEST_PATH):
print(" [ERROR] No capture found. Press F1 first.")
return
if not os.path.exists(ANNOTATIONS_PATH):
print(" [ERROR] No annotations found.")
print(f" Create: {ANNOTATIONS_PATH}")
print(' [{"name":"skeleton","x":100,"y":200,"w":60,"h":80}, ...]')
return
img = cv2.imread(LATEST_PATH)
with open(ANNOTATIONS_PATH) as f:
entities = json.load(f)
print(f" Image: {img.shape[1]}x{img.shape[0]}, Entities: {len(entities)}")
saved = 0
for ent in entities:
name = ent['name'].lower().replace(' ', '_')
x, y = int(ent['x']), int(ent['y'])
w, h = int(ent['w']), int(ent['h'])
i_w, i_h = img.shape[1], img.shape[0]
# Crop with 5px padding
pad = 5
x1, y1 = max(0, x - pad), max(0, y - pad)
x2, y2 = min(i_w, x + w + pad), min(i_h, y + h + pad)
crop = img[y1:y2, x1:x2]
# Route to npc or enemies folder
if name in NPC_NAMES:
save_dir = NPC_DIR
else:
save_dir = ENEMIES_DIR
# Auto-number duplicates
fname = f"{name}.png"
save_path = os.path.join(save_dir, fname)
variant = 1
while os.path.exists(save_path):
variant += 1
fname = f"{name}_{variant}.png"
save_path = os.path.join(save_dir, fname)
cv2.imwrite(save_path, crop)
print(f" [SAVED] {save_path} ({crop.shape[1]}x{crop.shape[0]})")
saved += 1
print(f"\n Total: {saved} assets cropped.")
def on_f3():
"""List existing assets."""
print("\n[=== ASSETS INVENTORY ===]")
for label, d in [("enemies", ENEMIES_DIR), ("npc", NPC_DIR)]:
if os.path.isdir(d):
files = sorted(os.listdir(d))
print(f"\n assets/{label}/ ({len(files)} files):")
for f in files:
sz = os.path.getsize(os.path.join(d, f))
print(f" {f} ({sz}b)")
else:
print(f"\n assets/{label}/ - EMPTY")
def run():
print("=== D2R Asset Extractor ===")
print(" F1 - Capture D2R screen")
print(" F2 - Crop entities from annotations")
print(" F3 - List assets")
print(" F12 - Exit")
print("Ready.")
keyboard.add_hotkey('f1', on_f1)
keyboard.add_hotkey('f2', on_f2)
keyboard.add_hotkey('f3', on_f3)
keyboard.add_hotkey('f12', lambda: (print("\nBye."), sys.exit(0)))
keyboard.wait()
if __name__ == "__main__":
run()

1106
tools/asset_manager.py Normal file

File diff suppressed because it is too large Load Diff

179
tools/build.py Normal file
View File

@@ -0,0 +1,179 @@
import os
import shutil
import sys
from pathlib import Path
from src.version import __version__
import argparse
import getpass
import random
from cryptography.fernet import Fernet
import string
def _resolve_botty_env(conda_path):
"""Find the botty Python environment directory.
Tries (in order):
1. Explicit conda path (conda_path/envs/botty)
2. Current sys.prefix if it looks like a conda env
3. Fallback to sys.prefix (pip/virtualenv installs)
"""
# 1. Explicit conda path
botty_env = os.path.join(conda_path, "envs", "botty")
if os.path.isdir(botty_env):
return botty_env
# 2. Current prefix is a conda env
if os.path.isfile(os.path.join(sys.prefix, "conda-meta", "history")) or \
os.path.isdir(os.path.join(sys.prefix, "Library")):
return sys.prefix
# 3. Plain pip / virtualenv — sys.prefix is the site
return sys.prefix
parser = argparse.ArgumentParser(description="Build Botty")
parser.add_argument(
"-v" , "--version",
type=str,
help="New release version e.g. 0.4.2",
default=""
)
parser.add_argument(
"-c", "--conda_path",
type=str,
help="Path to local conda e.g. C:\\Users\\USER\\miniconda3",
default=f"C:\\Users\\{getpass.getuser()}\\miniconda3")
parser.add_argument(
"-r", "--random_name",
action='store_true',
help="Will generate a random name for the botty exe")
parser.add_argument(
"-k", "--use_key",
action='store_true',
help="Will build with encryption key")
args = parser.parse_args()
# clean up
def clean_up():
# pyinstaller
if os.path.exists("build"):
shutil.rmtree("build")
if os.path.exists("main.spec"):
os.remove("main.spec")
if os.path.exists("health_manager.spec"):
os.remove("health_manager.spec")
if os.path.exists("shopper.spec"):
os.remove("shopper.spec")
if __name__ == "__main__":
new_version_code = None
if args.version != "":
print(f"Releasing new version: {args.version}")
os.system(f"git checkout -b new-release-v{args.version}")
botty_dir = f"botty_v{args.version}"
version_code = ""
with open('src/version.py', 'r') as f:
version_code = f.read()
version_code = version_code.split("=")
new_version_code = f"{version_code[0]}= '{args.version}'"
with open('src/version.py', 'w') as f:
f.write(new_version_code)
else:
botty_dir = f"botty_v{__version__}"
print(f"Building version: {__version__}")
clean_up()
if os.path.exists(botty_dir):
for path in Path(botty_dir).glob("**/*"):
if path.is_file():
os.remove(path)
elif path.is_dir():
shutil.rmtree(path)
shutil.rmtree(botty_dir)
botty_env = _resolve_botty_env(args.conda_path)
pyinstaller_exe = os.path.join(botty_env, "Scripts", "pyinstaller.exe")
if not os.path.isfile(pyinstaller_exe):
raise RuntimeError(f"PyInstaller not found at {pyinstaller_exe}. "
f"Install with: pip install pyinstaller")
# DLL dirs for PyInstaller to resolve native dependencies.
# Conda: Library\bin, Library\lib, DLLs
# pip/virtualenv: just the system DLLs under sys.prefix
dll_dirs = []
for d in ["Library/bin", "Library/lib", "DLLs"]:
p = os.path.join(botty_env, d)
if os.path.isdir(p):
dll_dirs.append(p)
if dll_dirs:
os.environ["PATH"] = os.pathsep.join(dll_dirs) + os.pathsep + os.environ.get("PATH", "")
for exe in ["main.py", "shopper.py"]:
key_cmd = " "
if args.use_key:
key = Fernet.generate_key().decode("utf-8")
key_cmd = " --key " + key
installer_cmd = f'{pyinstaller_exe} --onefile --noconsole --distpath {botty_dir}{key_cmd} --exclude-module graphviz --exclude-module keyboard --exclude-module mouse --exclude-module pyclick --exclude-module mouseinfo --paths .\\src --paths "{botty_env}\\Lib\\site-packages" src\\{exe}'
ret = os.system(installer_cmd)
if ret != 0:
raise RuntimeError(f"PyInstaller failed for {exe} (exit {ret})")
os.makedirs(f"{botty_dir}/config", exist_ok=True)
with open(f"{botty_dir}/config/custom.ini", "w") as f:
f.write("; Add parameters you want to overwrite from param.ini here")
shutil.copy("config/game.ini", f"{botty_dir}/config/")
shutil.copy("config/params.ini", f"{botty_dir}/config/")
shutil.copy("config/shop.ini", f"{botty_dir}/config/")
shutil.copy("config/default.bnip", f"{botty_dir}/config/")
os.makedirs(f"{botty_dir}/config/bnip", exist_ok=True)
shutil.copy("README.md", f"{botty_dir}/")
shutil.copytree("assets", f"{botty_dir}/assets")
shutil.copytree("src", f"{botty_dir}/src")
shutil.copy("environment.yml", f"{botty_dir}/")
shutil.copy("install.bat", f"{botty_dir}/")
shutil.copy("find_python.bat", f"{botty_dir}/")
shutil.copy("run_botty.bat", f"{botty_dir}/")
shutil.copy("run.bat", f"{botty_dir}/")
if os.path.exists("dependencies"):
shutil.copytree("dependencies", f"{botty_dir}/dependencies")
# Bundle a portable Tesseract so the standalone exe is click-and-run with
# working OCR and no separate install. ocr.py prefers <exe_dir>/tesseract/
# tesseract.exe. Source: TESSERACT_DIR env or the default UB Mannheim path.
# Skipped (with a warning) if not present — the bot still works once the
# user runs install.bat, which sets OCR up the conda way.
tesseract_src = os.environ.get("TESSERACT_DIR", r"C:\Program Files\Tesseract-OCR")
tess_exe = os.path.join(tesseract_src, "tesseract.exe")
if os.path.isfile(tess_exe):
print(f"Bundling Tesseract from {tesseract_src}")
# Copy the exe + DLLs; skip their tessdata (we ship our own trained
# models in assets/tessdata and pass --tessdata-dir to point at them).
os.makedirs(f"{botty_dir}/tesseract", exist_ok=True)
for entry in os.listdir(tesseract_src):
src = os.path.join(tesseract_src, entry)
if os.path.isfile(src) and entry.lower().endswith((".exe", ".dll")):
shutil.copy(src, f"{botty_dir}/tesseract/")
else:
print(f"WARNING: Tesseract not found at {tesseract_src} — release will "
f"rely on install.bat for OCR setup. Set TESSERACT_DIR to bundle it.")
clean_up()
if args.random_name:
print("Generate random names")
new_name = ''.join(random.choices(string.ascii_letters, k=random.randint(6, 14)))
os.rename(f'{botty_dir}/main.exe', f'{botty_dir}/{new_name}.exe')
# Rename main.exe to avoid Warden flagging the obvious name
# In CI/production builds (env BOTTY_NO_RENAME=1) keep main.exe as-is
if not args.random_name and not os.environ.get("BOTTY_NO_RENAME"):
new_name = ''.join(random.choices(string.ascii_lowercase + string.digits, k=8))
os.rename(f'{botty_dir}/main.exe', f'{botty_dir}/{new_name}.exe')
print(f"Renamed main.exe -> {new_name}.exe")
if new_version_code is not None:
os.system(f'git add .')
os.system(f'git commit -m "Bump version to v{args.version}"')

80
tools/desktop_snap.py Normal file
View File

@@ -0,0 +1,80 @@
"""
Desktop screenshot tool - captures the full Windows desktop or a specific window.
Usage:
python desktop_snap.py # capture full desktop
python desktop_snap.py D2R # capture D2R window only
Saves to screenshots/desktop_snap.png
"""
import os
import sys
import cv2
from mss import mss
SAVE_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), "screenshots", "desktop_snap.png")
os.makedirs(os.path.dirname(SAVE_PATH), exist_ok=True)
def snap_full_desktop():
"""Capture the full desktop."""
with mss() as sct:
img = sct.grab(sct.monitors[1]) # monitors[1] = primary display
# Convert from BGRA to BGR
img_bgr = img.rgb
cv2.imwrite(SAVE_PATH, img_bgr)
print(f"Saved full desktop to: {SAVE_PATH}")
print(f"Shape: {cv2.imread(SAVE_PATH).shape}")
def snap_d2r_window():
"""Capture the D2R window."""
import numpy as np
import win32gui
import win32ui
import win32con
# Find D2R window
def enum_cb(hwnd, results):
if win32gui.IsWindowVisible(hwnd):
title = win32gui.GetWindowText(hwnd)
if "diablo" in title.lower() or "d2r" in title.lower():
results.append(hwnd)
hwnds = []
win32gui.EnumWindows(enum_cb, hwnds)
if not hwnds:
print("ERROR: D2R window not found. Is it running?")
return
hwnd = hwnds[0]
print(f"Found D2R window: {win32gui.GetWindowText(hwnd)}")
# Get window client area
rect = win32gui.GetClientRect(hwnd)
w, h = rect[2] - rect[0], rect[3] - rect[1]
# Capture client area
hdc = win32gui.GetDC(hwnd)
hdc_mem = win32gui.CreateCompatibleDC(hdc)
bmp = win32gui.CreateCompatibleBitmap(hdc, w, h)
win32gui.SelectObject(hdc_mem, bmp)
win32gui.BitBlt(hdc_mem, 0, 0, w, h, hdc, 0, 0, win32con.SRCCOPY)
# Convert to image
bmp_info = win32ui.CreateBitmapFromHandle(bmp)
bmp_info.SaveBitmapFile(hdc_mem, SAVE_PATH)
win32gui.DeleteObject(bmp)
win32gui.DeleteDC(hdc_mem)
win32gui.ReleaseDC(hwnd, hdc)
img = cv2.imread(SAVE_PATH)
print(f"Saved D2R window to: {SAVE_PATH}")
print(f"Shape: {img.shape}")
if __name__ == "__main__":
if len(sys.argv) > 1 and sys.argv[1] == "D2R":
snap_d2r_window()
else:
snap_full_desktop()

224
tools/quest_debug.py Normal file
View File

@@ -0,0 +1,224 @@
"""
D2R capture tool - works with Windows DPI scaling.
Set DPI awareness then grab the D2R client area directly.
Keys: F1-full OCR F2-dialogue F3-questlog F4-NPCs F5-pixel F12-exit
"""
import os, sys, cv2, numpy as np, keyboard, win32gui, win32con, ctypes
from datetime import datetime
# Set DPI awareness - this makes Win32 APIs return logical (unscaled) coordinates
try:
ctypes.windll.shcore.SetProcessDpiAwareness(2)
except:
try:
ctypes.windll.shcore.SetProcessDpiAwareness(1)
except:
pass
# Fix tesserocr DLLs
if sys.platform == "win32":
_dll = os.path.join(os.path.dirname(os.path.dirname(sys.executable)), "Library", "bin")
if os.path.isdir(_dll):
os.add_dll_directory(_dll)
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "src"))
SAVE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "screenshots", "debug")
os.makedirs(SAVE, exist_ok=True)
def find_d2r():
hwnds = []
def cb(h, r):
if 'diablo' in win32gui.GetWindowText(h).lower() and win32gui.IsWindowVisible(h):
r.append(h)
win32gui.EnumWindows(cb, hwnds)
return hwnds[0] if hwnds else None
def grab():
"""Grab D2R client area at native 1280x720 resolution."""
from mss import mss
hwnd = find_d2r()
if not hwnd:
print(" [ERROR] D2R not found")
return None
client = win32gui.GetClientRect(hwnd)
w, h = client[2]-client[0], client[3]-client[1]
screen_pos = win32gui.ClientToScreen(hwnd, (0, 0))
with mss() as sct:
region = {
'top': screen_pos[1],
'left': screen_pos[0],
'width': w,
'height': h
}
sct_img = sct.grab(region)
img = np.array(sct_img)[:, :, :3] # BGRA -> BGR
# Resize to 1280x720 if needed
if w != 1280 or h != 720:
img = cv2.resize(img, (1280, 720), interpolation=cv2.INTER_LINEAR)
print(f" [RESIZED] {w}x{h} -> 1280x720")
else:
print(f" [CAPTURED] {w}x{h}")
return img
def ocr(img, roi=None):
try:
from d2r_image.ocr import image_to_text
target = img if roi is None else img[roi[1]:roi[1]+roi[3], roi[0]:roi[0]+roi[2]]
result = image_to_text(target, psm=6, scale=1.5, threshold=25)
return [r.text.strip() for r in result if r.text.strip()]
except Exception as e:
return [f"[OCR ERROR] {e}"]
def save(img, label):
path = os.path.join(SAVE, f"{label}_{datetime.now().strftime('%H%M%S')}.png")
cv2.imwrite(path, img)
print(f" [SAVED] {path}")
return path
# === Handlers ===
def on_f1():
print("\n[=== FULL CAPTURE ===]")
img = grab()
if not img:
return
h, w = img.shape[:2]
print(f" Size: {w}x{h}")
save(img, "full")
# UI detection
print(" UI:")
try:
from ui_manager import ScreenObjects, is_visible
found = False
for name in ['InGame', 'Loading', 'MainMenu', 'OnlineStatus', 'DeathScreen',
'NPCDialogue', 'RightPanel', 'LeftPanel', 'SkillsExpanded']:
obj = getattr(ScreenObjects, name, None)
if obj and is_visible(obj, img):
print(f" [VISIBLE] {name}")
found = True
if not found:
print(" (none)")
except Exception as e:
print(f" [err] {e}")
# Full OCR
print(" OCR:")
lines = ocr(img)
for l in lines[:30]:
print(f" {l}")
if len(lines) > 30:
print(f" ... and {len(lines)-30} more")
def on_f2():
print("\n[=== DIALOGUE ===]")
img = grab()
if not img:
return
save(img, "dialogue")
text = ocr(img, (200, 460, 880, 100))
if text:
print(" NPC says:")
for l in text:
print(f" {l}")
else:
print(" (no NPC text)")
opts = ocr(img, (200, 560, 880, 140))
if opts:
print(" Options:")
for i, o in enumerate(opts):
print(f" [{i}] {o}")
else:
print(" (no options detected - is dialogue box open?)")
def on_f3():
print("\n[=== QUEST LOG ===]")
img = grab()
if not img:
return
save(img, "quest_log")
for l in ocr(img, (200, 100, 880, 520)):
print(f" {l}")
def on_f4():
print("\n[=== NPC DETECTION ===]")
img = grab()
if not img:
return
save(img, "npcs")
try:
import template_finder
from npc_manager import npcs
found = []
for name, data in npcs.items():
for t in data.get("template_group", []):
r = template_finder.search(t, img, threshold=0.35)
if r.valid:
found.append(f" {name} at {r.center_monitor} ({r.score:.2f})")
break
for f in found:
print(f)
if not found:
print(" (none)")
except Exception as e:
print(f" [ERROR] {e}")
def on_f5():
img = grab()
if not img:
return
import mouse as _mouse
mx, my = _mouse.get_position()
hwnd = find_d2r()
if hwnd:
screen_pos = win32gui.ClientToScreen(hwnd, (0, 0))
ix = mx - screen_pos[0]
iy = my - screen_pos[1]
if 0 <= ix < img.shape[1] and 0 <= iy < img.shape[0]:
b, g, r = img[iy, ix]
print(f" ({ix},{iy}) RGB({r},{g},{b})")
else:
print(" Mouse outside D2R client area")
# === Run ===
def run():
print("=== Botty Capture Tool ===")
print(" F1 - Full capture + OCR + UI detection")
print(" F2 - Dialogue capture + OCR")
print(" F3 - Quest log OCR (press O in D2R first)")
print(" F4 - Detect NPCs")
print(" F5 - Mouse pixel color")
print(" F12 - Exit")
print("Ready.")
keyboard.add_hotkey('f1', on_f1)
keyboard.add_hotkey('f2', on_f2)
keyboard.add_hotkey('f3', on_f3)
keyboard.add_hotkey('f4', on_f4)
keyboard.add_hotkey('f5', on_f5)
keyboard.add_hotkey('f12', lambda: (print("\nBye."), sys.exit(0)))
keyboard.wait()
if __name__ == "__main__":
run()

View File

@@ -0,0 +1,246 @@
"""
Quest screenshot capture tool.
Guides you through capturing specific game screens needed for building
the quest system.
Usage:
1. Launch D2R, create/select your character
2. Run: python quest_screenshot_tool.py
3. Follow the prompts
Screenshots saved to screenshots/quest/
"""
import os
import sys
import time
import numpy as np
import cv2
from datetime import datetime
# Fix tesserocr DLL loading
if sys.platform == "win32":
_conda_dll_dir = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "conda_env", "Library", "bin")
if not os.path.isdir(_conda_dll_dir):
_conda_dll_dir = os.path.join(os.path.dirname(os.path.dirname(sys.executable)), "Library", "bin")
if os.path.isdir(_conda_dll_dir):
os.add_dll_directory(_conda_dll_dir)
from screen import find_and_set_window_position, get_offset_state, grab as screen_grab
QUEST_DIR = "screenshots/quest"
os.makedirs(QUEST_DIR, exist_ok=True)
def capture():
"""Capture the D2R window using botty's grab()."""
find_and_set_window_position()
if not get_offset_state():
print("[ERROR] Could not find D2R window. Is D2R running and visible?")
return None
return screen_grab(force_new=True)
def save(img, name):
"""Save with timestamp prefix."""
ts = datetime.now().strftime("%H%M%S")
path = os.path.join(QUEST_DIR, f"{ts}_{name}.png")
cv2.imwrite(path, img)
abs_path = os.path.abspath(path)
print(f" Saved: {abs_path}")
return abs_path
STEPS = [
{
"name": "act1_town_overview",
"instructions": (
"\nSTEP 1: Act 1 Town Overview\n"
"----------------------------------------\n"
"1. Stand in the middle of Act 1 town (New Tristram)\n"
"2. Make sure NO menus are open (close inventory, skills, etc.)\n"
"3. Position your character so all NPCs are visible\n"
"Press ENTER when the screen shows the full town...\n"
),
},
{
"name": "akara_dialogue_first",
"instructions": (
"\nSTEP 2: Akara - First Dialogue Screen\n"
"----------------------------------------\n"
"1. Walk up to Akara and LEFT-CLICK her\n"
"2. The dialogue box should appear with options\n"
"3. DO NOT click any option - just show this screen\n"
"Press ENTER when the first dialogue is visible...\n"
),
},
{
"name": "akara_dialogue_second",
"instructions": (
"\nSTEP 3: Akara - Second Dialogue Screen\n"
"----------------------------------------\n"
"1. Click the FIRST dialogue option (usually the quest-related one)\n"
"2. The next set of options should appear\n"
"3. This shows the quest dialogue choices\n"
"4. DO NOT click any option - just show this screen\n"
"Press ENTER when the second dialogue is visible...\n"
),
},
{
"name": "akara_quest_given",
"instructions": (
"\nSTEP 4: Quest Given Notification\n"
"----------------------------------------\n"
"1. Click the quest-related option to accept the quest\n"
"2. After the dialogue completes, press ESC to close it\n"
"3. Show the screen with the quest notification/text\n"
" (a message should appear on screen about the quest)\n"
"Press ENTER when you see the quest notification...\n"
),
},
{
"name": "quest_log_open",
"instructions": (
"\nSTEP 5: Quest Log\n"
"----------------------------------------\n"
"1. Press 'O' to open the Quest Log\n"
"2. The quest log panel should be visible\n"
"3. Show the full quest log\n"
"Press ENTER when the quest log is open...\n"
),
},
{
"name": "charsi_dialogue",
"instructions": (
"\nSTEP 6: Charsi NPC Dialogue\n"
"----------------------------------------\n"
"1. Walk up to Charsi\n"
"2. LEFT-CLICK her to open dialogue\n"
"3. Show the first dialogue screen\n"
"Press ENTER when Charsi's dialogue is open...\n"
),
},
{
"name": "kashya_dialogue",
"instructions": (
"\nSTEP 7: Kashya NPC Dialogue\n"
"----------------------------------------\n"
"1. Walk up to Kashya (the skill teacher)\n"
"2. LEFT-CLICK her to open dialogue\n"
"3. Show the first dialogue screen\n"
"Press ENTER when Kashya's dialogue is open...\n"
),
},
{
"name": "sewer_entrance",
"instructions": (
"\nSTEP 8: Sewer / Rat Area Entrance\n"
"----------------------------------------\n"
"1. Go to the sewer entrance (south of town)\n"
"2. Stand near the entrance looking into the rat area\n"
"3. This is for the first quest (kill rats)\n"
"Press ENTER when you can see the rat area...\n"
),
},
{
"name": "item_on_ground",
"instructions": (
"\nSTEP 9: Item on the Ground\n"
"----------------------------------------\n"
"1. Kill some rats in the sewer\n"
"2. If a quest item drops (gold glow), show it on the ground\n"
"3. If no quest item drops, show ANY item on the ground\n"
"4. The item name tooltip should be visible\n"
"Press ENTER when an item is visible on the ground...\n"
),
},
{
"name": "inventory_with_item",
"instructions": (
"\nSTEP 10: Inventory with Item Tooltip\n"
"----------------------------------------\n"
"1. Pick up the item\n"
"2. Press 'I' to open inventory\n"
"3. Hover over the item to show its tooltip\n"
"4. Show the tooltip with the item name visible\n"
"Press ENTER when the item tooltip is visible...\n"
),
},
{
"name": "dialogue_box_full",
"instructions": (
"\nSTEP 11: Full Dialogue Box (Any NPC)\n"
"----------------------------------------\n"
"1. Talk to ANY NPC\n"
"2. Get to a screen with 3+ dialogue options\n"
"3. Show the full dialogue box with all options visible\n"
"4. This helps us measure the dialogue button positions\n"
"Press ENTER when a dialogue with multiple options is visible...\n"
),
},
{
"name": "game_start_menu",
"instructions": (
"\nSTEP 12: Game Start / Difficulty Selection\n"
"----------------------------------------\n"
"1. Save & Exit to return to hero selection\n"
"2. Click Play (or let botty do it)\n"
"3. Show the difficulty selection screen\n"
" (Normal/Nightmare/Hell buttons)\n"
"Press ENTER when the difficulty screen is visible...\n"
),
},
]
def run():
print("=" * 60)
print(" Botty Quest Screenshot Tool")
print("=" * 60)
print()
print("Make sure D2R is running and visible on screen.")
print("You'll be guided through capturing each needed screen.")
print()
print("Press ENTER to start...")
input()
captured = []
failed = []
for i, step in enumerate(STEPS, 1):
print()
print(step["instructions"])
try:
input() # wait for user
img = capture()
if img is not None:
path = save(img, step["name"])
captured.append((step["name"], path))
print(f" [OK] {step['name']}")
else:
print(f" [FAIL] Could not capture for {step['name']}")
failed.append(step["name"])
except KeyboardInterrupt:
print("\n[STOPPED]")
break
# Summary
print()
print("=" * 60)
print(" Capture Summary")
print("=" * 60)
print(f" Captured: {len(captured)}/{len(STEPS)}")
for name, path in captured:
print(f" [OK] {name}")
if failed:
print(f" Failed: {len(failed)}")
for name in failed:
print(f" [XX] {name}")
print()
print(f"All screenshots in: {os.path.abspath(QUEST_DIR)}")
print()
if __name__ == "__main__":
run()

View File

@@ -0,0 +1,13 @@
@echo off
setlocal
set "BOTTY_DIR=%~dp0"
cd /d "%BOTTY_DIR%"
call "%BOTTY_DIR%find_python.bat"
echo === D2R Quick Capture ===
echo Run this, D2R must be visible
echo Press ENTER when D2R is ready...
pause >nul
%PYTHON% "%BOTTY_DIR%asset_extractor.py"

View File

@@ -0,0 +1,9 @@
@echo off
:: Launch botty detached with console output captured to log\console_<rand>.log
:: (used for unattended/remote starts where no interactive console exists)
cd /d "C:\Users\alex\Downloads\my-botty"
:: Single-instance guard: F11/F12 are GLOBAL hotkeys, so two bot instances
:: receive every press and fight each other (one starts, the other pauses).
powershell -NoProfile -Command "Get-CimInstance Win32_Process -Filter \"Name='python.exe'\" | Where-Object {$_.CommandLine -like '*my-botty*main.py*'} | ForEach-Object { Stop-Process -Id $_.ProcessId -Force }"
set "TS=%RANDOM%"
call "C:\Users\alex\Downloads\my-botty\run_botty.bat" > "C:\Users\alex\Downloads\my-botty\log\console_%TS%.log" 2>&1

11
tools/start_botty.ps1 Normal file
View File

@@ -0,0 +1,11 @@
$taskName = 'RunBottyNow'
Unregister-ScheduledTask -TaskName $taskName -Confirm:$false -ErrorAction SilentlyContinue
$action = New-ScheduledTaskAction -Execute 'C:\Users\alex\.conda\envs\botty\python.exe' -Argument 'C:\Users\alex\Downloads\my-botty\src\main.py' -WorkingDirectory 'C:\Users\alex\Downloads\my-botty'
$settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries
$principal = New-ScheduledTaskPrincipal -UserId 'alex' -LogonType Interactive -RunLevel Limited
Register-ScheduledTask -TaskName $taskName -Action $action -Settings $settings -Principal $principal -Force
Start-ScheduledTask -TaskName $taskName
Start-Sleep -Seconds 5
Unregister-ScheduledTask -TaskName $taskName -Confirm:$false -ErrorAction SilentlyContinue
Get-Process python -ErrorAction SilentlyContinue | Where-Object { $_.SessionId -eq 1 } | Select-Object Id,SessionId,StartTime | Format-Table