Change default settings (#105)

* Change default settings

* Add another shenk tempalte at start

* fix path

* actually make screenshot with item hovered

README.md

@@ -9,10 +9,10 @@ And please. I urge you to actually read that README! It will make your life a lo
 [![Watch the video](assets/docs/video_thumbnail.png)](https://streamable.com/67h9ay)
 
 ## Getting started
-The bot currently only supports the screen resolutions 1920x1080 and 1280x720. If your monitor has a higher resolution, you need to reduce it to the mentioned once for running the bot.
+Botty only supports English language and screen resolutions 1920x1080 and 1280x720. If your monitor has a higher resolution, you need to either reduce it to one of the two resolutions or set D2R to windowed mode and adjust offset_left and offset_top.
 
 ### 1) Graphics and Gameplay Settings
-All settings will automatically be set when you execute `run.exe` and press the hotkey for "Adjust D2R settings" (default f9). Note that D2R should not run during this process, or if it does you will have to restart afterwards. It is not a 100% thing, in rare cases you might still have to fiddle around with your brightness. I suggest using the "Color Test Mode" to verify your settings.</br>
+All settings will automatically be set when you execute `run.exe` and press the hotkey for "Adjust D2R settings" (default f9). Note that D2R should not run during this process, or if it does you will have to restart afterwards. It is not a 100% thing, in rare cases you might still have to fiddle around with your brightness. I suggest using the "Graphic Debugger" to verify your settings. Also, there are sample screenshots of how graphics should look like: <a href="/assets/docs/sample_graphics.png">Example 1</a>, <a href="/assets/docs/sample_graphics_2.png">Example 2</a></br>
 **Note**: There have been issues reported with image sharpening being truned on via the graphic card settings itself outside of D2R. Try turning it off when running botty.
 
 ### 2) Supported builds
@@ -30,9 +30,9 @@ Open up D2R and wait till you are at the hero selection screen. Make sure the ch
 Download the a prebuilt release [here](https://github.com/aeon0/botty/releases). Start `run.exe` in the botty folder. Switch (ALT+TAB) to your D2R window and press the start key (default f11). You can always force stop botty with f12.
 
 
-## Color Test Mode
-To check if you graphic settings are good and if the bot would pick up items there is a **Color Test Mode** built in. Start botty and press F10 (Default key). This will open up a (mostly black) window. Start a game in D2R and throw some items of different type on the ground. If you now bring forward the debug window all items should show up with their names while the background is black. If you throw an item on the ground that should be picked up, it will have a red circle.</br>
-<img src="assets/docs/color_checker.png" width="400">
+## Graphic Debugger
+To check if you graphic settings are good and if the bot would pick up items there is a **Graphic Debugger Mode** built in. Start botty and press F10 (Default key). This will open up a (mostly black) window. Start a game in D2R and go to A5. You should see some templates with blue circles detected and scores printed out to the console. E.g. for 720p you should see scores higher 0.8 for the templates. To check item finding, throw some items of different types on the ground. The debug window should show the item names with black background. If you throw an item on the ground that should be picked up, it will have a red circle. The console will print out the scores for each item that would be picked up. Scores should be well above 0.9 for these items.</br>
+<img src="assets/docs/graphic_debugger.png" width="900">
 
 ## Development
 Check out the [development.md](development.md) docu for infos on how to build from source and details of the project structure and code.
@@ -56,11 +56,10 @@ monitor | Select on which monitor D2R is running in case multiple are available
 res | Resolution settings can be any of [1920_1080, 1280_720]
 offset_top | Your D2R windows offset from top of the screen (including the window bar). For fullscreen leave at 0.
 offset_left | Your D2R window offset from left of screen. For fullscreen leave at 0.
-min_game_length_s | Games must have at least this length, will wait in hero selection for if game is too quick (to avoid server connection issues)
 max_game_length_s | Botty will attempt to stop whatever its doing and try to restart a new game. Note if this fails, botty will attempt to shut down D2R and Bnet
 exit_key | Pressing this key (anywhere), will force botty to shut down
 resume_key | After starting the exe botty will wait for this keypress to atually start botting away
-color_checker_key | Pressing this key will start a debug mode to check if the color filtering works with your settings. It also includes the item search and marks items it would pick up with red circles
+graphic_debugger_key | Pressing this key will start a debug mode to check if the color filtering works with your settings. It also includes the item search and marks items it would pick up with red circles
 logger_lvl | Can be any of [info, debug] and determines how much output you see on the command line
 randomize_runs | If 0, the order will always be pindle -> eldritch/shenk. If 1 the order will be random.
 difficulty | Set to `normal` `nightmare` or `hell` for game difficulty
@@ -71,6 +70,7 @@ loot_screenshots | If 1, the bot takes a screenshot with timestamp everytime he
  [routes]                       | Descriptions
 --------------------------------|---------------------------------------------
 run_pindle | Run Pindle in each new game. Select "1" to run it "0" to leave it out.
+run_eldritch | Run Eldritch in each new game. Select "1" to run it "0" to leave it out.
 run_shenk | Run shenk in each new game. Select "1" to run it "0" to leave it out.
 
  [char]                         | Descriptions
@@ -78,6 +78,7 @@ run_shenk | Run shenk in each new game. Select "1" to run it "0" to leave it out
 type | Build type. Currently only "sorceress" or "hammerdin" is supported
 casting_frames | Depending on your char and fcr you will have a specific casting frame count. Check it here: https://diablo2.diablowiki.net/Breakpoints and fill in the right number. Determines how much delay there is after each teleport for example.
 slow_walk | With this set to 1 the char will have a large delay for each running action in town. Set this to 1 if you keep getting stuck during traversing town.
+stash_gold | Bool value to stash gold each time when stashing items
 atk_len_pindle | Attack length for hdin or number of attack sequences for sorc when fighting pindle
 atk_len_eldritch | Attack length for hdin or number of attack sequences for sorc when fighting eldritch
 atk_len_shenk | Attack length for hdin or number of attack sequences for sorc when fighting shenk
@@ -91,11 +92,15 @@ show_items | Hotkey for "show items"
 inventory_screen | Hotkey to open up inventory
 stand_still | Hotkey for "stand still". Note this can not be the default shift key as it would interfere with the merc healing routine.
 tp | Hotkey for using a town portal
+belt_rows | Integer value of how many rows the char's belt has
 show_belt | Hotkey for "show belt"
-potion1 | Hotkey to take poition in slot 1
-potion2 | Hotkey to take poition in slot 2
-potion3 | Hotkey to take poition in slot 3
-potion4 | Hotkey to take poition in slot 4
+potion1 | Hotkey to take potion in slot 1
+potion2 | Hotkey to take potion in slot 2
+potion3 | Hotkey to take potion in slot 3
+potion4 | Hotkey to take potion in slot 4
+belt_rejuv_columns | Number of belt columns for rejuv potions
+belt_hp_columns | Number of belt columns for healing potions
+belt_mp_columns | Number of belt columns for mana potions
 cta_available | 0: no cta available, 1: cta is available and should be used during prebuff
 weapon_switch | Hotkey for "weapon switch" (only needed if cta_available=1)
 battle_order | Hotkey for battle order from cta (only needed if cta_available=1)
@@ -121,10 +126,11 @@ blessed_hammer | Hotkey for Blessed Hammer
  [advanced_options]             | Descriptions
 --------------------------------|---------------------------------------------
 pathing_delay_factor | A linear scaling factor, between 1 and 10, applied to pathing delays.
+template_threshold | Threshold to find templates used during pathing.
 
  [items]                        | Descriptions
 --------------------------------|---------------------------------------------
 item_type | 0: Item will not be picked up. 1: Item will be picked up. 2: Item will be picked up and a discord message will be sent.
 
 ## Support this project
-This project is free. Support it by contributing in any technical way, giving feedback, bug reports or submitting PRs.
+Support it by contributing in any technical way, giving feedback, bug reports or submitting PRs.

assets/d2r_settings.json

@@ -5,7 +5,7 @@
     "MaxLuminance": 600,
     "HDRContrast": 200,
     "Contrast": 100,
-    "Screen Resolution (Windowed)": "1920x1051",
+    "Screen Resolution (Windowed)": "1280x720",
     "Resolution Scale": 100,
     "Sharpening": 6,
     "Game Resolution": 1,
@@ -15,7 +15,7 @@
     "VSync": 1,
     "Framerate Cap": 60,
     "Framerate Target": 0,
-    "Window Mode": 1,
+    "Window Mode": 0,
     "Graphic Presets": 4,
     "Texture Quality": 4,
     "Texture Anisotropy": 0,
@@ -38,9 +38,9 @@
     "Color Blind Mode": 0,
     "Color Blind Strength": 100,
     "Show Clock": 1,
-    "Auto Gold Enabled": 1,
+    "Auto Gold Enabled": 0,
     "Chat Gem Mode": 1,
-    "Item Name Display": 0,
+    "Item Name Display": 1,
     "Chat Background": 0,
     "Always Run": 1
 }

game.ini

@@ -1,14 +1,20 @@
 [colors]
 ; min and max hsv range (opencv format: h: [0-180], s: [0-255], v: [0, 255])
 ; h_min, s_min, v_min, h_max, s_max, v_max
-white=0,0,150,4,4,255
-gray=0,0,90,4,4,130
-blue=114,108,190,125,132,255
-green=56,245,190,63,255,255
-yellow=27,122,190,33,145,255
+black=0,0,0,255,255,15
+item_highlight=90,235,135,115,255,160
+white=0,0,150,180,20,255
+gray=0,0,90,180,20,130
+blue=114,100,190,125,132,255
+green=56,190,190,63,255,255
+yellow=27,110,190,33,145,255
 gold=20,75,140,26,95,230
-orange=20,251,190,23,255,255
-red=3,245,160,9,255,220
+orange=20,190,190,23,255,255
+red=3,160,160,9,255,220
+health_potion_0=170,100,76,180,255,255
+health_potion_1=0,100,76,10,255,255
+mana_potion=105,20,76,135,255,255
+rejuv_potion=140,50,40,160,255,255
 
 [ui_pos_1920_1080]
 screen_width=1920
@@ -83,6 +89,8 @@ wp_first_btn_y=193
 wp_btn_height=70
 ; pickit
 item_dist=400
+item_height_min=9
+item_width_min=30
 ; pather
 reached_node_dist=150
 
@@ -107,6 +115,7 @@ repair_btn=477,709,135,120
 inventory=1290,515,591,247
 vendor_stash=44,126,600,600
 skill_right=996,1010,62,62
+hero_selection_logo=0,0,550,830
 
 [path]
 pindle_save_dist=1382,53, 1685,105, 1440,132

params.ini

@@ -1,15 +1,15 @@
+; There is detailed documentation for each parameter in the README.md
 [general]
 name=Botty
 monitor=0
-res=1920_1080
-offset_top=0
-offset_left=0
-min_game_length_s=65
+res=1280_1080
+offset_top=160
+offset_left=320
 max_game_length_s=320
 resume_key=f11
 exit_key=f12
 auto_settings_key=f9
-color_checker_key=f10
+graphic_debugger_key=f10
 logg_lvl=info
 randomize_runs=0
 difficulty=hell
@@ -33,7 +33,10 @@ atk_len_shenk=4
 num_loot_columns=5
 take_health_potion=0.8
 take_mana_potion=0.6
+take_rejuv_potion_health=0.4
+take_rejuv_potion_mana=0.1
 heal_merc=0.5
+heal_rejuv_merc=0.2
 chicken=0.2
 merc_chicken=0
 show_items=alt
@@ -41,11 +44,16 @@ inventory_screen=i
 ;note: stand_still can not be the default "shift" as it would interfere with merc healing
 stand_still=capslock
 tp=f9
-show_belt=~
+belt_rows=4
+;note: this is different from the default hotkey as "~" is for many keyboards not reachable without also pressing altgr
+show_belt=k
 potion1=1
 potion2=2
 potion3=3
 potion4=4
+belt_rejuv_columns=2
+belt_hp_columns=1
+belt_mp_columns=1
 cta_available=0
 weapon_switch=x
 battle_orders=f7
@@ -69,9 +77,31 @@ blessed_hammer=f5
 [advanced_options]
 ;===== don't touch unless you know what you're doing =====
 pathing_delay_factor=5
+template_threshold=0.68
 
 [items]
 ;=====MISC=====
+;potions
+misc_rejuvenation_potion=1
+misc_full_rejuvenation_potion=1
+misc_super_healing_potion=1
+misc_super_mana_potion=1
+
+;gems
+misc_chipped_amethyst=0
+misc_chipped_skull=0
+misc_chipped_diamond=0
+misc_chipped_emerald=0
+misc_chipped_ruby=0
+misc_chipped_sapphire=0
+misc_chipped_topaz=0
+misc_flawed_amethyst=0
+misc_flawed_skull=0
+misc_flawed_diamond=0
+misc_flawed_emerald=0
+misc_flawed_ruby=0
+misc_flawed_sapphire=0
+misc_flawed_topaz=0
 misc_flawless_amethyst=1
 misc_flawless_skull=0
 misc_flawless_diamond=0
@@ -79,38 +109,43 @@ misc_flawless_emerald=0
 misc_flawless_sapphire=0
 misc_flawless_ruby=1
 misc_flawless_topaz=0
-misc_rejuvenation_potion=1
-misc_full_rejuvenation_potion=1
-misc_super_healing_potion=1
-misc_super_mana_potion=0
+
+;other
 misc_tp_scroll=0
 misc_gold=0
+
 ;=====MAGIC=====
 ;helm
 magic_circlet=1
 magic_coronet=1
 magic_diadem=1
 magic_tiara=1
+
 ;etc
 magic_grand_charm=1
 magic_large_charm=0
 magic_small_charm=1
 magic_jewel=1
+
 ;shield
 magic_monarch=0
+
 ;javelin
 magic_matriarchal_javelin=0
 magic_ceremonial_javelin=0
+
 ;=====RARE=====
 ;helm
 rare_circlet=1
 rare_coronet=1
 rare_diadem=1
 rare_tiara=1
+
 ;jewelry
 rare_amulet=1
 rare_jewel=1
 rare_ring=1
+
 ;claws
 rare_blade_talons=1
 rare_cestus=1
@@ -121,6 +156,7 @@ rare_greater_talons=1
 rare_hand_scythe=1
 rare_runic_talons=1
 rare_war_fist=1
+
 ;pelts
 rare_earth_spirit=0
 rare_dream_spirit=0
@@ -137,6 +173,7 @@ rare_falcon_mask=0
 rare_antlers=0
 rare_hawk_helm=0
 rare_wolf_head=0
+
 ;boots
 rare_greaves=1
 rare_boots=1
@@ -153,6 +190,7 @@ rare_demonhide_boots=0
 rare_chain_boots=0
 rare_light_plated_boots=0
 rare_heavy_boots=0
+
 ;gloves - note: "rare_gloves" will pick up all items that have "gloves" in it
 rare_gloves=1
 rare_gauntlets=1
@@ -170,6 +208,7 @@ rare_vampirebone_gloves=0
 rare_vambraces=0
 rare_crusader_gauntlets=0
 rare_ogre_gauntlets=0
+
 ;barb
 rare_guardian_crown=0
 rare_conqueror_crown=0
@@ -186,6 +225,7 @@ rare_assault_helm=0
 rare_horned_helm=0
 rare_fanged_helm=0
 rare_jawbone_cap=0
+
 ;sorc
 rare_eagle_orb=0
 rare_sacred_globe=0
@@ -202,10 +242,12 @@ rare_eldritch_orb=0
 rare_demon_heart=0
 rare_vortex_orb=0
 rare_dimensional_shard=0
+
 ;javelin
 rare_hyperion_javelin=0
 rare_ceremonial_javelin=0
 rare_maiden_javelin=0
+
 ;=====RUNE=====
 rune_1_el=0
 rune_2_eld=0
@@ -240,6 +282,7 @@ rune_30_ber=2
 rune_31_jah=2
 rune_32_cham=2
 rune_33_zod=2
+
 ;=====ETH=====
 eth_armor_archon_plate=1
 eth_armor_dusk_shroud=1
@@ -248,6 +291,7 @@ eth_armor_light_plate=1
 eth_armor_mage_plate=1
 eth_armor_sacred_armor=1
 eth_armor_shadow_plate=1
+eth_armor_wire_fleece=0
 eth_helm_bone_visage=1
 eth_helm_corona=1
 eth_helm_diadem=1
@@ -272,6 +316,7 @@ eth_weapon_superior_grand_matron_bow=1
 eth_weapon_superior_thresher=1
 eth_weapon_superior_giant_thresher=0
 eth_weapon_war_scepter=1
+
 ;eth paladin shields
 eth_sacred_targe=0
 eth_sacred_rondache=0
@@ -281,6 +326,7 @@ eth_vortex_shield=0
 eth_akaran_targe=0
 eth_akaran_rondache=0
 eth_royal_shield=0
+
 ;=====SETS=====
 set_misc_amulets=1
 set_aldur_advance=0
@@ -298,6 +344,7 @@ set_tal_rasha_guardianship=2
 set_tal_rasha_horadric_crest=0
 set_trang-oul_claws=1
 set_trang-oul_girth=1
+
 ;=====UNIQ=====
 uniq_armor_duriel_shell=1
 uniq_armor_guardian_angel=1
@@ -330,7 +377,7 @@ uniq_helm_arreat_face=1
 uniq_helm_cerebus_bite=1
 uniq_helm_crown_of_ages=2
 uniq_helm_giant_skull=1
-uniq_helm_griffon_eye=1
+uniq_helm_griffon_eye=2
 uniq_helm_harlequin_crest=2
 uniq_helm_jalal_mane=1
 uniq_helm_kira_guardian=1

src/belt_manager.py

@@ -0,0 +1,184 @@
+from screen import Screen
+from template_finder import TemplateFinder
+from ui_manager import UiManager
+from config import Config
+from utils.misc import cut_roi, wait, color_filter
+from utils.custom_mouse import mouse
+import numpy as np
+from typing import List
+import keyboard
+import itertools
+import cv2
+from logger import Logger
+
+
+class BeltManager:
+    def __init__(self, screen: Screen, template_finder: TemplateFinder):
+        self._config = Config()
+        self._screen = screen
+        self._template_finder = template_finder
+        self._pot_needs = {"rejuv": 0, "health": 0, "mana": 0}
+        self._item_pot_map = {
+            "misc_rejuvenation_potion": "rejuv",
+            "misc_full_rejuvenation_potion": "rejuv",
+            "misc_super_healing_potion": "health",
+            "misc_super_mana_potion": "mana"
+        }
+
+    def get_pot_needs(self):
+        return self._pot_needs
+
+    def _potion_type(self, img: np.ndarray) -> str:
+        """
+        Based on cut out image from belt, determines what type of potion it is.
+        :param img: Cut out image of a belt slot
+        :return: Any of ["empty", "rejuv", "health", "mana"]
+        """
+        h, w, _ = img.shape
+        roi = [int(w * 0.4), int(h * 0.3), int(w * 0.4), int(h * 0.7)]
+        img = cut_roi(img, roi)
+        avg_brightness = np.average(img)
+        if avg_brightness < 47:
+            return "empty"
+        score_list = []
+        # rejuv
+        mask, _ = color_filter(img, self._config.colors["rejuv_potion"])
+        score_list.append((float(np.sum(mask)) / mask.size) * (1/255.0))
+        # health
+        mask1, _ = color_filter(img, self._config.colors["health_potion_0"])
+        mask2, _ = color_filter(img, self._config.colors["health_potion_1"])
+        mask_health = cv2.bitwise_or(mask1, mask2)
+        score_list.append((float(np.sum(mask_health)) / mask_health.size) * (1/255.0))
+        # mana
+        mask, _ = color_filter(img, self._config.colors["mana_potion"])
+        score_list.append((float(np.sum(mask)) / mask.size) * (1/255.0))
+        # find max score
+        max_val = np.max(score_list)
+        if max_val > 0.28:
+            idx = np.argmax(score_list)
+            types = ["rejuv", "health", "mana"]
+            return types[idx]
+        else:
+            return "empty"
+
+    def _cut_potion_img(self, img: np.ndarray, column: int, row: int) -> np.ndarray:
+        roi = [
+            self._config.ui_pos["potion1_x"] - (self._config.ui_pos["potion_width"] // 2) + column * self._config.ui_pos["potion_next"],
+            self._config.ui_pos["potion1_y"] - (self._config.ui_pos["potion_height"] // 2) - int(row * self._config.ui_pos["potion_next"] * 0.92),
+            self._config.ui_pos["potion_width"],
+            self._config.ui_pos["potion_height"]
+        ]
+        return cut_roi(img, roi)
+
+    def drink_potion(self, potion_type: str, merc: bool = False) -> bool:
+        img = self._screen.grab()
+        for i in range(4):
+            potion_img = self._cut_potion_img(img, i, 0)
+            if self._potion_type(potion_img) == potion_type:
+                key = f"potion{i+1}"
+                if merc:
+                    Logger.debug(f"Give {potion_type} potion in slot {i+1} to merc")
+                    keyboard.send(f"left shift + {self._config.char[key]}")
+                else:
+                    Logger.debug(f"Drink {potion_type} potion in slot {i+1}")
+                    keyboard.send(self._config.char[key])
+                return True
+        return False
+
+    def picked_up_pot(self, item_name: str):
+        """Adjust the _pot_needs when a specific pot was picked up by the pickit
+        :param item_name: Name of the item as it is in the pickit
+        """
+        if item_name in self._item_pot_map:
+            self._pot_needs[self._item_pot_map[item_name]] = max(0, self._pot_needs[self._item_pot_map[item_name]] - 1)
+        else:
+            Logger.warning(f"BeltManager does not know about item: {item_name}")
+
+    def update_pot_needs(self) -> List[int]:
+        """
+        Check how many pots are needed
+        :return: [need_rejuv_pots, need_health_pots, need_mana_pots]
+        """
+        self._pot_needs = {"rejuv": 0, "health": 0, "mana": 0}
+        rows_left = {
+            "rejuv": self._config.char["belt_rejuv_columns"],
+            "health": self._config.char["belt_hp_columns"],
+            "mana": self._config.char["belt_mp_columns"],
+        }
+        center_m = self._screen.convert_abs_to_monitor((-200, -120))
+        mouse.move(*center_m, randomize=100)
+        keyboard.send(self._config.char["show_belt"])
+        wait(0.5)
+        # first clean up columns that might be too much
+        img = self._screen.grab()
+        for column in range(4):
+            potion_type = self._potion_type(self._cut_potion_img(img, column, 0))
+            if potion_type != "empty":
+                rows_left[potion_type] -= 1
+                if rows_left[potion_type] < 0:
+                    rows_left[potion_type] += 1
+                    key = f"potion{column+1}"
+                    for _ in range(5):
+                        keyboard.send(self._config.char[key])
+                        wait(0.2, 0.3)
+        # calc how many potions are needed
+        img = self._screen.grab()
+        current_column = None
+        for column in range(4):
+            for row in range(self._config.char["belt_rows"]):
+                potion_type = self._potion_type(self._cut_potion_img(img, column, row))
+                if row == 0:
+                    if potion_type != "empty":
+                        current_column = potion_type
+                    else:
+                        for key in rows_left:
+                            if rows_left[key] > 0:
+                                rows_left[key] -= 1
+                                self._pot_needs[key] += self._config.char["belt_rows"]
+                                break
+                        break
+                elif current_column is not None and potion_type == "empty":
+                    self._pot_needs[current_column] += 1
+        wait(0.2)
+        Logger.debug(self._pot_needs)
+        keyboard.send(self._config.char["show_belt"])
+
+    def fill_up_belt_from_inventory(self, num_loot_columns: int):
+        """
+        Fill up your belt with pots from the inventory e.g. after death. It will open and close invetory by itself!
+        :param num_loot_columns: Number of columns used for loot from left
+        """
+        keyboard.send(self._config.char["inventory_screen"])
+        wait(0.7, 1.0)
+        img = self._screen.grab()
+        pot_positions = []
+        for column, row in itertools.product(range(num_loot_columns), range(4)):
+            center_pos, slot_img = UiManager.get_slot_pos_and_img(self._config, img, column, row)
+            found = self._template_finder.search("SUPER_HEALING_POTION", slot_img, threshold=0.9)[0]
+            found |= self._template_finder.search("SUPER_MANA_POTION", slot_img, threshold=0.9)[0]
+            found |= self._template_finder.search("FULL_REJUV_POTION", slot_img, threshold=0.9)[0]
+            found |= self._template_finder.search("REJUV_POTION", slot_img, threshold=0.9)[0]
+            if found:
+                pot_positions.append(center_pos)
+        keyboard.press("shift")
+        for pos in pot_positions:
+            x, y = self._screen.convert_screen_to_monitor(pos)
+            mouse.move(x, y, randomize=9, delay_factor=[1.0, 1.5])
+            wait(0.2, 0.3)
+            mouse.click(button="left")
+            wait(0.3, 0.4)
+        keyboard.release("shift")
+        wait(0.2, 0.25)
+        keyboard.send(self._config.char["inventory_screen"])
+        wait(0.5)
+
+
+if __name__ == "__main__":
+    keyboard.wait("f11")
+    config = Config()
+    screen = Screen(config.general["monitor"])
+    template_finder = TemplateFinder(screen)
+    ui_manager = UiManager(screen, template_finder)
+    belt_manager = BeltManager(screen, template_finder)
+    belt_manager.update_pot_needs()
+    print(belt_manager._pot_needs)

src/bot.py

@@ -5,6 +5,7 @@ from template_finder import TemplateFinder
 from item_finder import ItemFinder
 from screen import Screen
 from ui_manager import UiManager
+from belt_manager import BeltManager
 from pather import Pather, Location
 from logger import Logger
 from char.sorceress import Sorceress
@@ -12,8 +13,10 @@ from char.i_char import IChar
 from config import Config
 from health_manager import HealthManager
 from death_manager import DeathManager
+from game_recovery import GameRecovery
 from npc_manager import NpcManager, Npc
 from pickit import PickIt
+from game_stats import GameStats
 from utils.misc import kill_thread, wait
 import keyboard
 import threading
@@ -25,15 +28,18 @@ import random
 class Bot:
     def __init__(self):
         self._config = Config()
+        self._game_stats = GameStats()
+        self._game_recovery = GameRecovery()
         self._screen = Screen(self._config.general["monitor"])
         self._template_finder = TemplateFinder(self._screen)
         self._item_finder = ItemFinder()
         self._ui_manager = UiManager(self._screen, self._template_finder)
+        self._belt_manager = BeltManager(self._screen, self._template_finder)
         self._pather = Pather(self._screen, self._template_finder)
-        self._health_manager = HealthManager(self._screen, self._template_finder, self._ui_manager)
+        self._health_manager = HealthManager(self._screen, self._template_finder, self._ui_manager, self._belt_manager)
         self._death_manager = DeathManager(self._screen, self._template_finder)
         self._npc_manager = NpcManager(self._screen, self._template_finder)
-        self._pickit = PickIt(self._screen, self._item_finder, self._ui_manager)
+        self._pickit = PickIt(self._screen, self._item_finder, self._ui_manager, self._belt_manager, self._game_stats)
         if self._config.char["type"] == "sorceress":
             self._char: IChar = Sorceress(self._config.sorceress, self._config.char, self._screen, self._template_finder, self._ui_manager, self._pather)
         elif self._config.char["type"] == "hammerdin":
@@ -54,7 +60,6 @@ class Bot:
         self._picked_up_items = False
         self._tp_is_up = False
         self._curr_location: Location = None
-        self._timer = None
         self._tps_left = 20
         self._pre_buffed = 0
         self._stopping = False
@@ -102,9 +107,7 @@ class Bot:
             self.trigger(name)
 
     def current_game_length(self):
-        if self._timer is None:
-            return 0
-        return time.time() - self._timer
+        return self._game_stats.get_current_game_length()
 
     def shuffle_runs(self):
         tmp = list(self._do_runs.items())
@@ -120,14 +123,8 @@ class Bot:
         return not found_unfinished_run
 
     def on_create_game(self):
-        if self._timer is not None:
-            delay = self._config.general["min_game_length_s"] - (time.time() - self._timer)
-            if delay > 0.5:
-                Logger.info(f"Delay game creation for {delay:.2f} s")
-                wait(delay, delay + 5.0)
-        Logger.info("Start new game")
-        self._timer = time.time()
-        self._template_finder.search_and_wait("D2_LOGO_HS")
+        self._game_stats.log_start_game()
+        self._template_finder.search_and_wait("D2_LOGO_HS", roi=self._config.ui_roi["hero_selection_logo"])
         self._ui_manager.start_game()
         self._template_finder.search_and_wait(["A5_TOWN_1", "A5_TOWN_0"])
         self._tp_is_up = False
@@ -146,14 +143,16 @@ class Bot:
             self._death_manager.pick_up_corpse()
             # TODO: maybe it is time for a special BeltManager?
             wait(1.2, 1.5)
-            self._ui_manager.fill_up_belt_from_inventory(self._config.char["num_loot_columns"])
+            self._belt_manager.fill_up_belt_from_inventory(self._config.char["num_loot_columns"])
+
+        self._belt_manager.update_pot_needs()
 
         # Check if healing is needed, TODO: add shoping e.g. for potions
         img = self._screen.grab()
         # TODO: If tp is up we always go back into the portal...
         if not self._tp_is_up and (self._health_manager.get_health(img) < 0.6 or self._health_manager.get_mana(img) < 0.3):
-            Logger.info("Need some healing first. Go talk to Malah")
-            if not self._pather.traverse_nodes(self._curr_location, Location.MALAH, self._char): 
+            Logger.info("Need some healing first. Going to Malah.")
+            if not self._pather.traverse_nodes(self._curr_location, Location.MALAH, self._char):
                 self.trigger_or_stop("end_game")
                 return
             self._curr_location = Location.MALAH
@@ -165,21 +164,21 @@ class Bot:
 
         # Stash stuff
         if self._picked_up_items:
-            Logger.info("Stashing picked up items")
+            Logger.info("Stashing picked up items.")
             if not self._pather.traverse_nodes(self._curr_location, Location.A5_STASH, self._char):
                 self.trigger_or_stop("end_game")
                 return
             self._curr_location = Location.A5_STASH
             time.sleep(0.3)
             if self._char.select_by_template("A5_STASH"):
-                self._ui_manager.stash_all_items(self._config.char["num_loot_columns"])
+                self._ui_manager.stash_all_items(self._config.char["num_loot_columns"], self._item_finder)
                 self._picked_up_items = False
                 time.sleep(1.2) # otherwise next grab of screen will still have inventory
             else:
                 Logger.warning("Could not find stash, continue...")
 
         if self._tps_left < 4:
-            Logger.info("Repairing and buying tps at Lazurk")
+            Logger.info("Repairing and buying TPs at Larzuk.")
             if not self._pather.traverse_nodes(self._curr_location, Location.LARZUK, self._char):
                 self.trigger_or_stop("end_game")
                 return
@@ -195,7 +194,7 @@ class Bot:
         # Check if merc needs to be revived
         merc_alive, _ = self._template_finder.search("MERC", self._screen.grab(), threshold=0.9, roi=[0, 0, 200, 200])
         if not merc_alive:
-            Logger.info("Reviveing merc")
+            Logger.info("Reviving merc.")
             if not self._pather.traverse_nodes(self._curr_location, Location.QUAL_KEHK, self._char):
                 self.trigger_or_stop("end_game")
                 return
@@ -233,10 +232,17 @@ class Bot:
         if self._health_manager.did_chicken():
             # in case of chicken give the death manager some time to pick up on possible death flag
             # since death monitor does not check with the same frequency to save on runtime
-            wait(self._death_manager.get_loop_delay() + 1.0)
+            wait(self._death_manager.get_loop_delay() + 3.0)
         self._death_manager.stop_monitor()
         death_monitor_thread.join()
-
+        # some logging
+        if self._death_manager.died():
+            self._game_stats.log_death()
+        elif self._health_manager.did_chicken():
+            self._game_stats.log_chicken()
+        elif not run.success:
+            self._game_stats.log_failed_run()
+        # depending on what happend, trigger next state
         self._current_threads = []
         if self._death_manager.died() or self._health_manager.did_chicken() or self.is_last_run() or not run.success:
             self.trigger_or_stop("end_game")
@@ -333,36 +339,13 @@ class Bot:
     def on_end_game(self):
         self._pre_buffed = 0
         if self._health_manager.did_chicken() or self._death_manager.died():
-            Logger.info("End game while chicken or death happened. Checking where we are at.")
-            # This is a tricky state as we send different actions in different threads.
-            # Chicken could have been succesfull which means we are at hero selection screen
-            # Chicken could have been unsuccesfull, which means we are naked in a5_town
-            # Chicken could have been unsuccesfull, but it already stopped main thread and death manager did not have time to check death -> death screen
+            Logger.info("End game while chicken or death happened. Running game recovery to get back to hero selection.")
             time.sleep(1.5)
-            is_loading = True
-            while is_loading:
-                is_loading = self._template_finder.search("LOADING", self._screen.grab())[0]
-                time.sleep(0.5)
-            time.sleep(1.5)
-            # Okay we are sure we are not in loading screen, let's check if we are in hero selection or in a5 town
-            img = self._screen.grab()
-            if self._template_finder.search("A5_TOWN_1", img)[0]:
-                Logger.info("We are at town, save and exit game.")
-                self._ui_manager.save_and_exit()
-            elif self._template_finder.search("D2_LOGO_HS", img)[0]:
-                Logger.info("We are at hero selection.")
-            elif self._death_manager.handle_death_screen():
-                Logger.info("For some reason we were still at death screen, but Death Manager should have sent us back to town. save and exit game.")
-                self._ui_manager.save_and_exit()
-            else:
-                Logger.error("Could not determine location after chicken / death. Can not continue...")
-                os._exit(1)
+            self._game_recovery.go_to_hero_selection()
         else:
             self._ui_manager.save_and_exit()
 
-        if self._timer is not None:
-            elapsed_time = time.time() - self._timer
-            Logger.info(f"End game. Elapsed time: {elapsed_time:.2f}s")
+        self._game_stats.log_end_game()
         self._do_runs = {
             "run_pindle": self._route_config["run_pindle"],
             "run_shenk": self._route_config["run_shenk"] or self._route_config["run_eldritch"]

src/char/hammerdin.py

@@ -47,7 +47,7 @@ class Hammerdin(IChar):
         keyboard.send(self._char_config["stand_still"], do_press=False)
 
     def _do_redemption(self):
-        if "redemption" in self._skill_hotkeys:
+        if self._skill_hotkeys["redemption"]:
             keyboard.send(self._skill_hotkeys["redemption"])
             wait(1.5, 2.0)
 

src/char/i_char.py

@@ -75,13 +75,19 @@ class IChar:
     def tp_town(self):
         skill_before = cut_roi(self._screen.grab(), self._config.ui_roi["skill_right"])
         keyboard.send(self._char_config["tp"])
-        wait(0.1, 0.1)
+        wait(0.15, 0.2)
         skill_after = cut_roi(self._screen.grab(), self._config.ui_roi["skill_right"])
         _, max_val, _, _ = cv2.minMaxLoc(cv2.matchTemplate(skill_after, skill_before, cv2.TM_CCOEFF_NORMED))
         if max_val > 0.96:
             # found same skill again, thus no more tps available
             Logger.warning("Out of tps")
-            return False
+            time.sleep(1.0)
+            skill_after = cut_roi(self._screen.grab(), self._config.ui_roi["skill_right"])
+            _, max_val, _, _ = cv2.minMaxLoc(cv2.matchTemplate(skill_after, skill_before, cv2.TM_CCOEFF_NORMED))
+            if max_val > 0.96:
+                return False
+            else:
+                Logger.warning("Turns out skill change just took a long time. You ever considered getting a new internet provider or pc?")
         mouse.click(button="right")
         # TODO: Add hardcoded coordinates to ini file
         pos_away = self._screen.convert_abs_to_monitor((int(-250 * self._config.scale), -30))
@@ -120,9 +126,10 @@ class IChar:
 
     def _pre_buff_cta(self):
         # save current skill img
+        wait(0.1)
         skill_before = cut_roi(self._screen.grab(), self._config.ui_roi["skill_right"])
         keyboard.send(self._char_config["weapon_switch"])
-        wait(0.28, 0.35)
+        wait(0.3, 0.35)
         keyboard.send(self._char_config["battle_command"])
         wait(0.08, 0.19)
         mouse.click(button="right")
@@ -132,15 +139,20 @@ class IChar:
         mouse.click(button="right")
         wait(self._cast_duration + 0.08, self._cast_duration + 0.1)
         keyboard.send(self._char_config["weapon_switch"])
-        wait(0.25, 0.3)
+        wait(0.3, 0.35)
         # Make sure that we are back at the previous skill
         skill_after = cut_roi(self._screen.grab(), self._config.ui_roi["skill_right"])
         _, max_val, _, _ = cv2.minMaxLoc(cv2.matchTemplate(skill_after, skill_before, cv2.TM_CCOEFF_NORMED))
         if max_val < 0.96:
             Logger.warning("Failed to switch weapon, try again")
-            wait(1.0)
-            keyboard.send(self._char_config["weapon_switch"])
-            wait(0.4)
+            wait(1.2)
+            skill_after = cut_roi(self._screen.grab(), self._config.ui_roi["skill_right"])
+            _, max_val, _, _ = cv2.minMaxLoc(cv2.matchTemplate(skill_after, skill_before, cv2.TM_CCOEFF_NORMED))
+            if max_val < 0.96:
+                keyboard.send(self._char_config["weapon_switch"])
+                wait(0.4)
+            else:
+                Logger.warning("Turns out weapon switch just took a long time. You ever considered getting a new internet provider or to upgrade your pc?")
 
     @abstract
     def pre_buff(self):

src/char/sorceress.py

@@ -82,14 +82,13 @@ class Sorceress(IChar):
 
     def kill_eldritch(self) -> bool:
         delay = [0.2, 0.3]
-        pos_abs = self._pather.find_abs_node_pos(123, self._screen.grab())
+        pos_abs = self._pather.find_abs_node_pos(1, self._screen.grab())
         if pos_abs is not None:
             eld_pos_abs = [pos_abs[0] * 0.9, pos_abs[1] * 0.9]
         else:
             eld_pos_abs = self._screen.convert_screen_to_abs(self._config.path["eldritch_end"][0])
-        cast_pos_abs = [eld_pos_abs[0] * 0.9, eld_pos_abs[1] * 0.9]
-        if pos_abs is not None:
-            cast_pos_abs = [pos_abs[0] * 0.9, pos_abs[1] * 0.9]
+        if eld_pos_abs is not None:
+            cast_pos_abs = [eld_pos_abs[0] * 0.9, eld_pos_abs[1] * 0.9]
             for _ in range(int(self._char_config["atk_len_eldritch"])):
                 self._main_attack(cast_pos_abs, delay, 90)
                 self._left_attack(cast_pos_abs, delay, 90)
@@ -102,20 +101,23 @@ class Sorceress(IChar):
             return True
         return False
 
-    def kill_shenk(self):
+    def kill_shenk(self) -> bool:
         delay = [0.2, 0.3]
         pos_abs = self._pather.find_abs_node_pos(149, self._screen.grab())
         if pos_abs is not None:
             shenk_pos_abs = [pos_abs[0] * 0.9, pos_abs[1] * 0.9]
         else:
             shenk_pos_abs = self._screen.convert_screen_to_abs(self._config.path["shenk_end"][0])
-        cast_pos_abs = [shenk_pos_abs[0] * 0.9, shenk_pos_abs[1] * 0.9]
-        for _ in range(int(self._char_config["atk_len_shenk"])):
-            self._main_attack(cast_pos_abs, delay, 90)
-            self._left_attack(cast_pos_abs, delay, 90)
-        wait(0.2, 0.3)
-        # Move to items
-        self._pather.traverse_nodes(Location.SHENK_SAVE_DIST, Location.SHENK_END, self, time_out=2.0, force_tp=True)
+        if shenk_pos_abs is not None:
+            cast_pos_abs = [shenk_pos_abs[0] * 0.9, shenk_pos_abs[1] * 0.9]
+            for _ in range(int(self._char_config["atk_len_shenk"])):
+                self._main_attack(cast_pos_abs, delay, 90)
+                self._left_attack(cast_pos_abs, delay, 90)
+            wait(0.2, 0.3)
+            # Move to items
+            self._pather.traverse_nodes(Location.SHENK_SAVE_DIST, Location.SHENK_END, self, time_out=2.0, force_tp=True)
+            return True
+        return False
 
 
 if __name__ == "__main__":
@@ -134,4 +136,6 @@ if __name__ == "__main__":
     char = Sorceress(config.sorceress, config.char, screen, t_finder, ui_manager, pather)
     # char.pre_buff()
     # char.tp_town()
-    char.select_by_template(["A5_RED_PORTAL", "A5_RED_PORTAL_TEXT"], expect_loading_screen=True)
+    pather.traverse_nodes_fixed("eldritch_save_dist", char)
+    char.kill_eldritch()
+    # char.select_by_template(["A5_RED_PORTAL", "A5_RED_PORTAL_TEXT"], expect_loading_screen=True)

src/config.py

@@ -30,12 +30,11 @@ class Config:
             "res": self._select_val("general", "res"),
             "offset_top": int(self._select_val("general", "offset_top")),
             "offset_left": int(self._select_val("general", "offset_left")),
-            "min_game_length_s": float(self._select_val("general", "min_game_length_s")),
             "max_game_length_s": float(self._select_val("general", "max_game_length_s")),
             "exit_key": self._select_val("general", "exit_key"),
             "resume_key": self._select_val("general", "resume_key"),
             "auto_settings_key": self._select_val("general", "auto_settings_key"),
-            "color_checker_key": self._select_val("general", "color_checker_key"),
+            "graphic_debugger_key": self._select_val("general", "graphic_debugger_key"),
             "logg_lvl": self._select_val("general", "logg_lvl"),
             "randomize_runs": bool(int(self._select_val("general", "randomize_runs"))),
             "difficulty": self._select_val("general", "difficulty"),
@@ -56,15 +55,22 @@ class Config:
             "num_loot_columns": int(self._select_val("char", "num_loot_columns")),
             "take_health_potion": float(self._select_val("char", "take_health_potion")),
             "take_mana_potion": float(self._select_val("char", "take_mana_potion")),
+            "take_rejuv_potion_health": float(self._select_val("char", "take_rejuv_potion_health")),
+            "take_rejuv_potion_mana": float(self._select_val("char", "take_rejuv_potion_mana")),
             "heal_merc": float(self._select_val("char", "heal_merc")),
+            "heal_rejuv_merc": float(self._select_val("char", "heal_rejuv_merc")),
             "chicken": float(self._select_val("char", "chicken")),
             "merc_chicken": float(self._select_val("char", "merc_chicken")),
             "tp": self._select_val("char", "tp"),
+            "belt_rows": int(self._select_val("char", "belt_rows")),
             "show_belt": self._select_val("char", "show_belt"),
             "potion1": self._select_val("char", "potion1"),
             "potion2": self._select_val("char", "potion2"),
             "potion3": self._select_val("char", "potion3"),
             "potion4": self._select_val("char", "potion4"),
+            "belt_rejuv_columns": int(self._select_val("char", "belt_rejuv_columns")),
+            "belt_hp_columns": int(self._select_val("char", "belt_hp_columns")),
+            "belt_mp_columns": int(self._select_val("char", "belt_mp_columns")),
             "stash_gold": bool(int(self._select_val("char", "stash_gold"))),
             "cta_available": bool(int(self._select_val("char", "cta_available"))),
             "weapon_switch": self._select_val("char", "weapon_switch"),
@@ -89,7 +95,8 @@ class Config:
             self.hammerdin.update(self._custom["hammerdin"])
         
         self.advanced_options = {
-            "pathing_delay_factor": min(max(int(self._select_val("advanced_options", "pathing_delay_factor")),1),10)
+            "pathing_delay_factor": min(max(int(self._select_val("advanced_options", "pathing_delay_factor")),1),10),
+            "template_threshold": float(self._select_val("advanced_options", "template_threshold")),
         }
 
         self.items = {}

src/death_manager.py

@@ -48,17 +48,15 @@ class DeathManager:
             # first wait a bit to make sure health manager is done with its chicken stuff which obviously failed
             if run_thread is not None:
                 kill_thread(run_thread)
-            # clean up key presses that might be pressed in the run_thread
-            keyboard.release(self._config.char["stand_still"])
-            wait(0.1, 0.2)
-            keyboard.release(self._config.char["show_items"])
+                # clean up key presses that might be pressed in the run_thread
+                keyboard.release(self._config.char["stand_still"])
+                wait(0.1, 0.2)
+                keyboard.release(self._config.char["show_items"])
             time.sleep(6)
-            if self._template_finder.search("D2_LOGO_HS", self._screen.grab())[0]:
+            if self._template_finder.search("D2_LOGO_HS", self._screen.grab(), roi=self._config.ui_roi["hero_selection_logo"])[0]:
                 # in this case chicken executed and left the game, but we were still dead.
                 return True
             keyboard.send("esc")
-            self._template_finder.search_and_wait("A5_TOWN_1")
-            time.sleep(2)
             return True
         return False
 
@@ -69,6 +67,7 @@ class DeathManager:
             if self.handle_death_screen(run_thread):
                 self._do_monitor = False
                 return
+        Logger.debug("Stop death monitoring")
 
 
 # Testing:

src/game_recovery.py

@@ -17,8 +17,17 @@ class GameRecovery:
         self._ui_manager = UiManager(self._screen, self._template_finder)
 
     def go_to_hero_selection(self):
+        # clean up key presses that might be pressed in the run_thread
+        keyboard.release(self._config.char["stand_still"])
+        time.sleep(0.1)
+        keyboard.release(self._config.char["show_items"])
+        # make sure we are not on loading screen
+        is_loading = True
+        while is_loading:
+            is_loading = self._template_finder.search("LOADING", self._screen.grab())[0]
+            time.sleep(0.5)
         # first lets just see if you might already be at hero selection
-        found, _ = self._template_finder.search_and_wait("D2_LOGO_HS", time_out=1, take_ss=False)
+        found, _ = self._template_finder.search_and_wait("D2_LOGO_HS", time_out=1, take_ss=False, roi=self._config.ui_roi["hero_selection_logo"])
         if found:
             return True
         # would have been too easy, maybe we have died?
@@ -27,12 +36,17 @@ class GameRecovery:
             return self._ui_manager.save_and_exit()
         # we must be ingame, but maybe we are at vendor or on stash, press "esc" until we find a save and exit btn (max 5 times)
         for _ in range(5):
-            keyboard.press("esc")
+            keyboard.send("esc")
             time.sleep(1)
             templates = ["SAVE_AND_EXIT_NO_HIGHLIGHT","SAVE_AND_EXIT_HIGHLIGHT"]
             found, _ = self._template_finder.search_and_wait(templates, roi=self._config.ui_roi["save_and_exit"], time_out=1.5, take_ss=False)
             if found:
-                keyboard.press("esc")
+                keyboard.send("esc")
                 time.sleep(1)
                 return self._ui_manager.save_and_exit()
         return False
+
+
+if __name__ == "__main__":
+    game_recovery = GameRecovery()
+    game_recovery.go_to_hero_selection()

src/game_stats.py

@@ -0,0 +1,98 @@
+from logger import Logger
+import time
+import threading
+from config import Config
+from utils.misc import send_discord, hms
+import inspect
+
+
+class GameStats:
+    def __init__(self):
+        self._config = Config()
+        self._picked_up_items = []
+        self._start_time = time.time()
+        self._timer = None
+        self._game_counter = 0
+        self._chicken_counter = 0
+        self._death_counter = 0
+        self._runs_failed = 0
+
+    def _send_discord_thread(self, msg: str):
+        if self._config.general["custom_discord_hook"]:
+            msg = f"{self._config.general['name']}: {msg}"
+            send_discord_thread = threading.Thread(
+                target=send_discord,
+                args=(msg, self._config.general["custom_discord_hook"])
+            )
+            send_discord_thread.daemon = True
+            send_discord_thread.start()
+
+    def log_item_pickup(self, item_name: str, send_discord: bool):
+        self._picked_up_items.append(item_name)
+        if send_discord:
+            self._send_discord_thread(f"Found {item_name}")
+
+    def log_death(self):
+        self._death_counter += 1
+        self._send_discord_thread(f"You have died")
+
+    def log_chicken(self):
+        self._chicken_counter += 1
+        self._send_discord_thread(f"You have chickened")
+
+    def log_start_game(self):
+        if self._game_counter > 0:
+            self._save_stats_to_file()
+            if self._game_counter % 20 == 0:
+                # every 20th game send a discord update about current status
+                self._send_discord_status_update()
+        self._game_counter += 1
+        self._timer = time.time()
+        Logger.info(f"Starting game #{self._game_counter}")
+
+    def log_end_game(self):
+        elapsed_time = time.time() - self._timer
+        Logger.info(f"End game. Elapsed time: {elapsed_time:.2f}s")
+
+    def log_failed_run(self):
+        self._runs_failed += 1
+
+    def get_current_game_length(self):
+        if self._timer is None:
+            return 0
+        return time.time() - self._timer
+
+    def _create_msg(self):
+        elapsed_time = time.time() - self._start_time
+        elapsed_time_str = hms(elapsed_time)
+        avg_length_str = "n/a"
+        if self._game_counter > 0:
+            avg_length = elapsed_time / float(self._game_counter)
+            avg_length_str = hms(avg_length)
+        msg = inspect.cleandoc(f'''
+            Session length: {elapsed_time_str}
+            Games: {self._game_counter}
+            Avg Game Length: {avg_length_str}
+            Chickens: {self._chicken_counter}
+            Deaths: {self._death_counter}
+            Failed runs: {self._runs_failed}
+        ''')
+        return msg
+
+    def _send_discord_status_update(self):
+        msg = f"Status Report\n{self._create_msg()}\nVersion:"
+        self._send_discord_thread(msg)
+
+    def _save_stats_to_file(self):
+        msg = self._create_msg()
+        msg += "\nItems:"
+        for item_name in self._picked_up_items:
+            msg += f"\n  {item_name}"
+        with open("stats.log", "w+") as f:
+            f.write(msg)
+
+
+if __name__ == "__main__":
+    game_stats = GameStats()
+    game_stats._send_discord_status_update()
+    game_stats._save_stats_to_file()

src/health_manager.py

@@ -1,5 +1,6 @@
 from template_finder import TemplateFinder
 from ui_manager import UiManager
+from belt_manager import BeltManager
 import cv2
 import time
 import keyboard
@@ -13,13 +14,15 @@ from threading import Thread
 
 
 class HealthManager:
-    def __init__(self, screen: Screen, template_finder: TemplateFinder, ui_manager: UiManager):
+    def __init__(self, screen: Screen, template_finder: TemplateFinder, ui_manager: UiManager, belt_manager: BeltManager):
         self._config = Config()
         self._screen = screen
         self._template_finder = template_finder
         self._ui_manager = ui_manager
+        self._belt_manager = belt_manager
         self._do_monitor = False
         self._did_chicken = False
+        self._last_rejuv = time.time()
         self._last_health = time.time()
         self._last_mana = time.time()
         self._last_merc_healh = time.time()
@@ -30,25 +33,6 @@ class HealthManager:
     def did_chicken(self):
         return self._did_chicken
 
-    def _drink_poition(self, img: np.ndarray, potion_type: str, merc: bool = False):
-        for i in range(4):
-            roi = [
-                self._config.ui_pos["potion1_x"] - (self._config.ui_pos["potion_width"] // 2) + i * self._config.ui_pos["potion_next"],
-                self._config.ui_pos["potion1_y"] - (self._config.ui_pos["potion_height"] // 2),
-                self._config.ui_pos["potion_width"],
-                self._config.ui_pos["potion_height"]
-            ]
-            potion_img = cut_roi(img, roi)
-            if self._ui_manager.potion_type(potion_img) == potion_type:
-                key = f"potion{i+1}"
-                if merc:
-                    Logger.debug(f"Give {potion_type} potion in slot {i+1} to merc")
-                    keyboard.send(f"left shift + {self._config.char[key]}")
-                else:
-                    Logger.debug(f"Drink {potion_type} potion in slot {i+1}")
-                    keyboard.send(self._config.char[key])
-                break
-
     def get_health(self, img: np.ndarray) -> float:
         health_rec = [self._config.ui_pos["health_left"], self._config.ui_pos["health_top"], self._config.ui_pos["health_width"], self._config.ui_pos["health_height"]]
         health_img = cut_roi(img, health_rec)
@@ -98,25 +82,32 @@ class HealthManager:
         while self._do_monitor:
             time.sleep(0.1)
             img = self._screen.grab()
-            is_loading_black_roi = np.average(img[:, 0:self._config.ui_roi["loading_left_black"][2]]) < 1.0
+            is_loading_black_roi = np.average(img[:, 0:self._config.ui_roi["loading_left_black"][2]]) < 3.0
             if not is_loading_black_roi:
-                # check health
                 health_percentage = self.get_health(img)
-                last_drink = time.time() - self._last_health
-                if health_percentage < self._config.char["take_health_potion"] and last_drink > 3.5:
-                    self._drink_poition(img, "health")
-                    self._last_health = time.time()
-                # give the chicken a 6 sec delay to give time for a healing pot and avoid endless loop of chicken
-                elif health_percentage < self._config.char["chicken"] and (time.time() - start) > 6:
-                    Logger.warning(f"Trying to chicken, player HP {(health_percentage*100):.1f}%!")
-                    self._do_chicken(img, run_thread)
-                    break
-                # check mana
                 mana_percentage = self.get_mana(img)
-                last_drink = time.time() - self._last_mana
-                if mana_percentage < self._config.char["take_mana_potion"] and last_drink > 4:
-                    self._drink_poition(img, "mana")
-                    self._last_mana = time.time()
+                # check rejuv
+                success_drink_rejuv = False
+                if health_percentage < self._config.char["take_rejuv_potion_health"] or \
+                   mana_percentage < self._config.char["take_rejuv_potion_mana"]:
+                    success_drink_rejuv = self._belt_manager.drink_potion("rejuv")
+                # in case no rejuv was used, check for chicken, health pot and mana pot usage
+                if not success_drink_rejuv:
+                    # check health
+                    last_drink = time.time() - self._last_health
+                    if health_percentage < self._config.char["take_health_potion"] and last_drink > 3.5:
+                        self._belt_manager.drink_potion("health")
+                        self._last_health = time.time()
+                    # give the chicken a 6 sec delay to give time for a healing pot and avoid endless loop of chicken
+                    elif health_percentage < self._config.char["chicken"] and (time.time() - start) > 6:
+                        Logger.warning(f"Trying to chicken, player HP {(health_percentage*100):.1f}%!")
+                        self._do_chicken(img, run_thread)
+                        break
+                    # check mana
+                    last_drink = time.time() - self._last_mana
+                    if mana_percentage < self._config.char["take_mana_potion"] and last_drink > 4:
+                        self._belt_manager.drink_potion("mana")
+                        self._last_mana = time.time()
                 # check merc
                 merc_alive, _ = self._template_finder.search("MERC", img, roi=self._config.ui_roi["merc_icon"])
                 if merc_alive:
@@ -126,8 +117,11 @@ class HealthManager:
                         Logger.warning(f"Trying to chicken, merc HP {(merc_health_percentage*100):.1f}%!")
                         self._do_chicken(img, run_thread)
                         break
+                    if merc_health_percentage < self._config.char["heal_rejuv_merc"] and last_drink > 4.0:
+                        self._belt_manager.drink_potion("rejuv", merc=True)
+                        self._last_merc_healh = time.time()
                     elif merc_health_percentage < self._config.char["heal_merc"] and last_drink > 7.0:
-                        self._drink_poition(img, "health", merc=True)
+                        self._belt_manager.drink_potion("health", merc=True)
                         self._last_merc_healh = time.time()
         Logger.debug("Stop health monitoring")
 
@@ -138,7 +132,8 @@ if __name__ == "__main__":
     screen = Screen(config.general["monitor"])
     template_finder = TemplateFinder(screen)
     ui_manager = UiManager(screen, template_finder)
-    manager = HealthManager(screen, template_finder, ui_manager)
+    belt_manager = BeltManager(screen, template_finder)
+    manager = HealthManager(screen, template_finder, ui_manager, belt_manager)
     # manager.start_monitor(None)
     mana = manager.get_mana(screen.grab())
     health = manager.get_health(screen.grab())

src/item_finder.py

@@ -6,7 +6,7 @@ import os
 from dataclasses import dataclass
 import math
 from config import Config
-from utils.misc import color_filter
+from utils.misc import color_filter, cut_roi
 
 
 @dataclass
@@ -50,7 +50,7 @@ class ItemFinder:
             self._folder_name = "items"
             self._min_score = 0.84
         else:
-            self._gaus_filter = (15, 5)
+            self._gaus_filter = (17, 5)
             self._folder_name = "items_1280_720"
             self._min_score = 0.8
         # load all templates
@@ -80,11 +80,24 @@ class ItemFinder:
     def search(self, inp_img: np.ndarray) -> List[Item]:
         img = inp_img[:,:,:]
         start = time.time()
+        # Pre filter black and highlight
+        mask1, _ = color_filter(img, self._config.colors["black"])
+        mask2, _ = color_filter(img, self._config.colors["item_highlight"])
+        filtered_img = cv2.bitwise_or(mask1, mask2)
+        contours = cv2.findContours(filtered_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
+        contours = contours[0] if len(contours) == 2 else contours[1]
+        new_img = np.zeros(img.shape, np.uint8)
+        for cntr in contours:
+            x, y, w, h = cv2.boundingRect(cntr)
+            new_img[y:y+h, x:x+w] = img[y:y+h, x:x+w]
+        img = new_img
+        # Filter by item colors
         filtered_img = np.zeros(img.shape, np.uint8)
         for key in self._game_color_ranges:
             _, extracted_img = color_filter(img, self._game_color_ranges[key])
             filtered_img = cv2.bitwise_or(filtered_img, extracted_img)
         filtered_img_gray = cv2.cvtColor(filtered_img, cv2.COLOR_BGR2GRAY)
+        # Cluster item names
         cluster_img = np.clip(cv2.GaussianBlur(filtered_img_gray, self._gaus_filter, cv2.BORDER_DEFAULT), 0, 255)
         contours = cv2.findContours(cluster_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
         contours = contours[0] if len(contours) == 2 else contours[1]
@@ -95,7 +108,7 @@ class ItemFinder:
             y -= 5
             w += 10
             h += 10
-            # cv2.rectangle(img, (x, y), (x+w, y+h), (0, 255, 0), 1)
+            # cv2.rectangle(inp_img, (x, y), (x+w, y+h), (0, 255, 0), 1)
 
             cropped_input = filtered_img[y:y+h, x:x+w]
             best_score = None
@@ -103,12 +116,12 @@ class ItemFinder:
             for key in self._templates:
                 template: Template = self._templates[key]
                 if cropped_input.shape[1] > template.data.shape[1] and cropped_input.shape[0] > template.data.shape[0]:
-                    # sanity check if hist match okish (same color e.g. set and set or rare and rare)
+                    # sanity check if there is any color overlap of template and cropped_input
                     grayscale = cv2.cvtColor(cropped_input, cv2.COLOR_BGR2GRAY)
                     _, mask = cv2.threshold(grayscale, 0, 255, cv2.THRESH_BINARY)
                     hist = cv2.calcHist([cropped_input], [0, 1, 2], mask, [8, 8, 8], [0, 256, 0, 256, 0, 256])
                     hist_result = cv2.compareHist(template.hist, hist, cv2.HISTCMP_CORREL)
-                    same_type = hist_result > 0.65 and hist_result is not np.inf
+                    same_type = hist_result > 0.0 and hist_result is not np.inf
                     if same_type:
                         result = cv2.matchTemplate(cropped_input, template.data, cv2.TM_CCOEFF_NORMED)
                         _, max_val, _, max_loc = cv2.minMaxLoc(result)
@@ -121,12 +134,21 @@ class ItemFinder:
                                     item = None
                                 else:
                                     max_loc = [max_loc[0] + x, max_loc[1] + y]
-                                    item = Item()
-                                    item.center = (int(max_loc[0] + int(template.data.shape[1] * 0.5)), int(max_loc[1] + int(template.data.shape[0] * 0.5)))
-                                    item.name = key
-                                    item.score = max_val
-                                    center_abs = (item.center[0] - (inp_img.shape[1] // 2), item.center[1] - (inp_img.shape[0] // 2))
-                                    item.dist = math.dist(center_abs, (0, 0))
+                                    # Do another color hist check with the actuall found item template
+                                    cropped_roi = [*max_loc, template.data.shape[1], template.data.shape[0]]
+                                    cropped_item = cut_roi(filtered_img, cropped_roi)
+                                    grayscale = cv2.cvtColor(cropped_item, cv2.COLOR_BGR2GRAY)
+                                    _, mask = cv2.threshold(grayscale, 0, 255, cv2.THRESH_BINARY)
+                                    hist = cv2.calcHist([cropped_item], [0, 1, 2], mask, [8, 8, 8], [0, 256, 0, 256, 0, 256])
+                                    hist_result = cv2.compareHist(template.hist, hist, cv2.HISTCMP_CORREL)
+                                    same_type = hist_result > 0.65 and hist_result is not np.inf
+                                    if same_type:
+                                        item = Item()
+                                        item.center = (int(max_loc[0] + int(template.data.shape[1] * 0.5)), int(max_loc[1] + int(template.data.shape[0] * 0.5)))
+                                        item.name = key
+                                        item.score = max_val
+                                        center_abs = (item.center[0] - (inp_img.shape[1] // 2), item.center[1] - (inp_img.shape[0] // 2))
+                                        item.dist = math.dist(center_abs, (0, 0))
             if item is not None and self._config.items[item.name]:
                 item_list.append(item)
         elapsed = time.time() - start
@@ -148,6 +170,7 @@ if __name__ == "__main__":
         for item in item_list:
             print(item.name + " " + str(item.score))
             cv2.circle(img, item.center, 5, (255, 0, 255), thickness=3)
-        img = cv2.resize(img, None, fx=0.5, fy=0.5)
+            cv2.putText(img, item.name, item.center, cv2.FONT_HERSHEY_SIMPLEX, 0.8, (255, 255, 255), 1, cv2.LINE_AA)
+        # img = cv2.resize(img, None, fx=0.5, fy=0.5)
         cv2.imshow('test', img)
         cv2.waitKey(1)

src/pather.py

@@ -79,15 +79,15 @@ class Pather:
             122: {"ELDRITCH_2": (530, -218), "ELDRITCH_3": (-223, -178)},
             123: {"ELDRITCH_3": (-148, -498), "ELDRITCH_2": (604, -538), "ELDRITCH_4": (-163, -283)},
             # Shenk
-            140: {"SHENK_0": (-224, -340), "SHENK_15": (120, 20), "SHENK_1": (667, -242)},
-            141: {"SHENK_0": (-194, 66), "SHENK_15": (116, 440), "SHENK_1": (696, 161), "SHENK_2": (-251, -51)},
+            140: {"SHENK_0": (-224, -340), "SHENK_17": (-750, 353), "SHENK_15": (120, 20), "SHENK_1": (667, -242)},
+            141: {"SHENK_0": (-194, 66), "SHENK_17": (-780, 792), "SHENK_15": (116, 440), "SHENK_1": (696, 161), "SHENK_2": (-251, -51)},
             142: {"SHENK_1": (876, 564), "SHENK_2": (-78, 352), "SHENK_3": (535, -194), "SHENK_4": (-665, -155)},
             143: {"SHENK_2": (212, 758),"SHENK_3": (823, 209), "SHENK_4": (-377, 248), "SHENK_6": (-508, -103)},
             144: {"SHENK_6": (-162, 185), "SHENK_7": (721, 226)},
             145: {"SHENK_12": (146, -200), "SHENK_7": (1204, 558), "SHENK_6": (314, 520), "SHENK_8": (-367, 27)},
             146: {"SHENK_12": (408, 166), "SHENK_9": (-497, -216), "SHENK_8": (-108, 387)},
-            147: {"SHENK_9": (-100, 208), "SHENK_10": (-646, 100)},
-            148: {"SHENK_9": (451, 395), "SHENK_10": (-97, 282), "SHENK_11": (-459, 208)},
+            147: {"SHENK_16": (290, -150), "SHENK_9": (-100, 208), "SHENK_10": (-646, 100)},
+            148: {"SHENK_16": (967, 95), "SHENK_9": (451, 395), "SHENK_10": (-97, 282), "SHENK_11": (-459, 208)},
             149: {"SHENK_11": (532, 602), "SHENK_10": (882, 682), "SHENK_13": (730, 36)},
 
         }
@@ -130,11 +130,17 @@ class Pather:
         while 1:
             img = self._screen.grab()
             display_img = img.copy()
+            template_map = {}
             for node_idx in self._nodes:
                 for template_type in self._nodes[node_idx]:
                         if filter is None or filter in template_type:
-                            success, ref_pos_screen = self._template_finder.search(template_type, img)
+                            if template_type in template_map:
+                                success = True
+                                ref_pos_screen = template_map[template_type]
+                            else:
+                                success, ref_pos_screen = self._template_finder.search(template_type, img)
                             if success:
+                                template_map[template_type] = ref_pos_screen
                                 # Get reference position of template in abs coordinates
                                 ref_pos_abs = self._screen.convert_screen_to_abs(ref_pos_screen)
                                 # Calc the abs node position with the relative coordinates (relative to ref)
@@ -144,7 +150,7 @@ class Pather:
                                 x, y = self._screen.convert_abs_to_screen(node_pos_abs)
                                 cv2.circle(display_img, (x, y), 5, (255, 0, 0), 3)
                                 cv2.putText(display_img, str(node_idx), (x, y), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 0), 2, cv2.LINE_AA)
-                                x, y = self._screen.convert_abs_to_monitor(ref_pos_abs)
+                                x, y = self._screen.convert_abs_to_screen(ref_pos_abs)
                                 cv2.circle(display_img, (x, y), 5, (0, 255, 0), 3)
                                 cv2.putText(display_img, template_type, (x, y), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 0), 2, cv2.LINE_AA)
             display_img = cv2.resize(display_img, None, fx=0.5, fy=0.5)
@@ -265,6 +271,6 @@ if __name__ == "__main__":
     pather = Pather(screen, t_finder)
     ui_manager = UiManager(screen, t_finder)
     char = Sorceress(config.sorceress, config.char, screen, t_finder, ui_manager, pather)
-    pather.traverse_nodes_fixed("pindle_save_dist", char)
-    # pather.traverse_nodes(Location.A5_STASH, Location.LARZUK, char)
-    # pather._display_all_nodes_debug(filter="A5_TOWN")
+    # pather.traverse_nodes_fixed("pindle_save_dist", char)
+    # pather.traverse_nodes(Location.SHENK_START, Location.SHENK_SAVE_DIST, char)
+    pather._display_all_nodes_debug(filter="SHENK_17")

src/pickit.py

@@ -1,3 +1,5 @@
+from belt_manager import BeltManager
+from ui_manager import UiManager
 from item_finder import ItemFinder
 import time
 from utils.custom_mouse import mouse
@@ -8,15 +10,16 @@ from char.i_char import IChar
 from logger import Logger
 from screen import Screen
 from ui_manager import UiManager
-import threading
-from utils.misc import send_discord
+from game_stats import GameStats
 
 
 class PickIt:
-    def __init__(self, screen: Screen, item_finder: ItemFinder, ui_manager: UiManager):
+    def __init__(self, screen: Screen, item_finder: ItemFinder, ui_manager: UiManager, belt_manager: BeltManager, game_stats: GameStats = None):
         self._item_finder = item_finder
         self._screen = screen
+        self._belt_manager = belt_manager
         self._ui_manager = ui_manager
+        self._game_stats = game_stats
         self._config = Config()
 
     def pick_up_items(self, char: IChar) -> bool:
@@ -26,7 +29,7 @@ class PickIt:
         :return: Bool if any items were picked up or not. (Does not account for picking up scrolls and pots)
         """
         found_items = False
-        keyboard.send(self._config.char["show_items"], do_press=True, do_release=False)
+        keyboard.send(self._config.char["show_items"])
         time.sleep(1.0) # sleep needed here to give d2r time to display items on screen on keypress
         #Creating a screenshot of the current loot
         if self._config.general["loot_screenshots"]:
@@ -37,14 +40,22 @@ class PickIt:
         time_out = False
         picked_up_items = []
         while not time_out:
-            if (time.time() - start) > 20:
+            if (time.time() - start) > 24:
                 time_out = True
                 Logger.warning("Got stuck during pickit, skipping it this time...")
+                break
             img = self._screen.grab()
             item_list = self._item_finder.search(img)
-            if not self._ui_manager.check_free_belt_spots():
-                # no free slots in belt, do not pick up health or mana potions
-                item_list = [x for x in item_list if "potion" not in x.name]
+
+            # Check if we need to pick up certain pots more pots
+            need_pots = self._belt_manager.get_pot_needs()
+            if need_pots["mana"] <= 0:
+                item_list = [x for x in item_list if "mana_potion" not in x.name]
+            if need_pots["health"] <= 0:
+                item_list = [x for x in item_list if "healing_potion" not in x.name]
+            if need_pots["rejuv"] <= 0:
+                item_list = [x for x in item_list if "rejuvenation_potion" not in x.name]
+
             if len(item_list) == 0:
                 break
             else:
@@ -54,10 +65,13 @@ class PickIt:
                         closest_item = item
                 x_m, y_m = self._screen.convert_screen_to_monitor(closest_item.center)
                 if closest_item.dist < self._config.ui_pos["item_dist"]:
+                    # if potion is picked up, record it in the belt manager
+                    if "potion" in closest_item.name:
+                        self._belt_manager.picked_up_pot(closest_item.name)
                     # no need to stash potions, scrolls, or gold 
-                    if (("potion" not in closest_item.name) and ("tp_scroll" != closest_item.name) and ("misc_gold" not in closest_item.name)):
+                    if "potion" not in closest_item.name and "tp_scroll" != closest_item.name and "misc_gold" not in closest_item.name:
                         found_items = True
-                    Logger.info(f"Picking up {closest_item.name}")
+                    Logger.info(f"Picking up: {closest_item.name}")
                     mouse.move(x_m, y_m)
                     time.sleep(0.1)
                     mouse.click(button="left")
@@ -70,19 +84,13 @@ class PickIt:
                         break
                     else:
                         # send log to discord
-                        if found_items and self._config.items[closest_item.name] == 2 and closest_item.name not in picked_up_items:
-                            if self._config.general["custom_discord_hook"] != "":
-                                send_discord_thread = threading.Thread(
-                                    target=send_discord,
-                                    args=(f"{self._config.general['name']} just found: {closest_item.name}", self._config.general["custom_discord_hook"])
-                                )
-                                send_discord_thread.daemon = True
-                                send_discord_thread.start()
+                        if found_items and closest_item.name not in picked_up_items:
+                            self._game_stats.log_item_pickup(closest_item.name, self._config.items[closest_item.name] == 2)
                         picked_up_items.append(closest_item.name)
                 else:
                     char.move((x_m, y_m))
                     time.sleep(0.1)
-        keyboard.send(self._config.char["show_items"], do_press=False, do_release=True)
+        keyboard.send(self._config.char["show_items"])
         return found_items
 
 
@@ -100,7 +108,8 @@ if __name__ == "__main__":
     screen = Screen(config.general["monitor"])
     t_finder = TemplateFinder(screen)
     ui_manager = UiManager(screen, t_finder)
+    belt_manager = BeltManager(screen, t_finder)
     item_finder = ItemFinder()
     char = Sorceress(config.sorceress, config.char, screen, t_finder, item_finder, ui_manager)
-    pickit = PickIt(screen, item_finder, ui_manager)
+    pickit = PickIt(screen, item_finder, ui_manager, belt_manager)
     pickit.pick_up_items(char)

src/run.py

@@ -4,14 +4,15 @@ from logger import Logger
 import keyboard
 import os
 from config import Config
-from utils.color_checker import run_color_checker
+from utils.graphic_debugger import run_graphic_debugger
 from version import __version__
 from utils.auto_settings import adjust_settings
-from utils.misc import kill_thread, send_discord, close_down_d2
+from utils.misc import kill_thread, send_discord
 import threading
 from beautifultable import BeautifulTable
 import time
 import logging
+import cv2
 
 
 def run_bot(config: Config):
@@ -27,18 +28,19 @@ def run_bot(config: Config):
             Logger.info(f"Max game length reached. Attempting to restart {config.general['name']}!")
             bot.stop()
             kill_thread(bot_thread)
-            if game_recovery.go_to_hero_selection():
-                do_restart = False
+            do_restart = game_recovery.go_to_hero_selection()
             break
-        time.sleep(0.04)
+        time.sleep(0.5)
     bot_thread.join()
     if do_restart:
         run_bot(config)
     else:
+        if config.general["info_screenshots"]:
+            cv2.imwrite("./info_screenshots/info_could_not_recover_" + time.strftime("%Y%m%d_%H%M%S") + ".png", bot._screen.grab())
         Logger.error(f"{config.general['name']} could not recover from a max game length violation. Shutting down everything.")
         if config.general["custom_discord_hook"]:
             send_discord(f"{config.general['name']} got stuck and can not resume", config.general["custom_discord_hook"])
-        close_down_d2()
+        os._exit(1)
 
 
 if __name__ == "__main__":
@@ -63,7 +65,7 @@ if __name__ == "__main__":
     print("\nFor gettings started and documentation\nplease read https://github.com/aeon0/botty\n")
     table = BeautifulTable()
     table.rows.append([config.general['auto_settings_key'], "Adjust D2R settings"])
-    table.rows.append([config.general['color_checker_key'], "Color test mode "])
+    table.rows.append([config.general['graphic_debugger_key'], "Color test mode "])
     table.rows.append([config.general['resume_key'], "Start / Pause Botty"])
     table.rows.append([config.general['exit_key'], "Stop bot"])
     table.columns.header = ["hotkey", "action"]
@@ -76,8 +78,9 @@ if __name__ == "__main__":
             break
         if keyboard.is_pressed(config.general['auto_settings_key']):
             adjust_settings()
-        elif keyboard.is_pressed(config.general['color_checker_key']):
-            run_color_checker()
+            break
+        elif keyboard.is_pressed(config.general['graphic_debugger_key']):
+            run_graphic_debugger()
             break
         time.sleep(0.02)
 

src/template_finder.py

@@ -1,5 +1,4 @@
 import cv2
-from enum import Enum
 from screen import Screen
 from typing import Tuple, Union, List
 import numpy as np
@@ -74,6 +73,8 @@ class TemplateFinder:
             "SHENK_12": [load_template(f"assets/templates{res_str}/shenk/shenk_12.png", self._scale_factor), self._scale_factor],
             "SHENK_13": [load_template(f"assets/templates{res_str}/shenk/shenk_13.png", self._scale_factor), self._scale_factor],
             "SHENK_15": [load_template(f"assets/templates{res_str}/shenk/shenk_15.png", self._scale_factor), self._scale_factor],
+            "SHENK_16": [load_template(f"assets/templates{res_str}/shenk/shenk_16.png", self._scale_factor), self._scale_factor],
+            "SHENK_17": [load_template(f"assets/templates{res_str}/shenk/shenk_17.png", self._scale_factor), self._scale_factor],
             # Template Selectables
             "A5_STASH": [load_template(f"assets/templates{res_str}/a5_stash.png", self._scale_factor), self._scale_factor],
             "A5_WP": [load_template(f"assets/templates{res_str}/a5_wp.png", self._scale_factor), self._scale_factor],
@@ -101,6 +102,8 @@ class TemplateFinder:
             "TP_TOMB": [load_template(f"assets/templates{res_str}/tp_tomb.png", 1.0), 1.0],
             "SUPER_HEALING_POTION": [load_template(f"assets/templates{res_str}/super_healing_potion.png", 1.0), 1.0],
             "SUPER_MANA_POTION": [load_template(f"assets/templates{res_str}/super_mana_potion.png", 1.0), 1.0],
+            "FULL_REJUV_POTION": [load_template(f"assets/templates{res_str}/full_rejuv_potion.png", 1.0), 1.0],
+            "REJUV_POTION": [load_template(f"assets/templates{res_str}/rejuv_potion.png", 1.0), 1.0],
             # NPC: Qual-Kehk
             "QUAL_FRONT": [load_template(f"assets/npc{res_str}/qual_kehk/qual_front.png", 1.0), 1.0],
             "QUAL_SIDE": [load_template(f"assets/npc{res_str}/qual_kehk/qual_side.png", 1.0), 1.0],
@@ -145,7 +148,7 @@ class TemplateFinder:
         self, 
         ref: Union[str, np.ndarray],
         inp_img: np.ndarray,
-        threshold: float = 0.7, 
+        threshold: float = None, 
         roi: List[float] = None,
         normalize_monitor: bool = False, 
     ) -> Tuple[bool, Tuple[float, float]]:
@@ -157,6 +160,7 @@ class TemplateFinder:
         :param roi: Region of Interest of the inp_img to restrict search area. Format [left, top, width, height]
         :return: Returns found flag and the position as [bool, [x, y]]. If not found, position will be None. Position in image space.
         """
+        threshold = self._config.advanced_options["template_threshold"] if threshold is None else threshold
         if roi is None:
             # if no roi is provided roi = full inp_img
             roi = [0, 0, inp_img.shape[1], inp_img.shape[0]]
@@ -195,7 +199,7 @@ class TemplateFinder:
         ref: Union[str, List[str]],
         roi: List[float] = None,
         time_out: float = None,
-        threshold: float = 0.7,
+        threshold: float = None,
         take_ss: bool = True
     ) -> Tuple[bool, Tuple[float, float]]:
         """
@@ -206,6 +210,7 @@ class TemplateFinder:
         :param take_ss: Bool value to take screenshot on timeout or not (flag must still be set in params!)
         Rest of params same as TemplateFinder.search()
         """
+        threshold = self._config.advanced_options["template_threshold"] if threshold is None else threshold
         Logger.debug(f"Waiting for Template {ref}")
         start = time.time()
         while 1:
@@ -234,13 +239,19 @@ if __name__ == "__main__":
     config = Config()
     screen = Screen(config.general["monitor"])
     template_finder = TemplateFinder(screen)
+    search_templates = ["ELDRITCH_4", "ELDRITCH_3", "ELDRITCH_2", "ELDRITCH_1"]
+    scores = {}
     while 1:
-        img = screen.grab().copy()
-        success, pos = template_finder.search("A5_TOWN_1", img, threshold=0.67)
-        print(template_finder.last_score)
-        if success:
-            cv2.circle(img, pos, 7, (255, 0, 0), thickness=5)
-        img = cv2.resize(img, None, fx=0.5, fy=0.5, interpolation=cv2.INTER_NEAREST)
-        # cv2.imshow("temp", template_finder._templates["A5_TOWN_1"][0])
-        cv2.imshow('test', img)
+        # img = cv2.imread("")
+        img = screen.grab()
+        display_img = img.copy()
+        for template_name in search_templates:
+            success, pos = template_finder.search(template_name, img)
+            scores[template_name] = template_finder.last_score
+            if success:
+                cv2.putText(display_img, str(template_name), pos, cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 0), 2, cv2.LINE_AA)
+                cv2.circle(display_img, pos, 7, (255, 0, 0), thickness=5)
+        display_img = cv2.resize(display_img, None, fx=0.5, fy=0.5, interpolation=cv2.INTER_NEAREST)
+        print(scores)
+        cv2.imshow('test', display_img)
         key = cv2.waitKey(1)

src/ui_manager.py

@@ -3,15 +3,16 @@ from template_finder import TemplateFinder
 from typing import Tuple
 from utils.custom_mouse import mouse
 import keyboard
-import random
 import time
 import cv2
 import itertools
 import os
 import numpy as np
 from logger import Logger
-from utils.misc import wait, cut_roi, color_filter, close_down_d2
+from utils.misc import wait, cut_roi, color_filter, send_discord
 from config import Config
+from item_finder import ItemFinder
+
 
 class UiManager():
     """Everything that is clicking on some static 2D UI or is checking anything in regard to it should be placed here."""
@@ -94,45 +95,6 @@ class UiManager():
                 return True
         return False
 
-    @staticmethod
-    def potion_type(img: np.ndarray) -> str:
-        """
-        Based on cut out image from belt, determines what type of potion it is. TODO: Add rejuv support
-        :param img: Cut out image of a belt slot
-        :return: Any of ["empty", "health", "mana"]
-        """
-        avg_brightness = np.average(img)
-        if avg_brightness < 47:
-            return "empty"
-        red_channel = img[:,:,2]
-        redness = np.average(red_channel)
-        blue_channel = img[:,:,0]
-        blueness = np.average(blue_channel)
-        if redness > blueness and redness > 55:
-            return "health"
-        if blueness > redness and blueness > 55:
-            return "mana"
-        return "empty"
-
-    def check_free_belt_spots(self) -> bool:
-        """
-        Check if any column in the belt is free (only checking the bottom row)
-        :return: Bool if any column in belt is free
-        """
-        img = self._screen.grab()
-        for i in range(4):
-            roi = [
-                self._config.ui_pos["potion1_x"] - (self._config.ui_pos["potion_width"] // 2) + i * self._config.ui_pos["potion_next"],
-                self._config.ui_pos["potion1_y"] - (self._config.ui_pos["potion_height"] // 2),
-                self._config.ui_pos["potion_width"],
-                self._config.ui_pos["potion_height"]
-            ]
-            potion_img = cut_roi(img, roi)
-            potion_type = self.potion_type(potion_img)
-            if potion_type == "empty":
-                return True
-        return False
-
     def save_and_exit(self, does_chicken: bool = False) -> bool:
         """
         Performes save and exit action from within game
@@ -243,17 +205,19 @@ class UiManager():
         avg_brightness = np.average(slot_img[:, :, 2])
         return avg_brightness > 16.0
 
-    def _get_slot_pos_and_img(self, img: np.ndarray, column: int, row: int) -> Tuple[Tuple[int, int],  np.ndarray]:
+    @staticmethod
+    def get_slot_pos_and_img(config: Config, img: np.ndarray, column: int, row: int) -> Tuple[Tuple[int, int],  np.ndarray]:
         """
         Get the pos and img of a specific slot position in Inventory. Inventory must be open in the image.
+        :param config: The config which should be used
         :param img: Image from screen.grab() not cut
         :param column: Column in the Inventory
         :param row: Row in the Inventory
         :return: Returns position and image of the cut area as such: [[x, y], img]
         """
-        top_left_slot = (self._config.ui_pos["inventory_top_left_slot_x"], self._config.ui_pos["inventory_top_left_slot_y"])
-        slot_width = self._config.ui_pos["slot_width"]
-        slot_height= self._config.ui_pos["slot_height"]
+        top_left_slot = (config.ui_pos["inventory_top_left_slot_x"], config.ui_pos["inventory_top_left_slot_y"])
+        slot_width = config.ui_pos["slot_width"]
+        slot_height= config.ui_pos["slot_height"]
         slot = (top_left_slot[0] + slot_width * column, top_left_slot[1] + slot_height * row)
         # decrease size to make sure not to have any borders of the slot in the image
         offset_w = int(slot_width * 0.12)
@@ -274,12 +238,26 @@ class UiManager():
         :return: Bool if inventory still has items or not
         """
         for column, row in itertools.product(range(num_loot_columns), range(4)):
-            _, slot_img = self._get_slot_pos_and_img(img, column, row)
+            _, slot_img = self.get_slot_pos_and_img(self._config, img, column, row)
             if self._slot_has_item(slot_img):
                 return True
         return False
 
-    def stash_all_items(self, num_loot_columns: int):
+    def _keep_item(self, item_finder: ItemFinder) -> bool:
+        """
+        Check if an item should be kept, the item should be hovered and in own inventory when function is called
+        :param item_finder: ItemFinder to check if item is in pickit
+        :return: Bool if item should be kept
+        """
+        wait(0.2, 0.3)
+        img = self._screen.grab()
+        _, w, _ = img.shape
+        img = img[:, (w//2):,:]
+        item_list = item_finder.search(img)
+        item_list = [x for x in item_list if "potion" not in x.name]
+        return len(item_list) > 0
+
+    def stash_all_items(self, num_loot_columns: int, item_finder: ItemFinder):
         """
         Stashing all items in inventory. Stash UI must be open when calling the function.
         :param num_loot_columns: Number of columns used for loot from left
@@ -312,19 +290,39 @@ class UiManager():
             wait(0.4, 0.6)
             keyboard.send("enter") #if stash already full of gold just nothing happens -> gold stays on char -> no popup window
         # stash stuff
-        keyboard.send('ctrl', do_release=False)
+        center_m = self._screen.convert_abs_to_monitor((0, 0))
         for column, row in itertools.product(range(num_loot_columns), range(4)):
             img = self._screen.grab()
-            slot_pos, slot_img = self._get_slot_pos_and_img(img, column, row)
+            slot_pos, slot_img = self.get_slot_pos_and_img(self._config, img, column, row)
             if self._slot_has_item(slot_img):
                 x_m, y_m = self._screen.convert_screen_to_monitor(slot_pos)
                 mouse.move(x_m, y_m, randomize=10, delay_factor=[1.0, 1.3])
-                wait(0.2, 0.3)
-                mouse.press(button="left")
-                wait(0.25, 0.35)
-                mouse.release(button="left")
-                wait(0.4, 0.5)
-        keyboard.send('ctrl', do_press=False)
+                # check item again and discard it or stash it
+                if self._keep_item(item_finder):
+                    keyboard.send('ctrl', do_release=False)
+                    wait(0.2, 0.25)
+                    mouse.press(button="left")
+                    wait(0.2, 0.25)
+                    mouse.release(button="left")
+                    wait(0.2, 0.25)
+                    keyboard.send('ctrl', do_press=False)
+                else:
+                    # make sure there is actually an item
+                    time.sleep(0.3)
+                    hovered_item = self._screen.grab()
+                    slot_pos, slot_img = self.get_slot_pos_and_img(self._config, hovered_item, column, row)
+                    if self._slot_has_item(slot_img):
+                        if self._config.general["info_screenshots"]:
+                            cv2.imwrite("./info_screenshots/info_discard_item_" + time.strftime("%Y%m%d_%H%M%S") + ".png", hovered_item)
+                        mouse.press(button="left")
+                        wait(0.2, 0.4)
+                        mouse.release(button="left")
+                        mouse.move(*center_m, randomize=20)
+                        wait(0.2, 0.3)
+                        mouse.press(button="left")
+                        wait(0.2, 0.3)
+                        mouse.release(button="left")
+                        wait(0.5, 0.5)
         Logger.debug("Check if stash is full")
         time.sleep(0.6)
         # move mouse away from inventory, for some reason it was sometimes included in the grabed img
@@ -340,43 +338,18 @@ class UiManager():
             self._curr_stash += 1
             if self._curr_stash > 3:
                 Logger.error("All stash is full, quitting")
-                close_down_d2()
+                if self._config.general["custom_discord_hook"]:
+                    send_discord(f"{self._config.general['name']} all stash is full, quitting", self._config.general["custom_discord_hook"])
                 os._exit(1)
             else:
                 # move to next stash
                 wait(0.5, 0.6)
-                return self.stash_all_items(num_loot_columns)
+                return self.stash_all_items(num_loot_columns, item_finder)
 
         Logger.debug("Done stashing")
         wait(0.4, 0.5)
         keyboard.send("esc")
 
-    def fill_up_belt_from_inventory(self, num_loot_columns: int):
-        """
-        Fill up your belt with pots from the inventory e.g. after death. It will open and close invetory by itself!
-        :param num_loot_columns: Number of columns used for loot from left
-        """
-        keyboard.send(self._config.char["inventory_screen"])
-        wait(0.7, 1.0)
-        img = self._screen.grab()
-        pot_positions = []
-        for column, row in itertools.product(range(num_loot_columns), range(4)):
-            center_pos, slot_img = self._get_slot_pos_and_img(img, column, row)
-            found = self._template_finder.search("SUPER_HEALING_POTION", slot_img, threshold=0.9)[0]
-            found |= self._template_finder.search("SUPER_MANA_POTION", slot_img, threshold=0.9)[0]
-            if found:
-                pot_positions.append(center_pos)
-        keyboard.press("shift")
-        for pos in pot_positions:
-            x, y = self._screen.convert_screen_to_monitor(pos)
-            mouse.move(x, y, randomize=9, delay_factor=[1.0, 1.5])
-            wait(0.2, 0.3)
-            mouse.click(button="left")
-            wait(0.3, 0.4)
-        keyboard.release("shift")
-        wait(0.2, 0.25)
-        keyboard.send(self._config.char["inventory_screen"])
-
     def close_vendor_screen(self):
         keyboard.send("esc")
         # just in case also bring cursor to center and click
@@ -436,5 +409,6 @@ if __name__ == "__main__":
     config = Config()
     screen = Screen(config.general["monitor"])
     template_finder = TemplateFinder(screen)
+    item_finder = ItemFinder()
     ui_manager = UiManager(screen, template_finder)
-    ui_manager.save_and_exit()
+    ui_manager.stash_all_items(5, item_finder)

src/utils/auto_settings.py

@@ -2,8 +2,10 @@ import json
 import os
 
 def adjust_settings():
-    print("Warning: This will overwrite some of your graphics and gameplay settings. D2R must not be running during this action! Continue with Enter...")
-    input()
+    # You might not belive it, but there are cases where users press f9 and just dont read any console output and think everything is done
+    # so for now, removing this warning as it seems there has never been anybody actually backing up the original settings anyway...
+    # print("Warning: This will overwrite some of your graphics and gameplay settings. D2R must not be running during this action! Continue with Enter...")
+    # input()
     # Get D2r folder
     d2_saved_games = f"C:\\Users\\{os.getlogin()}\\Saved Games\\Diablo II Resurrected"
     if not os.path.exists(d2_saved_games):
@@ -21,7 +23,7 @@ def adjust_settings():
     # write back to settings.json
     with open(d2_saved_games + "\\Settings.json", 'w') as outfile:
         json.dump(curr_settings, outfile)
-    print("Changed settings succesfully. You can now start D2R to start the bot.")
+    print("Changed settings succesfully. You can now start D2R and botty.")
 
 
 if __name__ == "__main__":

src/utils/color_checker.py

@@ -1,29 +0,0 @@
-import cv2
-import numpy as np
-from utils.misc import color_filter
-from screen import Screen
-from item_finder import ItemFinder
-from config import Config
-
-
-def run_color_checker():
-    config = Config()
-    screen = Screen(config.general["monitor"])
-    item_finder = ItemFinder()
-    while 1:
-        img = screen.grab()
-        combined_img = np.zeros(img.shape, dtype="uint8")
-        for key in config.colors:
-            _, filterd_img = color_filter(img, config.colors[key])
-            combined_img = cv2.bitwise_or(filterd_img, combined_img)
-        item_list = item_finder.search(img)
-        for item in item_list:
-            cv2.circle(combined_img, item.center, 7, (0, 0, 255), 4)
-        combined_img = cv2.resize(combined_img, None, fx=0.5, fy=0.5)
-        cv2.imshow("debug img", combined_img)
-        cv2.setWindowProperty("debug img", cv2.WND_PROP_TOPMOST, 1)
-        cv2.waitKey(1)
-
-
-if __name__ == "__main__":
-    run_color_checker()

src/utils/graphic_debugger.py

@@ -0,0 +1,47 @@
+import cv2
+import numpy as np
+from utils.misc import color_filter
+from screen import Screen
+from item_finder import ItemFinder
+from config import Config
+from template_finder import TemplateFinder
+
+
+def run_graphic_debugger():
+    config = Config()
+    screen = Screen(config.general["monitor"])
+    item_finder = ItemFinder()
+    template_finder = TemplateFinder(screen)
+    search_templates = ["A5_TOWN_0", "A5_TOWN_1", "A5_TOWN_2", "A5_TOWN_3"]
+    while 1:
+        img = screen.grab()
+        # Show item detections
+        combined_img = np.zeros(img.shape, dtype="uint8")
+        for key in config.colors:
+            _, filterd_img = color_filter(img, config.colors[key])
+            combined_img = cv2.bitwise_or(filterd_img, combined_img)
+        item_list = item_finder.search(img)
+        for item in item_list:
+            cv2.circle(combined_img, item.center, 7, (0, 0, 255), 4)
+            cv2.putText(combined_img, item.name, item.center, cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 1, cv2.LINE_AA)
+        if len(item_list) > 0:
+            print(item_list)
+        # Show Town A5 template matches
+        scores = {}
+        for template_name in search_templates:
+            success, pos = template_finder.search(template_name, img, threshold=0.65)
+            if success:
+                scores[template_name] = template_finder.last_score
+                cv2.putText(combined_img, str(template_name), pos, cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)
+                cv2.circle(combined_img, pos, 7, (255, 0, 0), thickness=5)
+        if len(scores) > 0:
+            print(scores)
+        # Show img
+        combined_img = cv2.resize(combined_img, None, fx=0.5, fy=0.5)
+        cv2.imshow("debug img", combined_img)
+        cv2.setWindowProperty("debug img", cv2.WND_PROP_TOPMOST, 1)
+        cv2.waitKey(1)
+
+
+if __name__ == "__main__":
+    run_graphic_debugger()

src/utils/misc.py

@@ -5,7 +5,6 @@ from logger import Logger
 import cv2
 from typing import List, Tuple
 import requests
-import subprocess
 from version import __version__
 
 
@@ -44,6 +43,9 @@ def color_filter(img, color_range):
     filtered_img = cv2.bitwise_and(img, img, mask=color_mask)
     return color_mask, filtered_img
 
-def close_down_d2():
-    subprocess.call(["taskkill","/F","/IM","D2R.exe"])
-    subprocess.call(["taskkill","/F","/IM","Battle.net.exe"])
+def hms(seconds: int):
+    seconds = int(seconds)
+    h = seconds // 3600
+    m = seconds % 3600 // 60
+    s = seconds % 3600 % 60
+    return '{:02d}:{:02d}:{:02d}'.format(h, m, s)

src/utils/screen_recorder.py

@@ -1,19 +0,0 @@
-from screen import Screen
-import cv2
-import os
-import keyboard
-
-
-screen = Screen(1)
-
-os.system("mkdir generated")
-
-keyboard.wait("f11")
-i = 0
-while 1:
-    img = screen.grab()
-    img = cv2.resize(img, None, fx=0.5, fy=0.5)
-    cv2.imshow("x", img)
-    cv2.imwrite(f"generated/{i}.png", img)
-    cv2.waitKey(1)
-    i += 1

src/version.py

@@ -1 +1 @@
-__version__ = '0.2.2-dev'
+__version__ = '0.3.0-dev'