pyautogui / pyscreeze - locate works w/o region parameter, but doesn't work w/ region parameter

Question

Using pyautogui, I'm trying to locate an image on screen.

This finds the image (on primary monitor w/ 2560x1440 resolution), but takes nearly 5 sec:

icon1 = pyautogui.locateOnScreen('[filepath]\\icon1.png', grayscale=True)

To reduce search time, I provide a region parameter, but this doesn't find the image. It executes in ~0.7 sec

icon1 = pyautogui.locateOnScreen('[filepath]\\icon1.png', region=(1500,100,950,1100), grayscale=True)

Eliminating the grayscale parameter doesn't change the result - still ~0.7 execution time and image not found:

icon1 = pyautogui.locateOnScreen('[filepath]\\icon1.png', region=(1500,100,950,1100))

Then I tried setting the region parameter to the entire screen, but image not found and execution time is ~1.5sec, so it's searching (judging by longer execution time w/ larger region), just not finding.

icon1 = pyautogui.locateOnScreen('[filepath]\\icon1.png', region=(0,0,2559,1439))

Any suggestions on what to try next? Thanks.

Answer 1

I figured out what's happening. As a preface, I am working with multiple monitors, and have made modifications to pyautogui and pyscreeze as described in these 2 pyautogui github comments: https://github.com/asweigart/pyautogui/issues/9#issuecomment-527236475 https://github.com/asweigart/pyautogui/issues/321#issuecomment-629819495

With these modifications, pyautogui works with multiple monitors, and pyautogui.position() and pyautogui.moveTo() treat the upper left corner of the primary monitor as coordinate (0,0) - this means monitors above or to the left of primary monitor will have at least one negative coordinate.

But pyscreeze is returning image search results based on treatment of the upper left corner of the upper left monitor as coordinate (0,0) - so all coordinates on extended desktop are positive.

The region parameter I was passing to locateOnScreen() was based on using pyautogui.position() reported coordinates, so the search was being executed on an incorrect area of the screen.

Once I figured that out, I was able to pass the correct region specification.

To illustrate, I inserted a print statement in pyscreeze to show where the image is found. And a print of the image search return. And in my code, I printed pyautogui.position().

retVal = Box(left=8008, top=3171, width=29, height=31)

Point(x=4182, y=1026)

mouse position:
Point(x=4182, y=1026)

So you can see that coordinates are quite different. Pyscreeze is showing coordinates based on 0,0 being upper left corner of upper left monitor, while the image search return variable are coordinates based on 0,0 being upper left corner of primary monitor, so I'm able to take the image search result variable and pass that to pyautogui.moveTo() and move the mouse there (moveTo needs coordinates based on 0,0 being upper left corner of primary monitor).

The issue stems from adopting changes from two different people to pyautogui and pyscreeze. I suppose I could adjust the changes to one of these packages to offset the coordinates so that both are referencing the same monitor's upper left corner as coordinate (0,0).

Answer 2

Your region is set wrong. Use (0,0,2559,1439).

And if this doesn't work make a new screenshot specifying the region for it, save it and look if that just captured screenshot would be found on the screen.

The code below should always work as expected:

from time import perf_counter as T
import pyautogui
# NOTICE that in pyautogui: region=(left, top, width, height)
pyautogui.screenshot('pyautogui_region_screenshot.png', region=(300, 500, 300, 200))
sT = T()
img = pyautogui.locateOnScreen('pyautogui_region_screenshot.png', region=(300, 500, 300, 200))
eT = T()
print(f'img = pyautogui.locateOnScreen took: {eT-sT:9.5f} seconds')
print(img)

By the way: check out https://stackoverflow.com/questions/72410958/taking-a-screenshot-of-a-specific-portion-of-the-screen-in-python to switch to another Python module ( mss ) for doing screenshots of small regions of the screen.

https://pyautogui.readthedocs.io/en/latest/screenshot.html