0

I'm currently using PyAutoGUI for the locate function that searches a needleImage on a haystackImage. The example that the documentation provides takes in a path to the image. However, I have a function that compares a series of needleImage to a single haystackImages, and reading the same image file over the number of times it needs to check is quite inefficient.

Is there a way to avoid reading the heystackImage each time? If not, are there any alternative to pyautogui/pyscreeze's locate function that uses bufferedImage?

...
checks = {
        "recieve.png": 'recieve',
        "next.png": 'next',
        "start.png": 'start',
        "black.png": 'loading',
        "loading.png": 'loading',
        "gear.png": 'home',
        "factory.png": 'factory',
        "bathtub.png": 'bathtub',
        "refit.png": 'refit',
        "supply.png": 'supply',
        "dock.png": 'dock',

        # SPE
        "spepage.png": 'spe',
        "expeditionpage.png": 'expedition',
        "sortiepage.png": 'sortie',
        "practice.png": 'practice',
        "practiceinfo.png": 'practice',

        "oquest.png": 'quest',
        "quest.png": 'quest'
    }
    for key in checks:
        if (detect(key, cache=True)):
            return checks[key]

def detect(imgDir, confidence=0.85, cache=False):
    if (pyautogui.locate(os.path.join('images', imgDir), 'images\\capture.jpeg', confidence=confidence)) is not None:
        return True
    else:
        return False
Matt
  • 27
  • 7

1 Answers1

1

pyautogui.locate() also accepts numpy arrays and PIL images as inputs. You can read your haystack image into a numpy array (BGR) or PIL image, and pass that instead of the file name.

def detect(imgDir, haystackImage, confidence=0.85, cache=False):
    if (pyautogui.locate(os.path.join('images', imgDir), haystackImage, confidence=confidence)) is not None:
        return True
    else:
        return False

from matplotlib import image
hsImage = image.imread('images\\capture.jpeg')
hsImage = hsImage[:,:,::-1] # convert RGB to BGR
detect('needleImg.png', hsImage, cache=True)

# Alternate method
from PIL import Image
hsImage = Image.open('images\\capture.jpeg')
detect('needleImg.png', hsImage, cache=True)

The second method can be slower than the first since pyautogui.locate() ultimately loads the PIL image as a numpy array, which requires additional processing.

PG11
  • 155
  • 5
  • You're my hero! – Matt Jul 27 '21 at 06:56
  • `hsImage = hsImage[:,:,::-1] # convert RGB to BGR` Is there a reason why you are converting RGB to BGR? Reading from PyScreeze Doc., they seems to use RGB – Matt Jul 28 '21 at 06:29
  • 1
    @Matt PyScreeze ultimately calls [`_locateAll_opencv`](https://github.com/asweigart/pyscreeze/blob/0446e87235e0079f591f0c49ece7d487dedc2f9a/pyscreeze/__init__.py#L180), which loads images through [`_load_cv2`](https://github.com/asweigart/pyscreeze/blob/0446e87235e0079f591f0c49ece7d487dedc2f9a/pyscreeze/__init__.py#L138) Looking through that function, you will see that a numpy array input should be in the same channel order as what is read by [`imread()`](https://docs.opencv.org/3.4/d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56) of opencv. So BGR is the correct order – PG11 Jul 28 '21 at 20:18