Accelerating a screenshot function - Python

Question

I need my screenshot function to be as fast as possible, and now every call to the function takes about 0.2sec.

This is the function:

def get_screenshot(self, width, height):
    image = self.screen_capture.grab(self.monitor)
    image = Image.frombuffer('RGB', image.size, image.bgra, 'raw', 'BGRX')
    image = image.resize((int(width), int(height)), Image.BICUBIC) # Resize to the size of 0.8 from original picture
    image = np.array(image)
    image = np.swapaxes(image, 0, 1)
    # This code below supposed to replace each black color ([0,0,0]) to the color of [0,0,1]
    # r1,g1,b1 = [0,0,0] and r2,g2,b2 = [0,0,1]
    red, green, blue = image[:, :, 0], image[:, :, 1], image[:, :, 2]
    mask = (red == r1) & (green == g1) & (blue == b1)
    image[:, :, :3][mask] = [r2, g2, b2]
    return image

Do you notice any changes that I can do to make the function faster?

Edit: Some details that I forgot to mention:

1. My screen dimensions are 1920*1080

2. This function is a part of a live stream project that I am currently working on. The solution that Carlo has suggested below is not appropriate in this case because the remote computer will not be synchronized with our computer screen.

Answer 1

As your code is incomplete, I can only guess what might help, so here are a few thoughts...

I started with a 1200x1200 image, because I don't know how big yours is, and reduced it by a factor of 0.8x to 960x960 because of a comment in your code.

My ideas for speeding it up are based on either using a different interpolation method, or using OpenCV which is highly optimised SIMD code. Either, or both, may be appropriate, but as I don't know what your images look like, only you can say.

So, here we go, first with PIL resize() and different interpolation methods:

# Open image with PIL
i = Image.open('start.png').convert('RGB')

In [91]: %timeit s = i.resize((960,960), Image.BICUBIC)                                             
16.2 ms ± 28 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [92]: %timeit s = i.resize((960,960), Image.BILINEAR)                                            
10.9 ms ± 87.2 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [93]: %timeit s = i.resize((960,960), Image.NEAREST)                                             
440 µs ± 10.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

So, BILINEAR is 1.5x faster than BICUBIC and the real winner here is NEAREST at 32x faster.

Now, converting to a Numpy array (as you are doing anyway) and using the highly optimised OpenCV SIMD code to resize:

# Now make into Numpy array for OpenCV methods
n = np.array(i)

In [100]: %timeit s = cv2.resize(n, (960,960), interpolation = cv2.INTER_CUBIC)                     
806 µs ± 9.81 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [101]: %timeit s = cv2.resize(n, (960,960), interpolation = cv2.INTER_LINEAR)                    
3.69 ms ± 29 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [102]: %timeit s = cv2.resize(n, (960,960), interpolation = cv2.INTER_AREA)                      
12.3 ms ± 136 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [103]: %timeit s = cv2.resize(n, (960,960), interpolation = cv2.INTER_NEAREST)                   
692 µs ± 448 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each)

And the winner here looks like INTER_CUBIC which is 20x faster than PIL's resize().

Please try them all and see what works for you! Just remove the Python magic %timeit at the start of the line and run what's left.

Answer 2

This is just an example of what I mean. If it solve the problem let me know.

You can create two different thread. One that take the screenshot, the other that elaborate the screen later. Both add the result to a list. This improve the speed of the get_screenshot function. But for elaborate it you need the time that is required for the function execution.

import threading
#import your stuff

class your_class(object):
        def __init__(self):

                self.images = list()
                self.elaborated_images = list()

                threading.Thread(name="Take_Screen", target=self.get_screenshot, args=(width, height))
                threading.Thread(name="Elaborate_Screen", target=self.elaborate_screenshot)

        def get_screenshot(self, width, height):
                while True:
                        images.append(self.screen_capture.grab(self.monitor))

        def elaborate_screenshot(self):
                while True:
                        image = self.images[0]
                        image = Image.frombuffer('RGB', image.size, image.bgra, 'raw', 'BGRX')
                        image = image.resize((int(width), int(height)), Image.BICUBIC) # Resize to the size of 0.8 from original picture
                        image = np.array(image)
                        image = np.swapaxes(image, 0, 1)

                        # This code below supposed to replace each black color ([0,0,0]) to the color of [0,0,1]
                        # r1,g1,b1 = [0,0,0] and r2,g2,b2 = [0,0,1]

                        red, green, blue = image[:, :, 0], image[:, :, 1], image[:, :, 2]
                        mask = (red == r1) & (green == g1) & (blue == b1)
                        image[:, :, :3][mask] = [r2, g2, b2]

                        del self.images[0]

                        self.elaborated_images.append(image)

your_class()

Because I don't have your full code I can't build it the best as possible.