9

I am using Python 3.5.1 and OpenCV 3.0.0.

I am working on a python program that can play games, so it needs to 'see' what is going on, on the screen. How can this be achieved? 

import numpy as np
import cv2
cap = cv2.VideoCapture(0)
while(True):
    #work with frames here

Is there a int 'a' such that cv2.VideoCapture(a) will take the desktop screen as video input? I tried making it and I followed a rather clumsy approach, I captured screen repeatedly using:

import os
os.system('screencapture test.jpg')

Then opening test.jpg using cv2.imread. This was a very slow approach, on searching online I found this question Screen Capture with OpenCV and Python-2.7 which does the same thing, but more efficiently. But the fact still remains that it is capturing individual screenshots and processing them one by one and not a true video stream. I also found this How to capture the desktop in OpenCV (ie. turn a bitmap into a Mat)? which I think is close to what I am trying but is in C++, if someone can help me convert this to Python, I will highly appreciate it.

The main thing is that the program will be doing something like MarI/O, so speed is a concern, any help is appreciated, go easy on me, I am (relatively) new to OpenCV.

Thanks.

Community
  • 1
  • 1
Divyanshu Kalra
  • 6,420
  • 1
  • 15
  • 17

2 Answers2

1

Just an update on this question in case anyone wants a solution.

Taking screenshot can be achieved by using module pyautogui

import pyautogui
import matplotlib.pyplot as plt

image = pyautogui.screenshot()
plt.imshow(image)

If you want to read it as a stream, 

while(True):
    image = pyautogui.screenshot()
    #further processing
    if finished:
        break

According to the documentation,

On a 1920 x 1080 screen, the screenshot() function takes roughly 100 milliseconds

So this solution can be used if your application does not demand high fps rate.

Danny Fang
  • 3,843
  • 1
  • 19
  • 25
  • To further improve FPS, you could create a separate thread just for taking screenshots while having the main thread process the frames. You could place the screenshots into a queue so the main thread simply reads the next frame without having to wait for I/O operations. – nathancy Mar 22 '19 at 23:43
1

Taking screenshots in separate thread sounds good solution. Also you can use virtual webcam, but it is a heavy solution. Or you would capture desktop directly by using ffmpeg. https://trac.ffmpeg.org/wiki/Capture/Desktop

Владимир Демон
  • 11
  • 2