1

I would like to recognize objects of windows applications, mainly computer games. I would like to accomplish this by opening the window in OpenCV and applying all kinds of effects to the game application under execution and recognize objects such as UI elements, messages and even characters that are on the screen.

Since OpenCV only allows video and webcam as input, is there a way to open a running application as a source for OpenCV?

There maybe some testing applications used in game development that use similar methods for testing, but I couldn't find any.

I also don't know the right terms that that are used when discussing the recognition of virtual objects of a computer program or computer game, but that is exactly what I would like to do.

I tried to look up forums, articles or anything written about this, but found nothing. Any help would be appreciated.

Disane
  • 25
  • 4

2 Answers2

0

I used OpenCV for a year and I am not sure if you can pass the running application to it.

As an alternative, you might like to have a function which gives you the current screenshot of the desktop. So you can get it and pass to OpenCV.

This way you can periodically do screenshots and use your recognition.

If you are working under Windows you might need this discussion.

Hope this will somehow help you.

Community
  • 1
  • 1
mbaros
  • 825
  • 8
  • 31
  • Yep, this is the part that I already figured out. The problem is that I need to store the images on memory and not in files (which would destroy any HDD in the long run). I'm assuming that if I store a BMP screenshot in memory (doesn't matter if its assigned to a Python variable/reference or C++ pointer) I can still access it using OpenCV functions (no matter which language I'm using to call the API) and carry out the operations that I need. Am I wrong or is this the really the way I imagine it? – Disane Jul 18 '16 at 22:10
0

I've been trying all method using desktop as source in opencv few months ago as my experiment project and i success, but there's a trail inside the windows, as maybe the speed or processor depends on it. It using your own function to use desktop as source not from opencv libraries. And from there i continued custom things and got stuck in some bug that has something from memory, as it use lots of memory. I did post on stackoverflow. Hope this information helps.

mas bro
  • 312
  • 1
  • 12