2

I'm working with game based on Pygame. In the abstract - I need to run it in headless mode. By now, I'm using:

os.environ['SDL_VIDEODRIVER'] = 'dummy'

And the problem is, that I can't take the screen from game to NumPy array. The one solution which I found actually works, but only with saving the image to the file, what totally misses the point - but maybe it is a hint. I've tried things like parsing Surface to string:

pil_string_image = pygame.image.tostring(screen,"RGBA",False)
im = Image.frombytes("RGBA",(400, 400), pil_string_image)       
cv2.imshow('screen', np.asarray(im))

but afterall I've got only grey screen and that's all.

The reason is, that I need to prepare the game to run in Collab or in text env because I'm doing ML things using game screen.

Is there any way to take the screen properly? Or maybe I could trigger the headless mode in another way, which lets me take image simply like that?

pygame.surfarray.array3d(pygame.display.get_surface())
Rallyholic
  • 317
  • 1
  • 3
  • 21
  • 2
    You should be able to do `buffer = BytesIO()` then `pygame.image.save(..., buffer)` Something similar here... https://stackoverflow.com/a/52281257/2836621 – Mark Setchell Mar 09 '20 at 20:29
  • Thanks, did that, but got errors: 'UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start/continuation byte' and 'TypeError: a bytes-like object is required, not 'str''. Should I give some parameter to opened Buffer? I didn't tried to display that yet, those two instructions which you've given are throwing that errors. – Rallyholic Mar 09 '20 at 20:37
  • Sorry, I’m not near a computer for next few days so I can’t work it out for you at the minute. – Mark Setchell Mar 09 '20 at 20:40
  • Okay, so I would try to do it by myself. Thanks for the effort! If you would have some brilliant idea please let me know :D – Rallyholic Mar 09 '20 at 20:42

2 Answers2

2

It seems screen is a pygame.surface object, presumably returned from pygame.display.set_mode(..). You can call the surfarray methods using screen as the first argument.

E.g.

>>> import os
>>> import pygame
pygame 1.9.6
Hello from the pygame community. https://www.pygame.org/contribute.html
>>> WIDTH, HEIGHT = 640, 480
>>> os.environ["SDL_VIDEODRIVER"] = "dummy"
>>> pygame.init()
(6, 0)
>>> screen = pygame.display.set_mode((WIDTH, HEIGHT))
>>> screen.fill(pygame.color.Color("black"))
<rect(0, 0, 640, 480)>
>>> print(pygame.surfarray.array3d(screen)[0])
[[0 0 0]
 [0 0 0]
 [0 0 0]
 ...
 [0 0 0]
 [0 0 0]
 [0 0 0]]
>>> pygame.draw.line(screen, pygame.color.Color("blueviolet"), (0,0), (0,480))
<rect(0, 0, 1, 481)>
>>> print(pygame.surfarray.array3d(screen)[0])
[[146  36 255]
 [146  36 255]
 [146  36 255]
 ...
 [146  36 255]
 [146  36 255]
 [146  36 255]]
import random
  • 3,054
  • 1
  • 17
  • 22
  • I'm trying my best, but only what I can get is 'ValueError: unsupported bit depth 8 for 3D reference array' error. Tried with different drawing on that screen etc. :C – Rallyholic Mar 10 '20 at 19:37
  • Can you update your question to include a [mcve]? Perhaps the display mode is different? Does `surfarray.array2d()` return an error? – import random Mar 10 '20 at 22:20
  • No, it does not. But by using it, I'm still getting only grey image - where grey refers to arrays filled with zeroes in result image. I'll try to figure out how to reproduce it, but It's a big mess (not only by my fault, thankfully), so propably it could be almost impossible to do. I'm still in big concern, why saving image to file works like a charm, but saving to a buffer or np array doesn't... – Rallyholic Mar 10 '20 at 22:27
  • You answer actually would work, and it works. Sorry for my stupidity. – Rallyholic Mar 10 '20 at 23:32
0

I'm really stupid... Remember kids, when you're trying to diagnose if image was read correctly with OpenCV, sometimes 

    cv2.waitKey(1)

would be your best friend.

Thanks for: cv2.imshow command doesn't work properly in opencv-python

No response from the cv2 window gave me concern if displaying is working correctly...

Rallyholic
  • 317
  • 1
  • 3
  • 21