0

I am trying to use FFmpeg in python as a subprocess for capturing screen and converting to numpy array using pipe. This for a desktop sharing software. I have two codes I've written:

1st case: It doesn't work at all except for the FFmpeg cmd.

Code:

    cmd = "ffmpeg -f gdigrab -framerate 30 -i desktop -f h264 pipe:1"

pipe = subprocess.run(cmd,
                       stdout=subprocess.PIPE,
                       stderr=subprocess.PIPE,
                       bufsize=2)

print(pipe.stdout, pipe.stderr)

2nd case: It works but I get a numpy array every 3 - 4 seconds only.

Code:

cmd = 'ffmpeg -f gdigrab -framerate 30 -i desktop -f h264 pipe:1'

size = 480 * 240 * 3
proc = sp.Popen(cmd, stdout=sp.PIPE)

while True:
    try:
        tic = time.perf_counter()
        frame = proc.stdout.read(size)
        #print(frame)
        if frame is None:
            print('no input')
            break
        image = np.frombuffer(frame, dtype='uint8').reshape(240, 480, 3)
        toc = time.perf_counter()
        print(f"performed calc in {(toc - tic) * 1000:0.4f} miliseconds")
        cv2.imshow('received', image)
    except Exception as e:
        print(e)

cv2.destroyAllWindows()
proc.stdin.close()
proc.wait()

I think it is buffering the frames during that 3 seconds, but that would be terrible for a desktop sharing setup. I am trying to get as low latency between capturing and streaming as possible.

Any help would be greatly appreciated!

guidingfox
  • 77
  • 2
  • 12
  • module [vidgear](https://abhitronix.github.io/vidgear/v0.2.1-stable/) has class [ScreenGear](https://abhitronix.github.io/vidgear/v0.2.1-stable/gears/screengear/overview/) to grab screen. It has also [StreamGear](https://abhitronix.github.io/vidgear/v0.2.1-stable/gears/streamgear/overview/) for streaming. – furas May 06 '21 at 08:06
  • or maybe you should try to use module [ffmpeg-python](https://github.com/kkroening/ffmpeg-python) – furas May 06 '21 at 08:09
  • or maybe you should grab it in separated thread and then proccess it main thread. – furas May 06 '21 at 08:11
  • @furas I did find StreamGear while searching, but I wanted to implement it kinda from scratch. And when I use `proc = sp.Popen(cmd, stdout=sp.PIPE)` it does create another thread, right? – guidingfox May 06 '21 at 08:17
  • `Popen` runs `ffmpeg` in `process` but rest you have to do in main process - if you move it into thread then you can run more code in separated thread - ie. frombuffer(). Here example with cv2 and webcam [Increasing webcam FPS with Python and OpenCV](https://www.pyimagesearch.com/2015/12/21/increasing-webcam-fps-with-python-and-opencv/) – furas May 06 '21 at 09:14
  • I was scratching my had regarding latency issues in [this](https://stackoverflow.com/questions/60462840/ffmpeg-delay-in-decoding-h264) post. There are many factors that needs to be considered... – Rotem May 06 '21 at 19:37

0 Answers0