Using gstreamer with Python OpenCV to capture live stream?

Question

First of all I have Python 3 with the Gstreamer library in it.

print(cv2.getBuildInformation())

It shows Gstreamer with YES next to it.

Here is the transmitter code using gstreamer in RaspberryPi 3:

gst-launch-1.0 v4l2src device="/dev/video0" ! video/x-raw,width=320,height=240 ! videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host='my ip address' port=10000

and I will be using Python code to determine shapes, recognize objects, etc..

Here is my Python code:

import numpy as np
import cv2




def receive():
cap = cv2.VideoCapture("udpsrc port=10000 ! application/x-rtp,encoding-name=H264 ! rtpjitterbuffer ! rtph264depay ! avdec_h264 ! videoconvert ! ximagesin ", cv2.CAP_GSTREAMER)

while True:

    ret,frame = cap.read()

    if not ret:
        print('empty frame')
        continue 


    cv2.imshow('receive', frame)
    if cv2.waitKey(1)&0xFF == ord('q'):
        break


cap.release()



receive();

but it always shows empty frame.

When I tried this command in the terminal:

gst-launch-1.0 udpsrc port=10000 ! application/x-rtp,encoding-name=H264 ! rtpjitterbuffer ! rtph264depay ! avdec_h264 ! videoconvert ! ximagesink

it works just fine, so the problem is on my Python end.

What do you recommend?

In your python code, you wrote `ximagesin`, maybe that's the problem? — user8408080, Oct 18 '18 at 18:20

score 1 · Answer 1 · answered Oct 18 '18 at 22:15

cap = cv2.VideoCapture('udpsrc port=7000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! appsink', cv2.CAP_GSTREAMER)

use this pipeline, it will work just fine. change the port to the one you are using.

score 0 · Answer 2 · answered Oct 29 '18 at 12:45

I use different send-receive scripts that give me way better video latency (allso at higher resolutions) when run from commandline, but I have no clue how to import the resulting stream into OpenCV on the Ubuntu end.

(My OpenCV is also installed with ffmpeg and gstreamer ON.)

My GStreamer transmitter code on my Raspberry pi:

gst-launch-1.0 -v v4l2src ! video/x-raw,width=320,height=240 ! videoconvert ! jpegenc ! rtpjpegpay ! udpsink host=192.168.1.101 port=5200

My Gstreamer Receiver code at the Ubuntu 16.04 end:

gst-launch-1.0 -v udpsrc port=5200 ! application/x-rtp, encoding-name=JPEG,payload=26 ! rtpjpegdepay ! jpegdec ! videoconvert ! autovideosink

I hope this will help you improve your stream quality and latency. And I hope somebody may be able to sample me the OpenCV pipeline import code :)

Greetings, Peter Lunk

Using gstreamer with Python OpenCV to capture live stream?

2 Answers2

Linked