1

I'm trying to stream from Raspberry PI camera over network using raspivid and gstreamer cli. I want to be able to view the stream using VLC "open network stream" on the client.

This is related to question GStreamer rtp stream to vlc however mine it is not quite the same. Instead of encoding the raw output from my PI camera, my idea is to leverage the existing the h264 output of raspivid, mux it into an appropriate container and send it over TCP or UDP.

I was able to successfully capture the h264 output from raspivid into an mp4 file (with correct fps and length information) using this pipeline:

raspivid -n -w 1280 -h 720 -fps 24 -b 4500000 -a 12 -t 30000 -o - | \
gst-launch-1.0 -v fdsrc ! video/x-h264, width=1280, height=720, framerate=24/1 ! \ 
h264parse ! mp4mux ! filesink location="videofile.mp4"

However, when I try to stream this over a network:

raspivid -n -w 1280 -h 720 -fps 24 -b 4500000 -a 12 -t 0 -o - | \
gst-launch-1.0 -v fdsrc ! video/x-h264, width=1280, height=720, framerate=24/1 ! \
h264parse ! mpegtsmux ! rtpmp2tpay ! udpsink host=192.168.1.20 port=5000

...and try to open the stream using rtp://192.168.1.20:5000 on VLC, it reports an error.

Edit: Ok, I was mistaken to assume that the udpsink listens for incoming connections. However, after changing the last part of the pipeline to use my client's IP address ! udpsink host=192.168.1.77 port=5000 and tried opening that with udp://@:5000 on the VLC, the player does not display anything (both the PI and the receiving computer are on the same LAN and I can see the incoming network traffic on the client).

Does anyone know how to properly construct a gstreamer pipeline to transmit existing h264 stream over a network which can be played by vanilla VLC on the client?

Milos Mrdovic
  • 1,441
  • 1
  • 17
  • 30

1 Answers1

4

Assuming this is due to missing SPS/PPS data. E.g. probably it works if you start VLC first and then the video pipeline on the Raspberry PI. By default the SPS/PPS headers are most likely send only once at the beginning of the stream.

If the receiver misses SPS/PPS headers it will not be able to decode the H.264 stream. I guess this can be fixed by using the config-interval=-1 property of h264parse.

With that option SPS/PPS data should be send before each IDR-frame which should occur every couple of seconds - depending on the encoder.

Another thing is that you don't need to use rtpmp2tpay block. Just sending MPEG TS over UDP directly should be enough.

Having said that, the pipeline should look like this:

raspivid -n -w 1280 -h 720 -fps 24 -b 4500000 -a 12 -t 0 -o - | \
gst-launch-1.0 -v fdsrc ! \
video/x-h264, width=1280, height=720, framerate=24/1 ! \
h264parse config-interval=-1 ! mpegtsmux ! udpsink host=192.168.1.77 port=5000

The 192.168.1.77 is the IP address of the client running VLC at udp://@5000. Also, make sure no firewalls are blocking the incoming UDP trafict towards the client (Windows firewall, in particular).

Milos Mrdovic
  • 1,441
  • 1
  • 17
  • 30
Florian Zwoch
  • 6,764
  • 2
  • 12
  • 21
  • Thanks. I've tried with `raspivid -n -w 1280 -h 720 -fps 24 -b 4500000 -a 12 -t 0 -o - | gst-launch-1.0 -v fdsrc ! video/x-h264, width=1280, height=720, framerate=24/1 ! h264parse config-interval=-1 ! mpegtsmux ! rtpmp2tpay ! udpsink host=192.168.1.77 port=5000`, however the result is the same. The player just sits there without throwing any errors... Any other ideas? – Milos Mrdovic Jul 15 '19 at 13:23
  • `config-interval : Send SPS and PPS Insertion Interval in seconds (sprop parameter sets will be multiplexed in the data stream when detected.) (0 = disabled, -1 = send with every IDR frame) ` – Florian Zwoch Jul 15 '19 at 15:54
  • But since it still does not work there may be something else wrong. But nothing catches my eye immediately. Maybe you need some more digging why VLC does not want to play (or somehow confirm the SPS/PPS data is indeed send) – Florian Zwoch Jul 15 '19 at 15:56
  • 1
    P.S. Is there a reason you need to send MPEG TS over RTP? How about just sending MPEG TS over UDP directly? For RTP you will need some SDP info for VLC or an RTSP server. – Florian Zwoch Jul 15 '19 at 16:01
  • Thanks for the clarification Florian. No particular reason other than my limited gstreamer knowledge. I tried `! mpegtsmux ! ! udpsink host=192.168.1.77 port=5000` but that didn't work either... Already spent two days trying to get this to work without success... :( – Milos Mrdovic Jul 15 '19 at 16:37
  • I suggest you read up a bit upon the different streaming protocols and how they work in order to select the right one for your use case. It sounds to me that MPEG TS over UDP is a good start for you. Not sure about how to set the IP addresses in your use case though. Perhaps streaming to a multicast IP helps. – Florian Zwoch Jul 15 '19 at 17:54
  • 1
    I've found the issue: In addition to your recommendations (using `config-interval=-1` and removing the `rtpmp2tpay` part, it was actually the firewall that blocked the UDP traffic interfering with the streaming. I've edited your answer to include those 2 things. Thanks a lot man! – Milos Mrdovic Jul 18 '19 at 09:54