Sending OpenCV output to VLC stream

Question

This has been keeping me busy for a good part of the afternoon and I haven't been able to get it to work but I feel like I'm really close.

I've got openCV set up which takes the videofeed from a webcam. To be able to access this video feed (with openCV overlay) I want to pipe the output of the openCV python script to a VLC stream. I managed to get the stream up and running and can connect to it. VLC resizes to the correct aspect ratio and resolution so it gets some correct data but the image I get is just Jitter;

python opencv.py | cvlc --demux=rawvideo --rawvid-fps=30 --rawvid-width=320 --rawvid-height=240  --rawvid-chroma=RV24 - --sout "#transcode{vcodec=h264,vb=200,fps=30,width=320,height=240}:std{access=http{mime=video/x-flv},mux=ffmpeg{mux=flv},dst=:8081/stream.flv}" &

The output of the script is a constant video feed sent to stdout as follows

from imutils.video import WebcamVideoStream

vs = WebcamVideoStream(src=0)

while True: 
  frame = vs.read()
  sys.stdout.write(frame.tostring())

Above example is a dumbed down version of the script I'm using; Also as seen I'm making use of the imutils library; https://github.com/jrosebr1/imutils

If anyone could give me a nudge in the right direction I would appreciate it greatly. My guess is the stdout.write(frame.tostring()) is not what vlc expects but I haven't been able to figure it out myself.

Basically a really distorted image which moves upwards diagonally. Not really related to the actual video feed from the webcam as far as i could figure out. (It didn't change when i moved objects in front of the webcam) So i'm guessing VLC cannot interpet the frame.tostring() data. — KPNT, Apr 12 '16 at 17:14
its probably a mismatch in video format. Are you sure that RV24 is correct output format? Also ensure that the dimensions match — Dimitri Podborski, Apr 12 '16 at 17:24
Good point about the formats. They currently do not match. About RV24 i don't know. I'm no opencv or vlc expert. I mixmatched the stream creation command from examples and documentation. Any advice on how to figure out the correct output format? Thanks in advance @incBrain — KPNT, Apr 12 '16 at 18:03
you can try to search for CV_CAP_PROP_FOURCC or CV_CAP_PROP_FORMAT. Maybe this will help you to figure it out — Dimitri Podborski, Apr 12 '16 at 18:18
np. Don't forget to post an answer after you fixed it. It's interesting question. :) — Dimitri Podborski, Apr 12 '16 at 18:22
I gave up after struggling with it a few days. It was an internship project which is now finished. I didn't try the answer from @SergiusBond yet. Maybe that will help you out. Sorry — KPNT, Dec 18 '16 at 11:20

score 3 · Accepted Answer · answered Sep 25 '18 at 20:59

3

The following works for me under Python 3

import numpy as np
import sys
import cv2

cap = cv2.VideoCapture(0)

while(cap.isOpened()):
    ret, frame = cap.read()
    if ret==True:        
        sys.stdout.buffer.write(frame.tobytes())
    else:
        break

cap.release()

And the command line (my webcam has a different resolution, and I only display the result, but you did not have problems with that)

python opencv.py | vlc --demux=rawvideo --rawvid-fps=25 --rawvid-width=640 --rawvid-height=480 --rawvid-chroma=RV24 - --sout "#display"

Of course this requires a conversion from BGR to RGB as the former is default in OpenCV.

answered Sep 25 '18 at 20:59

Atnas

594
4
16

May you please let me know further how could we take this stream over web. Or by any other stream player? – Rupesh Arora Jun 15 '20 at 05:13
Its worth noting here that the sys.stdout.buffer doesn't exist in IDLE3, to test this you can't run it from IDLE as it will error. Use "python -I filename" in a terminal instead – Prophacy Nov 14 '22 at 18:42

score 2 · Answer 2 · answered Nov 17 '16 at 02:56

This worked for me, though I am sending to RTSP stream and not using imutils library:

import numpy as np
import sys
import cv2

input_rtsp = "rtsp://10.10.10.9:8080"
cap = cv2.VideoCapture(input_rtsp)

while(cap.isOpened()):
    ret, frame = cap.read()
    if ret==True:        
        sys.stdout.write(frame.tostring())
    else:
        break

cap.release()

Then in command line:

python opencv.py | cvlc --demux=rawvideo --rawvid-fps=25 --rawvid-width=1280 --rawvid-height=720  --rawvid-chroma=RV24 - --sout "#transcode{vcodec=h264,vb=200,fps=25,width=1280,height=720}:rtp{dst=10.10.10.10,port=8081,sdp=rtsp://10.10.10.10:8081/test.sdp}"

Note that you do not need to convert opencv BGR to RGB.

Sending OpenCV output to VLC stream

2 Answers2

Linked