OpenCV VideoCapture unable to parse MJPG stream

Question

I am trying to open a video stream with the OpenCV VideoCapture class in OpenCV version 3.1, Windows x64. On my raspberry pi I have mjpg_streamer running, and I can see the output through http://192.168.1.245:8080/?action=stream , however when I try to open the video stream in OpenCV, it fails to open the stream.

This is the code I'm using to debug that apparently worked for someone else who was having connection issues also.

#include <opencv2\core.hpp>
#include <opencv2\videoio.hpp>
#include <string>
#include <iostream>
using namespace std;

int main()
{
    cv::VideoCapture vcap;
    cv::Mat raw_image;

    const string videoStreamAddress = "http://192.168.1.245:8080/?action=stream";

    if (!vcap.open(videoStreamAddress))
    {

        cout << "Error opening video stream" << endl;
        system("pause");
        return -1;
    }

    cout << "Stream opened" << endl;
    system("pause");
    return 0;
}

Online, people are saying that OpenCV must have the video extension in the link. I have tried to use the extension trick that other people are using, like http://192.168.1.245:8080/?action=stream?dummy=param.mjpg, http://192.168.1.245:8080/?action=stream&type=.mjpg, &channel=0&.mjpg, and &type=.mjpeg but that is not working. Also, I have enabled ffmpeg in cmake and built with it. It seems like at this point it works for other people, and there doesn't seem to be anything else on the topic. What is the solution to this?

Answer 1

Thanks to @api55 's suggestion, I was able to fetch the stream and then pass the raw jpg data to opencv and do what I wanted to do in the first place.

Here's the c++ code for the solution in the link @api55 provided (link to python solution).

    try
    {

        boost::asio::io_service io_service;

        tcp::resolver resolver(io_service);
        tcp::resolver::query query("192.168.1.245", "8080");
        tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);

        tcp::socket socket(io_service);
        boost::asio::connect(socket, endpoint_iterator);

        std::string jpgimg = "";

        std::stringstream request_;

        request_ << "GET /?action=stream HTTP/1.1\r\n";
        request_ << "Host: 192.168.1.245\r\n";
        request_ << "Accept-Encoding: *\r\n";
        request_ << "\r\n";

        boost::system::error_code ignored_error;
        boost::asio::write(socket, boost::asio::buffer(request_.str()), ignored_error);

        for (;;)
        {
            char buf[1025];
            buf[1024] = '\0';
            boost::system::error_code error;

            size_t len = socket.read_some(boost::asio::buffer(buf,1024), error);

            if (error == boost::asio::error::eof)
                break; // Connection closed cleanly by peer.
            else if (error)
                throw boost::system::system_error(error); // Some other error.

            jpgimg.append(buf,buf+len);

            int a = jpgimg.find("\xff\xd8");
            int b = jpgimg.find("\xff\xd9");

            if (a != -1 && b != -1)
            {
                Mat rawData(1, b-a+2, CV_8UC1, (void*)(&jpgimg[a]));

                Mat i = cv::imdecode(rawData, CV_LOAD_IMAGE_COLOR);
                cv::imshow("i", i);
                if (cv::waitKey(1) == 27)
                break;

                jpgimg = jpgimg.substr(b+2);
            }

        }
    }
    catch (std::exception& e)
    {
        std::cerr << e.what() << std::endl;
    }

It is by no means optimized and is purely a base for implementations elsewhere. It works with Boost ASIO 1.65.1 by opening a stream to mjpg_streamer on my raspberry pi (on port 8080), sending it a HTTP GET request for the /?action=stream part of the original link, looking for the jpg data beginning and ending flags ("\xff\xd8" and "\xff\xd9"), then sending this data off to OpenCV through imdecode.

Resources:

Boost ASIO Client Code: http://www.boost.org/doc/libs/1_65_1/doc/html/boost_asio/tutorial/tutdaytime1.html

Info on how jpgs work: How to parse MJPEG HTTP Stream within C++?

Using raw jpg data in OpenCV: opencv read jpeg image from buffer

Hopefully OpenCV will have better mjpg support in the future.