c++ opencv webcam stream to html

Question

I am currently developing a project for my studies where I have to fetch a webcam stream, detect some objects and put some additional information on this stream. This is all done on the server side.

Now I also have to provide the modified image of the stream to the clients. The clients just open a HTML file with the following content:

<html>
    <head>
        <title></title>
    </head>
    <body>
        <h1>It works!</h1>
        <video width="320" height="240" src="http://127.0.0.1:4711/videostream" type="video/quicktime" autoplay controls>
            Your browser does not support the video tag.
        </video>
    </body>
</html>

This will result in a HTTP request on the server for /videostream. To handle this request on the server side I will use Boost 1.56.

Currently each frame of my webcam stream is of type IplImage. Do I have to convert the IplImage into a video MIME-Typespecific format?

I have tried to figure it out myself, how the whole thing is working, but I couldn't get it. I used Wireshark to analyze the communication, but it doesn't make sense. For testing purpose I have uploaded a video to my webspace and open the above file locally. The src of the video was the address of my webserver. First there is the TCP handshake stuff followed by this message:

HTTP    765 GET /MOV_4198.MOV HTTP/1.1

Followed the following message (it contains connection: Keep-Alive in the HTTP part):

HTTP    279 HTTP/1.1 304 Not Modified

Afterwards only TCP ACK and SYN folow, but no data. See the following picture: see picture

Where and how are the real data of the video sent? What have I missed here?

Would be great if you could provide me some information about the connection between the Browser (video-Tag) and the C++ socket connection.

Thank you, Stefan

[mjpeg](http://www.damonkohler.com/2010/10/mjpeg-streaming-protocol.html) is probably the most easy way to do this. (and please do not use opencv's deprecated c-api. IplImages are dead since 2010) — berak, Oct 26 '14 at 19:37
Thanks - I will have a look at it. About the dead IplImages: I was using the following code: capture = cvCaptureFromCAM(CV_CAP_ANY); IplImages* frame = cvQueryFrame(capture); I found [this post](http://docs.opencv.org/java/org/opencv/highgui/VideoCapture.html) after some research about the deprecated api. When you download OpenCV you can find in the following file: source\samples\cpp\starter_video.cpp; it is a great example how it is done correctly. (this was mentioned in the post I've found). Thanks a lot for the hint! — Skully, Oct 29 '14 at 07:03

score 10 · Answer 1 · edited May 23 '17 at 12:01

I want to share my made experiences - maybe it will help others too. To get my stream from the webcam I used OpenCV 2.4.9 and as protocol I used mjpeg streaming protocol (see also MJPEG over HTTP) - thanks to @berak - he mentioned MJPEG in his comment under my question-post.

The following code just gives an overview - I do not go into threading details. Since this is a students project and we are using GitHub, you can find the whole source code here on GitHub - project Swank Rats I want to mention here, that I am not a C++, OpenCV or Boost guru. This project is the first time I use all three of them.

Get you stream from your webcam

Do something like this (full code with threading and so search for WebcamService in the repo)

cv::VideoCapture capture();
cv::Mat frame;
while (1) {
    if (!capture.isOpened()) {
        break; //do some logging here or something else - webcam not available
    }

    //Create image frames from capture
    capture >> frame;

    if (!frame.empty()) {
        //do something with your image (e.g. provide it)
        lastImage = frame.clone();
    }
}

Provide your image via HTTP

Well I don't go into detail how you create a HTTP server with C++. There is a nice example provided by Boost for C++11. I have copied this code and adapted it to my needs. You can find the source code of my implementation in the repo mentioned above. The code is currently located at infrastructure / networking / videostreaming.

There is no need to use FFMPEG, GStreamer or something simular. You can create a in-memory JPEG using OpenCV like this (see code of StreamResponseHandler):

cv::Mat image = webcamService->GetLastImage();
// encode mat to jpg and copy it to content
std::vector<uchar> buf;
cv::imencode(".jpg", image, buf, std::vector<int>());

std::string content(buf.begin(), buf.end()); //this must be sent to the client

Thanks to @codeDr for his post here.

The content variable represents the image in bytes, which will be sent to the client. You have to follow the protocol of MJPEG.

Connect to the server by using HTML

Something like this is enough (as mentioned here)

<html>  
    <body>  
        <h1> Test for simple Webcam Live streaming </h1>  
        <img src="http://127.0.0.1:4711/videostream">
    </body>  
</html>

You have to change the IP, port and so on to your server connection.

I hope this helps.

The equivalent for Python is [here](https://gist.github.com/n3wtron/4624820). A minimal project showing one blue image can be found [here](https://github.com/VictorLamoine/Simple-Web-Server). — Victor Lamoine, Nov 30 '15 at 15:18
Hi, this seems to be consuming a lot of CPU resource. I wonder if `imencode` is the cause of this? Any idea? — Cabara, Oct 05 '20 at 08:33
@Skully can you please help me with [this question](https://stackoverflow.com/questions/65811174/how-to-listen-to-an-event-using-sockets)? — Hadi GhahremanNezhad, Jan 23 '21 at 14:53

Zabir Al Nazi · Answer 2 · 2021-06-15T12:40:12.010

I may be a little late, but as I didn't find a completely updated solution for C++ and mjpeg in StackOverflow, thought about writing a new answer.

There are now some good and simple libraries for the task in C++ (c++ mjpg streaming to html)

https://github.com/nadjieb/cpp-mjpeg-streamer

https://github.com/jacksonliam/mjpg-streamer

https://github.com/codewithpassion/mjpg-streamer/tree/master/mjpg-streamer

I found the first one to be very simple. You need CMake, and make installed in the system.

git clone https://github.com/nadjieb/cpp-mjpeg-streamer.git;
cd cpp-mjpeg-streamer;
mkdir build && cd build;
cmake ../;
make;
sudo make install;

Make sure you have the correct version of OpenCV installed.

Now, write the streamer:

mjpeg_server.cc

#include <opencv2/opencv.hpp>

#include <nadjieb/mjpeg_streamer.hpp>

// for convenience
using MJPEGStreamer = nadjieb::MJPEGStreamer;

int main()
{
    cv::VideoCapture cap;
    cap.open("demo.mp4"); 
    if (!cap.isOpened())
    {
        std::cerr << "VideoCapture not opened\n";
        exit(EXIT_FAILURE);
    }

    std::vector<int> params = {cv::IMWRITE_JPEG_QUALITY, 90};

    MJPEGStreamer streamer;

    // By default 1 worker is used for streaming
    // if you want to use 4 workers:
    //      streamer.start(8080, 4);
    streamer.start(8000);

    // Visit /shutdown or another defined target to stop the loop and graceful shutdown
    while (streamer.isAlive())
    {
        cv::Mat frame;
        cap >> frame;
        if (frame.empty())
        {
            std::cerr << "frame not grabbed\n";
            //continue;
            exit(EXIT_FAILURE);
        }

        // http://localhost:8080/bgr
        std::vector<uchar> buff_bgr;
        cv::imencode(".jpg", frame, buff_bgr, params);
        streamer.publish("/bgr", std::string(buff_bgr.begin(), buff_bgr.end()));

        cv::Mat hsv;
        cv::cvtColor(frame, hsv, cv::COLOR_BGR2HSV);

        // http://localhost:8080/hsv
        std::vector<uchar> buff_hsv;
        cv::imencode(".jpg", hsv, buff_hsv, params);
        streamer.publish("/hsv", std::string(buff_hsv.begin(), buff_hsv.end()));

        // std::cout<< "published" << std::endl;
    }

    streamer.stop();
}

Write the CMakeLists.txt

cmake_minimum_required(VERSION 3.1)

project(mjpeg_streamer CXX)

find_package(OpenCV 4.2 REQUIRED)
find_package(nadjieb_mjpeg_streamer REQUIRED)

include_directories(${OpenCV_INCLUDE_DIRS})

add_executable(stream_test
  "mjpeg_server.cc")
target_compile_features(stream_test PRIVATE cxx_std_11)
target_link_libraries(stream_test PRIVATE nadjieb_mjpeg_streamer::nadjieb_mjpeg_streamer
                     ${OpenCV_LIBS})


| --- mjpeg_server.cc
| --- CMakeLists.txt
| --- ...
| --- build  
      | --- demo.mp4
      | --- ...

Now, we can build the streamer.

mkdir build && cd build;
cmake ../;
make;
./stream_test

Now, if you go to "http://ip_address:port/bgr" or, "http://ip_address:port/hsv" you should be able to see the stream. In my case, ip = 192.168.1.7 / localhost, port = 8000.

If you want to grab the stream with another server,

index.html

<html>
  <body>
    <img src="http://localhost:8000/bgr">
    <img src="http://localhost:8000/hsv">
  </body>
</html>

serve.py

import http.server
import socketserver

class MyHttpRequestHandler(http.server.SimpleHTTPRequestHandler):
    def do_GET(self):
        if self.path == '/':
            self.path = 'index.html'
        return http.server.SimpleHTTPRequestHandler.do_GET(self)

# Create an object of the above class
handler_object = MyHttpRequestHandler

PORT = 8080
my_server = socketserver.TCPServer(("", PORT), handler_object)

# Star the server
my_server.serve_forever()

python3 serve.py

Finally, even though it's extremely simple, it's not secure.

very high CPU & resource usage. but good alternative. – manna Jun 24 '21 at 11:44 — manna, Jun 24 '21 at 11:44

c++ opencv webcam stream to html

2 Answers2

Get you stream from your webcam

Provide your image via HTTP

Connect to the server by using HTML

Linked