How to read h264 stream as a file from the USB webcam directly in c/c++ without using opencv?

Question

I am able to read a video file of h264 format and doing some machine learning inference on top of it. The code works absolutely fine for input from a file. Below code is a sample code from Deepstream SDK

FileDataProvider(const char *szFilePath, simplelogger::Logger *logger)
    : logger_(logger)
    {
         fp_ = fopen(szFilePath, "rb");

        //fp_ = fopen("/dev/video0", "rb");


        if (nullptr == fp_) {
            LOG_ERROR(logger, "Failed to open file " << szFilePath);
            exit(1);
        }
        pLoadBuf_ = new uint8_t[nLoadBuf_];
        pPktBuf_ = new uint8_t[nPktBuf_];
        assert(nullptr != pLoadBuf_);
    }
    ~FileDataProvider() {
        if (fp_) {
            fclose(fp_);
        }
        if (pLoadBuf_) {
            delete [] pLoadBuf_;
        }
        if (pPktBuf_) {
            delete [] pPktBuf_;
        }
    }

What is requirement ?

Read from the Logitech c920 webcam instead for video file.
I know How to read from webcam using opencv. But I don't want to use opencv here.

My Research

Using v4l we can get the stream and display it in vlc.
Camera supports below formats.

@ubox:~$ v4l2-ctl --device=/dev/video1 --list-formats

ioctl: VIDIOC_ENUM_FMT Index : 0 Type : Video Capture Pixel Format: 'YUYV' Name : YUYV 4:2:2

Index : 1 Type : Video Capture Pixel Format: 'H264' (compressed) Name : H.264

Index : 2 Type : Video Capture Pixel Format: 'MJPG' (compressed) Name : Motion-JPEG

Reading output of a USB webcam in Linux
vlc v4l2:///dev/video1 --v4l2-chroma=h264 - this displays the video from the webcam.

How to do this? - Now how to feed this live stream into above sample code such that it reads from the webcam rather than file?

[update-1] - In otherwords, does v4l has some options to write the video stream as h264 formant ? So that, I can read that file like before(above code) when its(v4l) writing to disk.

[update-2] - we can use ffmpeg instead of v4l. If any solutions for using ffmpeg to save the video stream into disk continuously, so that other programs reads that file ?

Are you having trouble understanding the `v4l` documentation? — Galik, Mar 06 '18 at 16:36
Possible duplicate of https://stackoverflow.com/questions/4009439/v4l2-very-simple-example — Galik, Mar 06 '18 at 16:42
all those are giving complex solutions. I am looking for something simple than that. For example, say if v4l has some command which will write the file to disk as h264 format. Now I can read that file in the code without changing much of the code right ? — ajayramesh, Mar 06 '18 at 16:52
`v4l` is a low level interface. If you want high level functions then you need to use a library that does all the nitty gritty stuff for you. — Galik, Mar 06 '18 at 16:53
thank you @Galik - let me research on how to use ffmpeg to do the same instead of v4l. — ajayramesh, Mar 06 '18 at 17:03
this answer might help the answer : 1. https://stackoverflow.com/a/37024251/6180077 — Abdullah Farweez, Apr 16 '18 at 06:31

Richard Tai · Answer 1 · 2022-11-23T12:30:00.687

Before using ioctl to capture frames from camera, you need to set the format like below first.

fp_ = open("/dev/video0", O_RDWR);
struct v4l2_format fmt = {0};
fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_H264;
ioctl(fp_, VIDIOC_S_FMT, &fmt);

then, initialize and map buffer

struct Buffer
{
    void *start;
    unsigned int length;
    unsigned int flags;
};

int buffer_count_ = 4;
Buffer *buffers_;

bool AllocateBuffer()
{
    struct v4l2_requestbuffers req = {0};
    req.count = buffer_count_;
    req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    req.memory = V4L2_MEMORY_MMAP;

    if (ioctl(fp_, VIDIOC_REQBUFS, &req) < 0)
    {
        perror("ioctl Requesting Buffer");
        return false;
    }

    buffers_ = new Buffer[buffer_count_];

    for (int i = 0; i < buffer_count_; i++)
    {
        struct v4l2_buffer buf = {0};
        buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
        buf.memory = V4L2_MEMORY_MMAP;
        buf.index = i;
        if (ioctl(fp_, VIDIOC_QUERYBUF, &buf) < 0)
        {
            perror("ioctl Querying Buffer");
            return false;
        }

        buffers_[i].start = mmap(NULL, buf.length, PROT_READ | PROT_WRITE, MAP_SHARED, fd_, buf.m.offset);
        buffers_[i].length = buf.length;

        if (MAP_FAILED == buffers_[i].start)
        {
            printf("MAP FAILED: %d\n", i);
            for (int j = 0; j < i; j++)
                munmap(buffers_[j].start, buffers_[j].length);
            return false;
        }

        if (ioctl(fp_, VIDIOC_QBUF, &buf) < 0)
        {
            perror("ioctl Queue Buffer");
            return false;
        }
    }

    return true;
}

STREAMON to start capturing

v4l2_buf_type type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
ioctl(fp_, VIDIOC_STREAMON, &type);

finally read a frame from the mapped buffer. Generally, CaptureImage() will be in the while loop.

Buffer CaptureImage()
{
    fd_set fds;
    FD_ZERO(&fds);
    FD_SET(fd_, &fds);
    struct timeval tv = {0};
    tv.tv_sec = 1;
    tv.tv_usec = 0;
    int r = select(fd_ + 1, &fds, NULL, NULL, &tv);
    if (r == 0)
    {
        // timeout
    }

    struct v4l2_buffer buf = {0};
    buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    buf.memory = V4L2_MEMORY_MMAP;

    while (ioctl(fp_, VIDIOC_DQBUF, &buf) < 0)
    {
        perror("Retrieving Frame");
    }

    struct Buffer buffer = {.start = buffers_[buf.index].start,
                            .length = buf.bytesused,
                            .flags = buf.flags};

    if (ioctl(fp_, VIDIOC_QBUF, &buf) < 0)
    {
        perror("Queue buffer");
    }

    return buffer;
}

How to read h264 stream as a file from the USB webcam directly in c/c++ without using opencv?

1 Answers1