OpenCV and GoPro - empty frames in VideoCapture stream

Question

I have a GoPro Hero 3+ (Black) which is connected to a video capture card (AverMedia Game Broadcaster HD). I simply want to get the video stream in OpenCV. With a Logitech Webcam there are no problems. The used code is below.

VideoCapture cap;
cap.open(0);

waitKey(300);

//cap.set(CV_CAP_PROP_FRAME_WIDTH, 1280);
//cap.set(CV_CAP_PROP_FRAME_HEIGHT, 720);

if (cap.isOpened()){
    cout << "Cam identified" << endl;
}

namedWindow("dst", 1);

while (1){

Mat frame;

if (!cap.read(frame)) {
    std::cout << "Unable to read frame from video stream" << std::endl;
    continue;
}

imshow("dst", frame);

[...]

}

With the GoPro the following happens: OpenCV is able to open the VideoCapture ("Cam identified") but can't read any frames (just a gray screen and the output: "Unable to read frame from video stream"). I also checked this with frame.empty();.

I know that the video capture card works correct because Unity opens a WebCamTexture with the GoPro stream without any issues. I read about codec problems in OpenCv and so I already tried to compile OpenCV with FFMPEG support. Now the recorded MP4-Videos of the GoPro can be displayed but the stream still doesn't work.

I use OpenCV 2.48, Windows 7 and Visual Studio 2013.

EDIT: Here is the code of libVLC solution:

struct ctx
{
uint8_t* pixeldata;
std::mutex imagemutex;
};

static void display(void *data, void *id);
static void unlock(void *data, void *id, void *const *p_pixels);
static void *lock(void *data, void **p_pixels);

struct ctx ctx;

libvlc_instance_t *inst;
libvlc_media_player_t *mp;
libvlc_media_t *m;

int main(int argc, char* argv[])
{
    ctx.pixeldata = new uint8_t[1280 * 720 * 3];

    char const *vlc_argv[] =
    {
        "-vvv",
        "--no-audio", /* skip any audio track */
        "--no-xlib", /* tell VLC to not use Xlib */
    };

    int vlc_argc = sizeof(vlc_argv) / sizeof(*vlc_argv);
    inst = libvlc_new(vlc_argc, vlc_argv);

    const char *options[] =
    {
        ":dshow-vdev=AVerMedia HD Capture",
        ":dshow-adev=none",
        //":dshow-size=1280x720",
        ":dshow-fps=24",
        ":dshow-chroma=YUY2",
        ":dshow-video-input=1",
        ":dshow-video-output=1",
        ":dshow-aspect-ratio=16\:9",
        ":live-caching=80",
        NULL
    };

    m = libvlc_media_new_location(inst, "dshow://");
    for (const char **opt = options; *opt; opt++)
        libvlc_media_add_option(m, *opt);

    mp = libvlc_media_player_new_from_media(m);
    libvlc_media_release(m);
    libvlc_video_set_callbacks(mp, lock, unlock, display, &ctx);
    libvlc_video_set_format(mp, "RV24", 1280, 720, 1280 * 3);
    libvlc_media_player_play(mp);

    namedWindow("all", 1);

    Mat frame(720, 1280, CV_8UC3);

    while (1){

        ctx.imagemutex.lock();
        memcpy(gesamt.data, ctx.pixeldata, 1280 * 720 * sizeof(uint8_t) * 3);
        ctx.imagemutex.unlock();

        imshow("all", gesamt);

        if (waitKey(30) == 27) //wait for 'esc' key press for 30ms. If 'esc' key is pressed, break loop
        {
            cout << "esc key is pressed by user" << endl;
            break;
        }

    }

    libvlc_media_player_stop(mp);
    libvlc_media_player_release(mp);
    libvlc_release(inst);
    delete[] ctx.pixeldata;

    return 0;
}

void display(void *data, void *id){
    (void)data;
    assert(id == NULL);
}

void unlock(void *data, void *id, void *const *p_pixels){
    struct ctx *ctx = (struct ctx*)data;
    ctx->imagemutex.unlock();
    assert(id == NULL);
}

void *lock(void *data, void **p_pixels){
    struct ctx *ctx = (struct ctx*)data;
    ctx->imagemutex.lock();
    *p_pixels = ctx->pixeldata;
    return NULL;
}

score 2 · Accepted Answer · answered May 05 '14 at 22:28

2

I've had the same issues trying to stream a Sony action cam through an Avermedia capture card. One quick fix seems to be using DVDrive, which makes your capture card output look like a webcam. I've had success using this as a workaround.

answered May 05 '14 at 22:28

user9294

36
2

Thanks! You are right. With DVDriver it works. Another workaround I found is using the directshow capabilities of libVLC. Would still be nice to know a solution that only uses openCV framework. – Novatar May 08 '14 at 13:44
libVLC is a good tip. Anything complicated about getting it to work? I'll probably give it a go as DVDrive takes up a decent chunk of cpu cycles at high-res. – user9294 May 09 '14 at 17:12
Took some time for me to figure it out. You can see my code in the post above. Biggest problems were the video format settings and the correct conversion to opencv::Mat. In my first solution I used "libvlc_video_set_format(mp, "YUY2", 1280, 720, 1280 * 2);" which was kind of hard to convert because it's a packed. So the first und third byte were the Y-Values of first and second pixel. The second and fourth byte were Cb and Cr Values for both pixels. Later I find out that the conversion could be done directly by choosing RV24 instead of YUY2 in the set_format command. – Novatar May 19 '14 at 13:25

OpenCV and GoPro - empty frames in VideoCapture stream

1 Answers1

Linked