Reading frames from video file using openCV and Native code C++

Question

I'm trying to write an app that gets all the frames of a video and manipulating them, I found oout that the best way to extract frames on Android is using OpenCv lib.

I saw in the sample code that uses VideoCapture object that receives the video path and can grab frames out of it, so I wrote the following code but the capture.open() doen't really open the video file, the capture.isOpen() is always false.

Source code:

#include <jni.h>
//opencv

#include <opencv2/highgui/highgui.hpp>
#include <opencv2/video/background_segm.hpp>
#include <opencv2/core/mat.hpp>
//C
#include <stdio.h>
#include <string.h>
//C++
#include <iostream>
#include <sstream>
//Android
#include <android/log.h>

//#define  LOGI(TAG,INFO)  __android_log_print(ANDROID_LOG_INFO,INFO,__VA_ARGS__)

using namespace cv;

extern "C" {

JNIEXPORT void JNICALL Java_com_example_nativeopencvcheck_MainActivity_processVideo(JNIEnv* env,
    jobject thsObj,jstring fileName) {

const char * fileNameNative;
jboolean isCopy;
fileNameNative = env->GetStringUTFChars(fileName, &isCopy);

//create the capture object
__android_log_print(ANDROID_LOG_ERROR, "From_Native",
            "trying to open file: %s", fileNameNative);
VideoCapture capture(fileNameNative);
capture.open(fileNameNative);

if (!capture.isOpened()) {  //!!!!!! ALWAYS CLOSED !!!!!
    __android_log_write(ANDROID_LOG_ERROR, "From_Native",
            "capture isn't open. closing..");
    exit( EXIT_FAILURE);
}

Mat iplimage;
capture.retrieve(iplimage,0); 
if (iplimage.size > 0) {
    jclass cls = env->FindClass( "com/example/opencvframesext/MainActivity");
    if (cls == 0) {
        return;
    }

    jmethodID javamethod = env->GetMethodID(cls, "getCurrentFrameFromNative", "()V");
    if (javamethod == 0) {
    //  LOGI("From_Native","GetMethodID error");
        return;
    }

    jobject obj; // TODO
    env->CallVoidMethod(obj, javamethod);

   return;
}


/*bool gotFrame = capture.read(mat);
 while (gotFrame) {
 mats.addref(mat);
 capture.read(mat);
 }*/

//delete capture object
capture.release();
}
}

Another approach: http://bigflake.com/mediacodec/#ExtractMpegFramesTest — fadden, Nov 08 '13 at 00:13
Hi man, thank you for the reply. I tested it and it still takes 200 millis for constructing a frame. Too slow for me and I'll continue the search for finding a faster extraction method. Cheers — Nativ, Nov 08 '13 at 17:41
I tweaked the test to extract full frames and show some timing data, and fed it a 720p screenrecord video. On Nexus 5 I get roughly 17ms (decode) + 177ms (glReadPixels) + 110ms (byte reorder) + 468ms (PNG save). Presumably you don't need the reorder+png, but you'll need to avoid glReadPixels, which is probably doing a YUV->RGB conversion. In theory, the API 19 `android.media.ImageReader` class would let you access the YUV data on the Surface, but in Android 4.4 it only works for Camera output (not MediaCodec decoder). — fadden, Nov 08 '13 at 23:03
Hi @fadden, can you run native C++ code, getting camera frame on Nexus 5? I cannot access the camera, same as a issue report in http://code.opencv.org/issues/3490. Please, I need your help. Just bought the device, love the price and quality, but my final project code cannot run there :(( — bonchenko, Feb 05 '14 at 10:49
FWIW, I got `glReadPixels()` down to a few milliseconds on Nexus 5, but it's still painfully slow on Nexus 10 (different GPU). This post *almost* got native buffer access working, but something isn't quite right: http://stackoverflow.com/questions/21151259/replacing-glreadpixels-with-egl-khr-image-base-for-faster-pixel-copy/ — fadden, Feb 05 '14 at 16:00
@bonchenko: if you have a new question, you should post it separately. — fadden, Feb 05 '14 at 16:01

score 4 · Accepted Answer · answered Nov 07 '13 at 16:45

4

yea, that wont work, unfortunately.

there's no ffmpeg backend for VideoCapture on android.

answered Nov 07 '13 at 16:45

berak

39,159
9
91
89

Unbeleivable, so there's no way to extract frames from video with openCv? – Nativ Nov 07 '13 at 17:04
unless, you come up with your own ffmpeg port, - no. the javacv ppl kinda solved that, but that# a total different story – berak Nov 07 '13 at 17:29
yes but javacv's license is GPL, and if your adding it to your project you'll have to open source it(or pay for every download from the play store) – Nativ Nov 07 '13 at 17:48
yep. also using deprecated c-api under the hood for most things, and *hand-rolled* – berak Nov 07 '13 at 17:50
1

Is that still the case in 2020? (Using c++) – Haim Bendanan Oct 08 '20 at 21:30

Reading frames from video file using openCV and Native code C++

1 Answers1

Linked