For the past two weeks I am trying to find a proper way to read frames from .mts video file and process them in OpenCV. When .mts file is in 25p (25 fps progressive) format VideoCapture of OpenCV works fine for seeking video frames but when it is in 50i (25 fps interlaced) format VideoCapture of OpenCV can not properly decode it frame by frame. (e.g. in a sample scenario when I get frame #1 and then read frame #300 and later read frame #1, it returns a corrupted image different from my previous read of frame #1) (i am using OpenCV 2.4.6)
I decided to replace video decoder part of the program.
I tried FFmpegSource2 but the problem of proper frame seeking for .mts was not resolved (most of the time FFMS_GetFrame function returns same output for several consecutive frames for 50i .mts file).
I also tried DirectShow. But IsFormatSupported method of IMediaSeeking for TIME_FORMAT_FRAME does not return S_OK for 50i .mts video file and it only supports TIME_FORMAT_MEDIA_TIME for this kind of video file. I have not tried myself but a friend said even using TIME_FORMAT_MEDIA_TIME for frame seeking will result in the same problem as above and I may not be able to jump back and forward to individual frames and read their data.
Now I am going to try gstreamer. I found sample method for linking gstreamer and openCV in the following link: Adding opencv processing to gstreamer application
When I try to compile it in gstreamer 1.0, I get the following error:
error C3861: 'gst_app_sink_pull_buffer': identifier not found
I have included gst/gst.h, gst/app/gstappsink.h, gst/app/gstappsrc.h
Looked at the following help link and there was not gst_app_sink_pull_buffer function there too. http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-appsink.html
I am using gstreamer 1.0 (v1.2.0) from gstreamer.freedesktop.org
May be gstreamer SDK from www.gstreamer.com (based on gstreamer 0.1) work for that, but I have not tried it yet and prefer to use gstreamer from gstreamer.freedesktop.org
I don't know where gst_app_sink_pull_buffer is defined. Anybody knows how I can compile the sample method provided for gstreamer 0.1 in Adding opencv processing to gstreamer application for gstreamer 1.0?
Thank you in advance.
UPDATE 1: I am new to gstreamer. Now I know that have to port the sample method of Adding opencv processing to gstreamer application from gstreamer 0.1 to gstreamer 1.0. I replaced gst_app_sink_pull_buffer function with gst_app_sink_pull_sample and gst_sample_get_buffer. Have to work more on the other parts of the code and see if can open a desired frame from 50i .mts video file and process it with OpenCV.
UPDATE 2: I found a very good example at http://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/section-data-spoof.html#section-spoof-appsink And I easily replaced the part which saves the snapshot using GTk with functions that load frame data buffer to an OpenCV Mat. This program works fine for many video file types and I can grab frames of the video file in OpenCV Mat. But when the input video file is a 50i .mts video file, it returns the following errors and I can not read the frame data:
No accelerated IMDCT transform found
0:00:00.405110839 4632 0B775380 ERROR libav :0:: get_buffer() failed (-1 2 00000000)
0:00:00.405740899 4632 0B775380 ERROR libav :0:: decode_slice_header error
0:00:00.406401077 4632 0B7756A0 ERROR libav :0:: Missing reference picture
0:00:00.406705867 4632 0B7756A0 ERROR libav :0:: Missing reference picture
0:00:00.416044436 4632 0B7759C0 ERROR libav :0:: Cannot combine reference and non-reference fields in the same frame
0:00:00.416813339 4632 0B7759C0 ERROR libav :0:: decode_slice_header error
0:00:00.417725301 4632 0B775CE0 ERROR libav :0:: Missing reference picture
The step by step debug shows that "No accelerated IMDCT transform found" appears after running
ret = gst_element_get_state( pipeline, NULL, NULL, 5 * GST_SECOND );
and google search shows that I can ignore it as a warning.
All of the other errors emerge just after running
g_signal_emit_by_name( sink, "pull-preroll", &sample, NULL );
I have no idea how to resolve this issue? I have already played this .mts file in another example using playbin and gstreamer can play this .mts video file well when I use playbin.
