6

Seeking to random points in a video file with OpenCV seems to be much slower than in media players like Windows Media Player or VLC. I am trying to seek to different positions on a video file encoded in H264 (or MPEG-4 AVC (part10)) using VideoCapture and the time taken to seek to the position seems to be proportional to the frame number queried. Here's a small code example of what I'm trying to do:

import cv2

cap = cv2.VideoCapture('example_file')   

frame_positions = [200, 400, 8000, 200000]
for frame_position in frame_positions:
   cap.set(cv2.cv.CV_CAP_PROP_FRAMES, frame_position)
   img = cap.read()
   cv2.imshow('window', img)
   cv2.waitKey(0)

The perceived times for when the images are displayed from above are proportional to the frame number. That is, frame number 200 and 400, barely have any delay, 8000 some noticeable lag, but 200000 would take almost half a minute.

Why isn't OpenCV able to seek as "quickly" as say Windows Media Player? Could it be that OpenCV is not using the FFMPEG codecs correctly while seeking? Would building OpenCV from sources with some alternate configuration for codecs help? If so, could someone tell me what the configuration could be?

I have only tested this on Windows 7 and 10 PCs, with OpenCV binaries as is, with relevant FFMPEG DLLs in system path.

Another observation: With OpenCV (binaries) versions greater than 2.4.9 (Example 2.4.11, 3.3.0), the first seek works, but not the subsequent ones. That is, it can seek to frame 200 from above example, but not to 400 and the rest; the video just jumps back to frame 0. But since it works for me with 2.4.9, I'm happy for now.

Aravind Battaje
  • 96
  • 8

3 Answers3

4

GPU acceleration should not matter for seeking, because you are not decoding frames. In addition, even if you were decoding frames, doing so on the GPU would be slower than on the CPU, because your CPU nowadays has video codecs "soldered" into the chip, which makes video decoding very fast, and there would have to be some book-keeping to shovel data from main memory into the GPU.

It sounds like OpenCV implements a "safe" way of seeking: Video files can contain stream offsets. For example, your audio stream may be set off against your video stream. As another example, you might have cut away the beginning of a video and saved the result. If your cut did not happen precisely at a key frame, video editing software like ffmpeg will include a small number of frames before your cut in the output file, in order to allow the frame at which your cut happened to be decoded properly (for which the previous frames might be necessary). In this case, too, there will be a stream offset.

In order to make sure that such offsets are interpreted the right way, that is, to really hit exactly the desired frame relative to "time 0", the only "easy", but expensive way is to really eat and decode all the video frames. And that's apparently what openCV is doing here. Your video players do not bother about this, because everyday users don't notice and the controls in the GUI are anyway much to imprecise.

I might be wrong about this. But answers to other questions and some experiments I conducted to evaluate them showed that only the "slow" way of counting the frames in a video gave accurate results.

Gereon Fox
  • 49
  • 1
  • 2
1

It's likely because that is a very basic code example and the mentioned applications are doing something more clever.

A few points:

If speed for seeking is important, you almost definitly want to work with the GPU when doing video operations:

https://github.com/opencv/opencv/blob/master/samples/gpu/video_reader.cpp

GPPK
  • 6,546
  • 4
  • 32
  • 57
  • That was very helpful, especially to make me realize that even video viewing is pretty GPU dependent. I also dug around a bit to confirm your suspicion that VLC uses graphics drivers for almost all video rendering. That's also when I stumbled upon [libVLC](https://wiki.videolan.org/LibVLC/), which leverages VLC's near-ubiquity with a very nice API. It even has Python bindings. Also, OpenCV GPU functions are still not equipped for "random seeking" yet. I should perhaps make a feature request for that. – Aravind Battaje Feb 19 '18 at 19:57
0

Here are some related github issues:

https://github.com/opencv/opencv/issues/4890

https://github.com/opencv/opencv/issues/9053

Re-encode your video with ffmpeg. It works for me.

mljack
  • 142
  • 2
  • 8