Getting individual frames using CV_CAP_PROP_POS_FRAMES in cvSetCaptureProperty

Question

I am trying to jump to a specific frame by setting the CV_CAP_PROP_POS_FRAMES property and then reading the frame like this:

cvSetCaptureProperty( input_video, CV_CAP_PROP_POS_FRAMES, current_frame );
frame = cvQueryFrame( input_video );

The problem I am facing is that, OpenCV 2.1 returns the same frame for the 12 consecutive values of current_frame whereas I want to read each individual frame, not just the key frames. Can anyone please tell me what's wrong?

---

I did some research and found out that the problem is caused by the decompression algorithm.

The MPEG-like algorithms (including HD, et all) do not compress each frame separately, but save a keyframe from time to time, and then only the differences between the last frame and subsequent frames.

The problem you reported is caused by the fact that, when you select a frame, the decoder (ffmpeg, likely) automatically advances to the next keyframe.

So, is there a way around this? I don't want only key frames but each individual frame.

Answer 1

I don't know whether or not this would be precise enough for your purpose, but I've had success getting to a particular point in an MPEG video by grabbing the frame rate, converting the frame number to a time, then advancing to the time. Like so:

cv::VideoCapture sourceVideo("/some/file/name.mpg");
double frameRate = sourceVideo.get(CV_CAP_PROP_FPS);
double frameTime = 1000.0 * frameNumber / frameRate;
sourceVideo.set(CV_CAP_PROP_POS_MSEC, frameTime);

Answer 2

Due to this limitation in OpenCV, it may be wise to to use FFMPEG instead. Moviepy is a nice wrapper library.

# Get nth frame from a video
from moviepy.video.io.ffmpeg_reader import FFMPEG_VideoReader
cap = FFMPEG_VideoReader("movie.mov",True)
cap.initialize()
cap.get_frame(n/FPS)

Performance is great too. Seeking to the nth frame with get_frame is O(1), and a speed-up is used if (nearly) consecutive frames are requested. I've gotten better-than-realtime results loading three 720p videos simultaneously.

Answer 3

CV_CAP_PROP_POS_FRAMES jumps to a key frame. I had the same issue and worked around it using this (python-)code. It's probably not totally efficient, but get's the job done:

def seekTo(cap, position):
  positiontoset = position
  pos = -1
  cap.set(cv.CV_CAP_PROP_POS_FRAMES, position)
  while pos < position:
    ret, image = cap.read()
    pos = cap.get(cv.CV_CAP_PROP_POS_FRAMES)
    if pos == position:
      return image
    elif pos > position:
      positiontoset -= 1
      cap.set(cv.CV_CAP_PROP_POS_FRAMES, positiontoset)
      pos = -1

Answer 4

I've successfully used the following on OpenCV 3 / Python 3:

 # Skip to 150 frame then read the 151th frame
 cap.set(cv2.CAP_PROP_POS_FRAMES, 150))
 ret, frame = cap.read()

Answer 5

After some years assuming this as a unsavable bug, I think I've figured out a way to use with a good balance between speed and correctness.

A previous solution suggested to use the CV_CAP_PROP_POS_MSEC property before reading the frame:

cv::VideoCapture sourceVideo("/some/file/name.mpg");
const auto frameRate = sourceVideo.get(CV_CAP_PROP_FPS);

void readFrame(int frameNumber, cv::Mat& image) {
  const double frameTime = 1000.0 * frameNumber / frameRate;
  sourceVideo.set(CV_CAP_PROP_POS_MSEC, frameTime);
  sourceVideo.read(image);
}

It does return the expected frame, but the problem is that using CV_CAP_PROP_POS_MSEC may be very slow, for example for a video conversion.

Note: using global variables for simplicity.

---

On the other hand, if you just want to read the video sequentially, it is enough to read frame without seeking at all.

for (int frameNumber = 0; frameNumber < nFrames; ++frameNumber) {
  sourceVideo.read(image);
}

---

The solution comes from combining both: using a variable to remember the last queried frame, lastFrameNumber, and only seek when requested frame is not the next one. In this way it is possible to increase the speed in a sequential reading while allowing random seek if necessary.

cv::VideoCapture sourceVideo("/some/file/name.mpg");
const auto frameRate = sourceVideo.get(CV_CAP_PROP_FPS);
const int lastFrameNumber = -2; // guarantee seeking the first time

void readFrame(int frameNumber, cv::Mat& image) {
  if (lastFrameNumber + 1 != frameNumber) { // not the next frame? seek
    const double frameTime = 1000.0 * frameNumber / frameRate;
    sourceVideo.set(CV_CAP_PROP_POS_MSEC, frameTime);
  }

  sourceVideo.read(image);
  lastFrameNumber = frameNumber;
}