11

Is there any API which can give me the list of time stamps of the key frames for the given video file in Android?

Ilya Shinkarenko
  • 2,134
  • 18
  • 35

5 Answers5

10

For an elegant solution, it seems as if you'd need to use ffmpeg for android to achieve this. Someone else tried it that way: How to get the frame from video file in android ; and was suggested to use ffmpeg.

I checked the reference manual, but could only find a way to get a keyframe itself (without timestamp) relative to a given position in time using the MediaMetadataRetriever() as explained here: http://developer.android.com/reference/android/media/MediaMetadataRetriever.html#extractMetadata%28int%29

If you don't want to use ffmpeg you could try the following:

private ArrayList<int> getKeyFrameTimestamps(String filename) {
  ArrayList<int> timeStamps = new ArrayList<int>();

  try {
    MediaMetadataRetriever mRetriever = new MediaMetadataRetriever();
    mRetriever.setDataSource(getDataSource(filename));
    mRetriever.getFrameAtTime(i, MediaMetadataRetriever.OPTION_CLOSEST_SYNC);

    Bitmap lastFame = mRetriever.getFrameAtTime(0, MediaMetadataRetriever.OPTION_NEXT_SYNC);
    result.add(0);

    for(int time=1; frame ; ++time) {
      Bitmap frame = mRetriever.getFrameAtTime(time, MediaMetadataRetriever.OPTION_PREVIOUS_SYNC);
      if (frame && !frame.sameAs(lastFrame)) {
        result.add(time);
        lastFrame = frame;
      }
    }
  } catch (Exception e) {  /* TODO: HANDLE THIS */ }

  return timeStamps;
}

This is of course not very elegant, but would probably work without hassle to install ffmpeg... It's probably also quite slow (I don't know). But maybe it's just what you need.

PS: An ffmpeg tutorial that fits your problem can be found here: http://dranger.com/ffmpeg/tutorial07.html

Community
  • 1
  • 1
SDwarfs
  • 3,189
  • 5
  • 31
  • 53
  • 1
    Hi Stefan, thanks for your efforts, but I have already tried this solution and it works damn slow and delivers strange results. I will investigate further on (e.g. ffmpeg / ffprobe) and keep this thread updated. – Ilya Shinkarenko Aug 17 '12 at 08:51
  • Thats what I've expected. An idea to speed this up a bit is trying `time += lastdistance;` Where you use set `lastdistance = lastFrameTime - time;` if a keyframe is found, check if `lastFrameTime + lastdistance - 1` is still the last frame. If so, go on... else use the "slow search" to find the next frame – SDwarfs Aug 17 '12 at 09:03
  • 1
    cross compile ffmpeg onto android. use ffprobe command with -show_frames option to get the list of I pictures. This should work for all types of videos and is quite fast. Check it out on the PC first. [If you have installed ffmpeg, you should have ffprobe as well]. You will get all the information you need. There is -show_packets which is even faster and can prove sufficient if all you want to know is reference vs non reference frames. – av501 Aug 20 '12 at 10:39
5

I believe the correct answer is http://code.google.com/p/mp4parser/

The isoparser API can read and write the MP4 file structure. It is a low level tool dealing with the so called boxes but it is as well as dealing with structure like tracks and movies.

Nice API and ways much faster than ffmpeg behemoth

Ilya Shinkarenko
  • 2,134
  • 18
  • 35
2

4.1+ - faster than mp4parser because done in native underneath and the code is cleaner

mp4parser you have to calculate the timestamps from the getsyncsamples + getsampledurations which is also annoying and they don't match up with ffprobe. this gets accurate results

MediaExtractor extractor = new MediaExtractor();
extractor.setDataSource(getVideoPath());

int trackindex = MediaExtractorUtil.selectVideoTrack(extractor);
extractor.selectTrack(trackindex);

while (extractor.getSampleTime() != -1) {
    long sampleTime = extractor.getSampleTime();

    // check not really necessary but JIC
    if ((extractor.getSampleFlags() & MediaExtractor.SAMPLE_FLAG_SYNC) > 0) {
        mKeyframeTimestampsMS.add(sampleTime / 1000);
    }

    extractor.seekTo(sampleTime + 1, MediaExtractor.SEEK_TO_NEXT_SYNC);
}
ryan gordon
  • 140
  • 10
0

Building on @ryan gordon's solution. seekTo may seek to the same frame, even when SEEK_TO_NEXT_SYNC is specified. To avoid this, you can terminate once frames start repeating.

For example:

    private fun getSyncFrameTimestamps(source: MediaExtractor): List<Long> {
        val result = mutableListOf<Long>()

        var lastSampleTime = -1L
        var sampleTime = source.sampleTime

        while (sampleTime >= 0L && sampleTime != lastSampleTime) {
            if (source.sampleFlags and MediaExtractor.SAMPLE_FLAG_SYNC != 0) {
                result.add(sampleTime)
            }
            source.seekTo(sampleTime + 1L, MediaExtractor.SEEK_TO_NEXT_SYNC)
            lastSampleTime = sampleTime
            sampleTime = source.sampleTime
        }

        return result
    }

Note that before calling this, source should have the data source set, and video track selected.

Vasiliy Kulakov
  • 5,401
  • 1
  • 20
  • 15
-1

Using MediaExtractor, you can have all key(sync)frames times and/or any specific keyframe at any desired time . +some more useful data.

Then by MediaMetadataRetriever -> getFrameAtTime you can get frame Bitmaps.

here is a sample code to use MediaExtractor:

    mediaExtractor = new MediaExtractor() ;
    mediaExtractor.setDataSource(mediaPath);
    int numTracks = mediaExtractor.getTrackCount();
    for (int i = 0; i < numTracks; ++i) {
        MediaFormat format = mediaExtractor.getTrackFormat(i);
        String mime = format.getString(MediaFormat.KEY_MIME);
        if ( mime.startsWith("video") ) {
            mediaExtractor.selectTrack(i);
        }
        Log.i( "LOG_Extractor" , "track "+i+" "+mime );
    }

    currentTime = mediaPlayer.getCurrentPosition();
    mediaExtractor.seekTo( (long)currentTime*1000 , MediaExtractor.SEEK_TO_PREVIOUS_SYNC );
    previousKeyFrameTimeMilis = mediaExtractor.getSampleTime() /1000 ;
David AfsharPoor
  • 1
  • 1