A video from GPUImageMovie has a red frame at the beginning

Question

Thank you for seeing this question.

I create a movie from GPUImageMovieComposition and GPUImageWriter, and sometimes (5% ~ 10 %) the movie has red frames at the beginning.

Please teach me why this phenomenon occur.

I use AVFileTypeMPEG4 as a sample filetype but AVFileTypeQuickTimeMovie is also same.

_movieFile = [[GPUImageMovieComposition alloc] initWithComposition:composition andVideoComposition:videoComposition andAudioMix:nil];
_movieFile.playAtActualSpeed = YES;

_movieWriter = [[GPUImageMovieWriter alloc] initWithMovieURL:processedMovieURL
                                                        size:CGSizeMake(1280.0, 720.0)
                                                    fileType:AVFileTypeMPEG4
                                              outputSettings:videoSetting];
_movieWriter.shouldPassthroughAudio = NO;
[_movieWriter setVideoInputReadyCallback:nil];
[_movieWriter setHasAudioTrack:YES audioSettings:audioSetting];
[_movieFile addTarget:_movieWriter];
_movieFile.audioEncodingTarget = _movieWriter;
[_movieFile enableSynchronizedEncodingUsingMovieWriter:_movieWriter];    

[_movieWriter startRecording];
[_movieFile startProcessing];

SOLUTION

Finally I could find the way to solve... but not perfect way...

I modified
- (void)processMovieFrame:(CVPixelBufferRef)movieFrame withSampleTime:(CMTime)currentSampleTime
at GPUImageMovie.m little bit.

When currentSampleTime is set, all red frame has currentSampleTime.value == 0 so I avoided setting currentSampleTime when currentSampleTime.value == 0

Here are some codes which I actually used.

 for (id<GPUImageInput> currentTarget in targets)
  {
       NSInteger indexOfObject = [targets indexOfObject:currentTarget];
       NSInteger targetTextureIndex = [[targetTextureIndices objectAtIndex:indexOfObject] integerValue];
       if(currentSampleTime.value != 0){
           [currentTarget newFrameReadyAtTime:currentSampleTime atIndex:targetTextureIndex];
       }
   }

score 2 · Answer 1 · answered Jul 07 '14 at 05:16

In my case there was not red but just blank frames in the beginning of video recorded by GPUImageMovieWriter.

The problem was that the audio samples appears earlier then the video frames and assetWriter session has been started earlier then first video frames became available.

I've fix that by modifying processAudioBuffer function by replacing this code

if (CMTIME_IS_INVALID(startTime))
{
  runSynchronouslyOnContextQueue(_movieWriterContext, ^{
    if ((audioInputReadyCallback == NULL) && (assetWriter.status != AVAssetWriterStatusWriting))
    {
      [assetWriter startWriting];
    }
    [assetWriter startSessionAtSourceTime:currentSampleTime];
    startTime = currentSampleTime;
  });
}

on this

if (CMTIME_IS_INVALID(startTime))
{
  NSLog(@"0: Had to drop an audio frame: %@", CFBridgingRelease(CMTimeCopyDescription(kCFAllocatorDefault, currentSampleTime)));
  if (_shouldInvalidateAudioSampleWhenDone)
  {
    CMSampleBufferInvalidate(audioBuffer);
  }
  CFRelease(audioBuffer);
  return;
}

The fix is applicable to the latest versions of GPUImage from May 27 to Jul 01, 2014.

score 1 · Answer 2 · answered May 14 '14 at 19:21

The only solution I found so far that saved me from this issue is reverting back to commit e98cc813b. I figured it out by using git bisect and performing batch of processing tests on the same video. This commit already had all required functionality for my project, and only few changes were required to make it more stable. You can take a look at the changes here: https://github.com/crazyjooe/GPUImage.

Besides, after lots of testing, I can say that processing itself became much more stable, especially in terms of cancelling. I wonder how video processing became less reliable and stable after all changes introduced.

Thank you for a comment. Reverted commit was created on Sep 3, 2013... it was too old for me. I need newer commit info... Do you have other way to solve it? — Takahiro Ishihama, Jun 01 '14 at 11:00

A video from GPUImageMovie has a red frame at the beginning

SOLUTION

2 Answers2