8

In my app I am rendering a Video generated from images I retrieve from the users photos. I have set up an AVAssetwriter with a AVAssetwriterInput has an AVAssetWriterInputPixelBufferAdaptor. I'm able to transform the ALAsset objects I retrieve from the users library to CVPixelBuffers and add them to the Video, which then is saved as an mp4. Adding all the images to the video is done on a background thread which sends a Notification to the main thread every frame, so the interface can be updated. All this works well, and I get a usable Movie file out of the app.

My problem now is, that when the user enters another application, after becoming active again the status of the ALAssetWriter changes to "failed", an I am not able to add any more images to the movie file. First I thought I might have to end the current Session on the writer and reopen a new one, once the app has become active again, but that doesn't seem to help.

I was just wondering how the general approach would be when I'd like the user to enter other applications. The best solution would be, if the rendering could continue in the background. I suppose I'd need a background thread from the UIApplication. But for now I'd be happy, if rendering could just continue after resuming my app.

I won't post any code for now, because it's really a lot, and my question possibly is conceptual. If you need to see code, I'll post it.

Edit 1: Tested on iOS 4.3 and iOS 5. I've seen background rendering on other apps such as iTimelapse, but I'm not sure which frameworks they use.

Edit2: I now have the information of an apple devforum member, that AVAssetWriter does not work in the background. So is there any other framework out there capable of rendering quicktime videos?

huesforalice
  • 2,418
  • 2
  • 24
  • 29
  • Is it possible, that the AVAssetWriter looses the connection to the file it is writing too when the application goes to the background? – huesforalice Oct 03 '11 at 20:14

2 Answers2

5

Turns out that AVAssetWriter just won't survive the app being suspended. You can add an extra 10 Minutes of rendertime by requesting background time, but after that the AssetWriter fails. Same happens if you use certain services on the phone. For example making or answering a call will make the AVAssetWriter fail as well.

huesforalice
  • 2,418
  • 2
  • 24
  • 29
1

If there are any OpenGL calls made when your app is in the background then that would explain this behaviour, looks fairly likely. From the OpenGL ES Pragramming Guide

Background Applications May Not Execute Commands on the Graphics Hardware

An OpenGL ES application is terminated if it attempts to execute OpenGL ES commands on the graphics hardware. This not only refers to calls made to OpenGL ES while your application is in the background, but also refers to previously submitted commands that have not yet completed. The main reason for preventing background applications from processing OpenGL ES commands is to make the graphics processor completely available to the frontmost application. The frontmost application should always present a great experience to the user. Allowing background applications to hog the graphics processor might prevent that. Your application must ensure that all previously submitted commands have been finished prior to moving into the background.

The docs go on to enumerate a set of guidelines for the enter background/foreground app delegate callbacks. I think finding a way to do the rendering without the graphics hardware would be tricky, also the frameworks that allow mp4 encoding (like ffMpeg) are mainly GPL/LGPL, so you need to be careful if dealing with a commercial product (LGPL means you can link to the library dynamically, not statically, which is useless on iOS), as the license would propagate to your code.

jbat100
  • 16,757
  • 4
  • 45
  • 70
  • 2
    Thanks for your thoughts. I thought about that possibility too, but the app actually does not get terminated. It's just that the AVAssetWriters state changes to "failed" when appending stuff to the image buffer after the app was in the background. Even if I stop the renderprocess on "ApplicationWillEnterBackground" and restart it when coming back, the process fails. And I'm not using any OpenGL ES calls, I don't know if any of the AVFoundation or CoreVideo Frameworks do.. – huesforalice Oct 10 '11 at 09:48
  • It might be that the AV stuff has inbuilt protections against making OpenGL calls and checks whether the app has entered background, entering "failed" state if it has instead of killing the app altogether. I don't understand why it would fail again after the app has re-entered foreground though. Do you create new AV objects? (maybe once failed they cannot get out of that state). – jbat100 Oct 10 '11 at 10:08
  • Actually I don't use new AV Objects. And the state of the AssetWriter still is ok when re-entering the foreground. It will only go to failed as soon as I add another pixelbuffer after foregrounding the app. – huesforalice Oct 10 '11 at 10:16
  • Do you get any solution on that? – souvickcse Jul 19 '18 at 11:09