Audio is missing after adding filter effect to captured video in Swift3 iOS

Question

I am developing video based application, where I need to add CIFilter to the captured video selected from device gallery. For this I am using below VideoEffects library:

https://github.com/FlexMonkey/VideoEffects

Using this, I can able to add filters to my video, but the problem is Audio is missing in the final video output. I tried below code to add Audio assets but not working:

videoOutputURL = documentDirectory.appendingPathComponent("Output_\(timeDateFormatter.string(from: Date())).mp4")

    do {
      videoWriter = try AVAssetWriter(outputURL: videoOutputURL!, fileType: AVFileTypeMPEG4)
    }
    catch {
      fatalError("** unable to create asset writer **")
    }

    let outputSettings: [String : AnyObject] = [
      AVVideoCodecKey: AVVideoCodecH264 as AnyObject,
      AVVideoWidthKey: currentItem.presentationSize.width as AnyObject,
      AVVideoHeightKey: currentItem.presentationSize.height as AnyObject]

    guard videoWriter!.canApply(outputSettings: outputSettings, forMediaType: AVMediaTypeVideo) else {
      fatalError("** unable to apply video settings ** ")
    }


    videoWriterInput = AVAssetWriterInput(
      mediaType: AVMediaTypeVideo,
      outputSettings: outputSettings)


    //setup audio writer
    let audioOutputSettings: Dictionary<String, AnyObject> = [
        AVFormatIDKey : Int(kAudioFormatMPEG4AAC) as AnyObject,
        AVSampleRateKey:48000.0 as AnyObject,
        AVNumberOfChannelsKey:NSNumber(value: 1),
        AVEncoderBitRateKey : 128000 as AnyObject
    ]

    guard videoWriter!.canApply(outputSettings: audioOutputSettings, forMediaType: AVMediaTypeAudio) else {
        fatalError("** unable to apply Audio settings ** ")
    }

    audioWriterInput = AVAssetWriterInput(
        mediaType: AVMediaTypeAudio,
        outputSettings: audioOutputSettings)


    if videoWriter!.canAdd(videoWriterInput!) {
      videoWriter!.add(videoWriterInput!)
      videoWriter!.add(audioWriterInput!)
    }
    else {
      fatalError ("** unable to add input **")
    }

Is there any other way to add filter to video? Please suggest me.

Also I tried to use GPUImage to add CIFilter, but this is working only for Live video, not captured video.

Do you actually append any _aac_ buffers to `audioWriterInput`? You're not showing enough code to tell. — Rhythmic Fistman, Apr 05 '18 at 13:21
I can see that, but do you append buffers to that? Can you show the code for that? — Rhythmic Fistman, Apr 05 '18 at 14:21

Jabson · Accepted Answer · 2018-04-05T07:27:36.630

1

Since iOS 9.0 you can use AVVideoComposition to apply core image filter to video frame by frame.

let filter = CIFilter(name: "CIGaussianBlur")!
let composition = AVVideoComposition(asset: asset, applyingCIFiltersWithHandler: { request in
        // Clamp to avoid blurring transparent pixels at the image edges
        let source = request.sourceImage.imageByClampingToExtent()
        filter.setValue(source, forKey: kCIInputImageKey)

        // Vary filter parameters based on video timing
        let seconds = CMTimeGetSeconds(request.compositionTime)
        filter.setValue(seconds * 10.0, forKey: kCIInputRadiusKey)

        // Crop the blurred output to the bounds of the original image
        let output = filter.outputImage!.imageByCroppingToRect(request.sourceImage.extent)

        request.finish(with: output, context: nil)
})

now here we can create AVPlayerItem using the asset created earlier and play it using AVPlayer

let playerItem = AVPlayerItem(asset: asset)
playerItem.videoComposition = composition
let player = AVPlayer(playerItem: playerItem)
player.play()

core image filter added realtime frame by frame. You can also export video using AVAssetExportSession class.

here is WWDC 2015 great introduction: Link

edited Apr 05 '18 at 07:27

answered Apr 04 '18 at 12:56

Jabson

1,585
1
12
16

Hi @Jaba, I tried above code. But It's not coming inside 'let composition = AVVideoComposition(asset: asset, applyingCIFiltersWithHandler: { request in ' loop. – Anand Gautam Apr 05 '18 at 06:59
Did you try to play item? it's coming when you are playing item or exporting. – Jabson Apr 05 '18 at 07:24
try to use updated code i had some extra unnecessary declaration in closure – Jabson Apr 05 '18 at 07:28
@JabaIt's not going inside the request loop, I am using updated code. – Anand Gautam Apr 05 '18 at 10:03
I checked your MagicVideo App, it's working fine when you want to add video filter to the live recording video. Is there anyway that we can add filter to pre captured video along with Audio? – Anand Gautam Apr 05 '18 at 10:39
ok, share your code. t will try to help you. if you want to add CIFilter already recorded video or imported video this way is best i mentioned above. if your target is ios 9.0. – Jabson Apr 05 '18 at 11:33
Is it possible to customize MagaicVideo App for pre recorded video @jaba? – Anand Gautam Apr 05 '18 at 11:47
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/168301/discussion-between-anand-gautam-and-jaba-odishelashvili). – Anand Gautam Apr 05 '18 at 11:47
yeap, it's possible, you can create From UILabel->UIImage->CIImage and this cover image you can add over source image using CISourceOverCompositing filter... every x time you can change your cover image position using transform translation... – Jabson Apr 26 '18 at 09:56
but it's another question we can't continue another question/answer here. – Jabson Apr 26 '18 at 09:57
I have no time now. when i will i check your new question. – Jabson Apr 26 '18 at 10:48
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/169852/discussion-between-anand-gautam-and-jabson). – Anand Gautam Apr 26 '18 at 11:37
please look into this: https://stackoverflow.com/questions/51017511/how-to-add-watermark-on-final-video-after-merging-video-and-audio-asset-into-one – user2786 Jun 25 '18 at 06:52

Audio is missing after adding filter effect to captured video in Swift3 iOS

1 Answers1