Save Speech Utterance to File - Buffering error - Swift

Question

Last year I asked the question on how to save speech to a file. Stack Overflow Question - Recording speech synthesis to a saved file

Thanks to kakaiikaka for the answer. Although it does work, there was a bit of an error with buffering. The following code isolates the problem. With iOS 16, although there is an error it does work as intended. The completion handler that I have prints as intended. The following error is printed 20 times or so.

2023-06-17 15:35:33.811838-0400 RecordSpeechFix[3899:1958883] [AXTTSCommon] TTSPlaybackEnqueueFullAudioQueueBuffer: error -66686 enqueueing buffer

With iOS 17 (the first beta) there is a more descriptive error, and it does not work. The completion handler does not print. The following error is printed 20 or so times.

Input data proc returned inconsistent 512 packets for 2,048 bytes; at 2 bytes per packet, that is actually 1,024 packets

I'm making the assumption that this is the same problem. Fixing the error for iOS16 will also fix the error for iOS17. I could be wrong with that assumption.

//
//  ContentView.swift
//  RecordSpeechFix
//
//  Created by Dennis Sargent on 6/16/23.
//

import AVFoundation
import SwiftUI

struct ContentView: View {
    var body: some View {
        VStack {
            Image(systemName: "globe")
                .imageScale(.large)
                .foregroundColor(.accentColor)
            Text("Record Speech")
        }
        .padding()
        .onTapGesture {
            saveSpeechUtteranceToFile(phrase: "This produces warnings.") {
                print("In iOS 16, this will print.  In iOS17, this will not.")
            }
        }
    }
    
    func documentsDirectory(fileName: String, ext: String) -> URL {
        let paths = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)
        let documentsDirectoryURL = paths[0]
    
        return documentsDirectoryURL.appendingPathComponent("\(fileName).\(ext)")
    }
    
    let synthesizer = AVSpeechSynthesizer()
    
    func saveSpeechUtteranceToFile(phrase: String, completionHandler: @escaping () -> ()) {
        
        let fileURL = documentsDirectory(fileName: "Test", ext: ".caf")
        
        let utterance = AVSpeechUtterance(string: phrase)
        utterance.voice = AVSpeechSynthesisVoice(language: "en-US")
        utterance.rate = 0.50
        utterance.volume = 1.0
    
        var output: AVAudioFile?
        
        synthesizer.write(utterance) {(buffer: AVAudioBuffer) in
            guard let pcmBuffer = buffer as? AVAudioPCMBuffer else {
                fatalError("unknown buffer type: \(buffer)")
            }
            
            
            if pcmBuffer.frameLength == 0 {
                // Done
                completionHandler()
            } else {
                do{
                    if output == nil {
                        try  output = AVAudioFile(
                            forWriting: fileURL,
                            settings: pcmBuffer.format.settings,
                            commonFormat: .pcmFormatInt16,
                            interleaved: false)
                    }
                    try output?.write(from: pcmBuffer)
                }catch {
                    print("Buffer has an error")
                }
            }
        }
    }
}

I'm not too familiar with general audio recording. It appears that there is some sort of buffering problem. Any ideas what settings are needed to clear up these errors?

Answer 1

It turns out that iOS 17 changed the AVAudioPCMBuffer format from Int16 to Float32. Only Apple knows whether that was intentional or not. That broke every app that used the code snippet from https://stackoverflow.com/a/58118583/5280117

The workaround is twofold:

1. Change the AVAudioFile creation from a fixed .pcmFormatInt16 to one that matches the AVAudioPCMBuffer format.
2. Change the done test from a bufferLength of 0 to <= 1 for Float32 format buffers. They apparently spit out a final buffer of length 1 (content 00).

Here's my code that works on iOS 17 beta 4 and iOS 16.6:

func saveAVSpeechUtteranceToFile(utterance: AVSpeechUtterance, fileURL: URL) throws {
    // init
    totalBufferLength = 0
    bufferWriteCount = 0
    output = nil
    
    // delete file if it already exists
    try? FileManager.default.removeItem(at: fileURL)
    
    // synthesize to buffers
    synthesizer.write(utterance) { [self] buffer in
        guard let pcmBuffer = buffer as? AVAudioPCMBuffer else {
            return
        }
        
        // float32 buffers spit out a final buffer of length 1, contents 00
        let doneLength: Int
        if pcmBuffer.format.commonFormat == .pcmFormatInt16 || pcmBuffer.format.commonFormat == .pcmFormatInt32 {
            doneLength = 0
        } else {
            doneLength = 1
        }
        
        if pcmBuffer.frameLength <= doneLength {
            // done
            playAudioFile(url: fileURL)  // or whatever
            
        } else {
            totalBufferLength += pcmBuffer.frameLength
            bufferWriteCount += 1
            
            if output == nil {
                do {
                    output = try AVAudioFile(forWriting: fileURL, settings: pcmBuffer.format.settings, commonFormat: pcmBuffer.format.commonFormat, interleaved: false)
                } catch {
                    print("create AVAudioFile error: \(error.localizedDescription)")
                }
            }
            
            do {
                try output!.write(from: pcmBuffer)
            } catch {
                print("output!.write failed, error: \(error.localizedDescription)")
            }
        }
    }
}