IOS, No output when AUGraphConnectNodeInput is set

Question

I am stuck with AUGraph for a while now and would really appreciate if my problem is resolved. What is am trying to do right now is to play data(bytes) coming from UDP. I have successfully achieved how to play data using AUGraph but i can't figure out how to change its playback speed.

My current scenario is to get Data from UDP, pass it a converter -> newTimePitch -> converter -> ioUnit.

Converters: Converters, convert ASBD of 48000 to desired format for timePitch and again convert it back to 48000 to play from ioUnit.

IOUnit Currently for demonstration i have removed the UDP and played revert back by using 2 different ioUnit (One for recording and other for playing).

CircularBuffer In Recording callback i am pushing data into my circular buffer and in playback Callback i am poping data from it and memcpy to my ioData

Don't be overwhelmed by the code you see. Its simple.

Starting AUGraph and Recording AudioUnit.

private func startRecordingUnit() 
{
    check(error: AUGraphStart(graph!), description: "Failed to start AUGraph")
    check(error: AudioOutputUnitStart(audioUnit!), description: "Failed to start recording
        audio unit.")
}

Setup AUGraph

private func setupRecordingUnit(){

    var description = AudioComponentDescription(
        componentType: OSType(kAudioUnitType_Output),
        componentSubType: OSType(kAudioUnitSubType_VoiceProcessingIO),
        componentManufacturer: OSType(kAudioUnitManufacturer_Apple),
        componentFlags: 0,
        componentFlagsMask: 0
    )

    let inputComponent = AudioComponentFindNext(nil, &description)
    check(error: AudioComponentInstanceNew(inputComponent!, &audioUnit), description: 
    "Failed to init recording audio unit.")

    check(error: AudioUnitSetProperty(
        audioUnit!,
        AudioUnitPropertyID(kAudioOutputUnitProperty_EnableIO),
        AudioUnitScope(kAudioUnitScope_Input),
        kInputBus,
        &flag,
        MemoryLayoutStride.SizeOf32(flag)
        ),
          description: "Failed to set enable IO for recording."
    )

    check(error: AudioUnitSetProperty(
        audioUnit!,
        AudioUnitPropertyID(kAudioUnitProperty_StreamFormat),
        AudioUnitScope(kAudioUnitScope_Output),
        kInputBus,
        &ioFormat!,
        MemoryLayoutStride.SizeOf32(ioFormat)
        ),
          description: "Failed to set stream format on output unit with scope input."
    )

    check(error: NewAUGraph(&graph), description: "Failed to create AU Graph")

    check(error: AUGraphOpen(graph!), description: "Failed to open AUGraph")

    var outputDesc = AudioComponentDescription(
        componentType: OSType(kAudioUnitType_Output),
        componentSubType: OSType(kAudioUnitSubType_VoiceProcessingIO),
        componentManufacturer: OSType(kAudioUnitManufacturer_Apple),
        componentFlags: 0,
        componentFlagsMask: 0
    )

    var firstConverterDesc = AudioComponentDescription(
        componentType: OSType(kAudioUnitType_FormatConverter),
        componentSubType: OSType(kAudioUnitSubType_AUConverter),
        componentManufacturer: OSType(kAudioUnitManufacturer_Apple),
        componentFlags: 0,
        componentFlagsMask: 0
    )

    var pitchConverterDesc = AudioComponentDescription(
        componentType: OSType(kAudioUnitType_FormatConverter),
        componentSubType: OSType(kAudioUnitSubType_NewTimePitch),
        componentManufacturer: OSType(kAudioUnitManufacturer_Apple),
        componentFlags: 0,
        componentFlagsMask: 0
    )

    var secondConverterDesc = AudioComponentDescription(
        componentType: OSType(kAudioUnitType_FormatConverter),
        componentSubType: OSType(kAudioUnitSubType_AUConverter),
        componentManufacturer: OSType(kAudioUnitManufacturer_Apple),
        componentFlags: 0,
        componentFlagsMask: 0
    )

    check(error: AUGraphAddNode(graph!, &firstConverterDesc, &firstConverterNode), 
    description: "Failed to Add Node of First desc")
    check(error: AUGraphAddNode(graph!, &pitchConverterDesc, &newTimePitchNode), 
    description: "Failed to Add Node of new Time Desc")
    check(error: AUGraphAddNode(graph!, &secondConverterDesc, &secondConverterNode), 
    description: "Failed to Add Node of Second desc")
    check(error: AUGraphAddNode(graph!, &outputDesc, &outputNode), description: "Failed to 
    add node of output desc")

    check(error: AUGraphNodeInfo(graph!, firstConverterNode, nil, &firstConverterUnit), 
    description: "Failed to get Node Info of FIRST Unit")
    check(error: AUGraphNodeInfo(graph!, newTimePitchNode, nil, &newTimePitchUnit), 
    description: "Failed to get Node Info of new Time Unit")
    check(error: AUGraphNodeInfo(graph!, secondConverterNode, nil, &secondConverterUnit), 
    description: "Failed to get Node Info of SECOND Unit")
    check(error: AUGraphNodeInfo(graph!, outputNode, nil, &outputUnit), description: "Failed 
    to get Node Info of output Unit");

    var sizeASBD = MemoryLayoutStride.SizeOf32(AudioStreamBasicDescription())
    var ioASBDin = AudioStreamBasicDescription()
    var ioASBDout = AudioStreamBasicDescription()
    AudioUnitGetProperty(newTimePitchUnit!, kAudioUnitProperty_StreamFormat, 
    kAudioUnitScope_Input, kOutputBus, &ioASBDin, &sizeASBD);
    AudioUnitGetProperty(newTimePitchUnit!, kAudioUnitProperty_StreamFormat, 
    kAudioUnitScope_Output, kOutputBus, &ioASBDout, &sizeASBD);

    check(error: AudioUnitSetProperty(firstConverterUnit!, kAudioUnitProperty_StreamFormat, 
    kAudioUnitScope_Input, kOutputBus, &ioFormat, MemoryLayoutStride.SizeOf32(ioFormat)), 
    description: "Failed to set property of FIRST Unit")
    check(error: AudioUnitSetProperty(firstConverterUnit!, kAudioUnitProperty_StreamFormat, 
    kAudioUnitScope_Output, kOutputBus, &ioASBDin, MemoryLayoutStride.SizeOf32(ioASBDin)), 
    description: "Failed to set property first unit to temp format")

    check(error: AudioUnitSetProperty(newTimePitchUnit!, kAudioUnitProperty_StreamFormat, 
    kAudioUnitScope_Input, kOutputBus, &ioASBDin, MemoryLayoutStride.SizeOf32(ioASBDin)), 
    description: "Failed to set property of new Time Pitch")
    check(error: AudioUnitSetProperty(newTimePitchUnit!, kAudioUnitProperty_StreamFormat, 
    kAudioUnitScope_Output, kOutputBus, &ioASBDout, MemoryLayoutStride.SizeOf32(ioASBDout)), 
    description: "Failed to set property of new Time Pitch OUTOUT")

    check(error: AudioUnitSetProperty(secondConverterUnit!, kAudioUnitProperty_StreamFormat, 
    kAudioUnitScope_Input, kOutputBus, &ioASBDout, MemoryLayoutStride.SizeOf32(ioASBDout)), 
    description: "Failed to set property of Second Converter Unit")
    check(error: AudioUnitSetProperty(secondConverterUnit!, kAudioUnitProperty_StreamFormat, 
    kAudioUnitScope_Output, kOutputBus, &ioFormat, MemoryLayoutStride.SizeOf32(ioFormat)), 
    description: "Failed to set property of Second Converter Unit to io Format")

    check(error: AudioUnitSetProperty(outputUnit!, kAudioUnitProperty_StreamFormat, 
    kAudioUnitScope_Input, kOutputBus, &ioFormat, MemoryLayoutStride.SizeOf32(ioFormat)), 
    description: "Failed to set property of OUTPUT Unit")

    // ************************************* RECORDING 
    ****************************************************

    var recordingCallback = AURenderCallbackStruct(
        inputProc: AudioController_RecordingCallback,
        inputProcRefCon: UnsafeMutableRawPointer(Unmanaged.passUnretained(self).toOpaque())
    )

    check(error: AudioUnitSetProperty(
        audioUnit!,
        AudioUnitPropertyID(kAudioOutputUnitProperty_SetInputCallback),
        AudioUnitScope(kAudioUnitScope_Global),
        kInputBus,
        &recordingCallback,
        MemoryLayout<AURenderCallbackStruct>.size.ui
        ),
          description: "Failed to set property on recording callback."
    )
    // *************************************** RECORDING END 
    ****************************************************

    var playbackCallback = AURenderCallbackStruct(
        inputProc: AudioController_PlaybackCallback,
        inputProcRefCon: UnsafeMutableRawPointer(Unmanaged.passUnretained(self).toOpaque())
    )

    check(error: AUGraphConnectNodeInput(graph!, firstConverterNode, 0, newTimePitchNode, 
    0), description: "Failed to connect FIRST AND timePitch Nodes")
    check(error: AUGraphConnectNodeInput(graph!, newTimePitchNode, 0, secondConverterNode, 
    0), description: "Failed to connect timePitch AND Second Nodes")
    check(error: AUGraphConnectNodeInput(graph!, secondConverterNode, 0, outputNode, 0), 
    description: "Failed to connect Second AND OUTPUT Nodes")

    check(error: AudioUnitSetProperty(outputUnit!, kAudioUnitProperty_SetRenderCallback, 
    kAudioUnitScope_Output, kOutputBus, &playbackCallback, 
    MemoryLayout<AURenderCallbackStruct>.size.ui), description: "Failed to set Property for 
    varispeed Unit 1")

        check(error: AUGraphInitialize(graph!), description: "Unable to initialize AUGraph")

}

Recording is working fine but playbackCalback doesn't run after I connect it with `AUConvertor`.

For testing purposes I removed connections AUGraphConnectNodeInput and it started playing. I think something is wrong with connections and I would really appreciate if I understand what is causing the problem.

I have made couple of searches but can't find any useful help.

score 0 · Accepted Answer · answered Apr 03 '20 at 00:36

0

The problem might be that you are setting two inputs to your IO node (remote or voice processing), and no inputs to your audio graph.

answered Apr 03 '20 at 00:36

hotpaw2

70,107
14
90
153

i know, but scenario as i described above is: we are sending and getting data from UDP. To replicate this i have created 2 IOUnits, one is recording and other is playing, IF i don't use the AUGraphConnectNodeInput i am able to record and play data using these 2 IOUnits, – Muhammad Faizan Apr 03 '20 at 08:00
Draw a picture of your graph (directed graph with arrows including IO and callbacks) and you will see your problem. – hotpaw2 Apr 03 '20 at 09:18
Thanks. I figured out the issue and you are right that i am not inputting anything into my AUGraph. But i am still stuck in a scenario where i am getting data from UDP. UDP -> Converter -> Varispeed -> output Please help me with connecting UDP data to my converter node. i have completed and tested my code with revertback and it worked fine but not with UDP. – Muhammad Faizan Apr 03 '20 at 10:53
I am confused on how to connect my UDP incoming data to my converter – Muhammad Faizan Apr 03 '20 at 10:55

IOS, No output when AUGraphConnectNodeInput is set

Setup AUGraph

Recording is working fine but playbackCalback doesn't run after I connect it with AUConvertor.

1 Answers1

Recording is working fine but playbackCalback doesn't run after I connect it with `AUConvertor`.