Decode MP3 File from NSData

Question

For my application, I need to decode an MP3 file which is stored in an NSData object.

For security reasons, it is undesirable to write the NSData object to disk and re-open it using a System URL reference, even if its only locally stored for a few moments.

I would like to take advantage Extended Audio File Services (or Audio File Services) to do this, but I'm having trouble getting a representation of the NSData, which exists only in memory, that can be read by these Audio File Services.

Edit: I want to decode the MP3 data so I can get access to linear, PCM audio samples for manipulation. Playing back from NSData object is not a problem.

My code is as follows:

 decryptedData; //an NSData object which has already been initialized
 const void *dataBytes   = decryptedData.bytes; //pointer to the bytes in my NSData object 

//this creates a CFURLRef from the pointer to the byte data
//I have printed out the resulting CFURL and have confirmed that it is indeed reading the bytes correctly
CFURLRef audioFileURLFromBytes = CFURLCreateWithBytes (kCFAllocatorDefault,
                                                               dataBytes,
                                                               decryptedData.length,
                                                               kCFStringEncodingASCII,
                                                               NULL);

//attempt to open the the URL using Extended Audio File Services
        ExtAudioFileRef outExtAudioFile;
        OSStatus err = 0;
        err = ExtAudioFileOpenURL(audioFileURLFromBytes, &outExtAudioFile);
        if (err != noErr) {
            NSLog(@"ExtAudioFileOpenURL failed with OSStatus Code %i \n", err);
        }

//Attempt to open the URL using Audio File Services
        AudioFileID audioFile;
        OSStatus res = 0;
        res = AudioFileOpenURL(audioFileURLFromBytes, kAudioFileReadPermission,  kAudioFileMP3Type, &audioFile);
        if (res != noErr) {
            NSLog(@"AudioFileOpenURL failed with OSStatus Code %i \n", res);
        }

Both attempts at opening the URL result in an OSStatus Code 43, which is "file not found".

I have verified that my pointer is pointing to the correct address in memory for the NSData and that the bytes can be read correctly.

Is there some limitation to the Extended Audio File Services that prohibit references to bytes stored in memory?

Thanks for any help you can provide.

Edit: I figured out how to do it using Sbooth's suggestion. Code below: This function takes an NSData object containing an mp3 representation of an audio file. It decodes it as linear PCM so you can get the samples and then re-encodes it as AAC. I don't think MP3 encoding is available in CoreAudio across all platforms (mobile/desktop). This code was tested on my Mac and gets the job done.

-(void) audioFileReaderWithData: (NSData *) audioData {

        AudioFileID         refAudioFileID;
        ExtAudioFileRef     inputFileID;
        ExtAudioFileRef     outputFileID;

        OSStatus result = AudioFileOpenWithCallbacks(audioData, readProc, 0, getSizeProc, 0, kAudioFileMP3Type, &refAudioFileID);
        if(result != noErr){
            NSLog(@"problem in theAudioFileReaderWithData function: result code %i \n", result);
        }

        result = ExtAudioFileWrapAudioFileID(refAudioFileID, false, &inputFileID);
        if (result != noErr){
            NSLog(@"problem in theAudioFileReaderWithData function Wraping the audio FileID: result code %i \n", result);
        }

        // Client Audio Format Description
    AudioStreamBasicDescription clientFormat;
    memset(&clientFormat, 0, sizeof(clientFormat));
    clientFormat.mFormatID          = kAudioFormatLinearPCM;
    clientFormat.mFramesPerPacket   = 1;
    clientFormat.mChannelsPerFrame  = 2;
    clientFormat.mBitsPerChannel    = 32;
    clientFormat.mBytesPerPacket    = clientFormat.mBytesPerFrame = 4 *   clientFormat.mChannelsPerFrame;
    clientFormat.mFormatFlags       = kAudioFormatFlagsNativeFloatPacked;
    clientFormat.mSampleRate        = 44100;

     //Output Audio Format Description
     AudioStreamBasicDescription outputFormat;
     memset(&outputFormat, 0, sizeof(outputFormat));
     outputFormat.mChannelsPerFrame  = 2;
     outputFormat.mSampleRate        = 44100;
     outputFormat.mFormatID          = kAudioFormatMPEG4AAC;
     outputFormat.mFormatFlags       = kMPEG4Object_AAC_Main;
     outputFormat.mBitsPerChannel    = 0;
     outputFormat.mBytesPerFrame     = 0;
     outputFormat.mBytesPerPacket    = 0;
     outputFormat.mFramesPerPacket   = 1024;

     // create the outputFile that we're writing to here....
     UInt32 outputFormatSize = sizeof(outputFormat);
     result = 0;
     result = AudioFormatGetProperty(kAudioFormatProperty_FormatInfo, 0, NULL, &outputFormatSize, &outputFormat);
     if(result != noErr)
            NSLog(@"could not set the output format with status code %i \n",result);

     NSMutableString *outputFilePath = [NSMutableString stringWithCapacity: 100];
     [outputFilePath setString:@"/Users/You/Desktop/testAudio.m4a"];
     NSURL *sourceURL = [NSURL fileURLWithPath:outputFilePath];

     result      =  0;
     result      =  ExtAudioFileCreateWithURL((CFURLRef)sourceURL, kAudioFileM4AType, &outputFormat, NULL, kAudioFileFlags_EraseFile, &outputFileID);
     if(result != noErr){
           NSLog(@"ExtAudioFileCreateWithURL failed for outputFileID with status %i \n", result);
      }

     int size = sizeof(clientFormat);
     result = 0;
     result = ExtAudioFileSetProperty(inputFileID, kExtAudioFileProperty_ClientDataFormat, size, &clientFormat);

     if(result != noErr)
        NSLog(@"error on ExtAudioFileSetProperty for input File with result code %i \n", result);

     size = sizeof(clientFormat);
     result = 0;
     result = ExtAudioFileSetProperty(outputFileID, kExtAudioFileProperty_ClientDataFormat, size, &clientFormat);
     if(result != noErr)
         NSLog(@"error on ExtAudioFileSetProperty for output File with result code %i \n", result);

     int totalFrames = 0;
     UInt32 outputFilePacketPosition = 0; //in bytes  
     UInt32 encodedBytes = 0;

     while (1) {
        UInt32 bufferByteSize       = 22050 * 4 * 2;
        char srcBuffer[bufferByteSize];
        UInt32 numFrames            = (bufferByteSize/clientFormat.mBytesPerFrame);

        AudioBufferList fillBufList;
        fillBufList.mNumberBuffers  = 1;
        fillBufList.mBuffers[0].mNumberChannels     = clientFormat.mChannelsPerFrame;
        fillBufList.mBuffers[0].mDataByteSize       = bufferByteSize;
        fillBufList.mBuffers[0].mData               = srcBuffer;
        result = 0;
        result = ExtAudioFileRead(inputFileID, &numFrames, &fillBufList);

        if (result != noErr) {
            NSLog(@"Error on ExtAudioFileRead with result code %i \n", result);
                totalFrames = 0;
            break;
        }
        if (!numFrames)
            break;

        totalFrames = totalFrames + numFrames;

        result = 0;
        result = ExtAudioFileWrite(outputFileID,
                                   numFrames,
                                   &fillBufList);

        if(result!= noErr){
            NSLog(@"ExtAudioFileWrite failed with code %i \n", result);
        }

        encodedBytes += numFrames  * clientFormat.mBytesPerFrame;
    }

    //Clean up

    ExtAudioFileDispose(inputFileID);
    ExtAudioFileDispose(outputFileID);
    AudioFileClose(refAudioFileID);

}

And you'll need these functions as well...

static OSStatus readProc(void* clientData,
                         SInt64 position,
                         UInt32 requestCount,
                         void* buffer,
                         UInt32* actualCount)
{

    NSData *inAudioData = (NSData *) clientData;

    size_t dataSize = inAudioData.length;
    size_t bytesToRead = 0;

    if(position < dataSize) {
        size_t bytesAvailable = dataSize - position;
        bytesToRead = requestCount <= bytesAvailable ? requestCount : bytesAvailable;

        [inAudioData getBytes: buffer range:NSMakeRange(position, bytesToRead)];
    } else {
        NSLog(@"data was not read \n");
        bytesToRead = 0;
    }

    if(actualCount)
        *actualCount = bytesToRead;

    return noErr;
}

static SInt64 getSizeProc(void* clientData) {
    NSData *inAudioData = (NSData *) clientData;
    size_t dataSize = inAudioData.length;
    return dataSize;
}

It's another topic where he or she just wants the straight answer without showing the whole picture of what they wish to accomplish. — El Tomato, Jun 26 '13 at 20:52
@TBlue My apologies if my problem and question wasn't clear enough. Basically it goes like this: 1) Through a separate decryption process, I have an mp3 audio file in Memory stored in an NSData object. 2) I would to process the raw linear, PCM samples decoded from that NSData object 3) I want to avoid writing the NSData to an mp3 file on disk for security reasons. I realize this seems like a strange way to go about things but security is an issue. Does that make more sense? — rmigneco, Jun 26 '13 at 21:23
@SpinalTapFan11, Could you explain where in the code is the MP3 decoded to PCM? I would to manipulate the PCM, then convert it to AAC. Thanks. — cube, Oct 21 '13 at 21:49
The AudioBufferList "mData" attribute points towards the PCM data. You can use that buffer to get the samples and manipulate them. — rmigneco, Oct 22 '13 at 18:01
Strangely this code works only when I write the file to macs desktop for the first time and then when I write the documents directly its strangely save the file but gives `Fail: AudioFileOpenURL failed` error both on simulator and terminal. @rmigneco do you have this code available on github or somewhere? — SpaceDust__, Feb 13 '15 at 19:14
@Ugenlik Unfortunately I do not have a version of the code that I can share. Were you able to gather any information from the error codes? — rmigneco, Feb 19 '15 at 02:09
@rmigneco aac file I was downloading had extra payloads in it so I had to cut the parts that are irrelevant to audio. So I figured it out, it was not related to this code. But I would still appreciate a sample code that feeds nsdata to buffers without writing the file. — SpaceDust__, Feb 19 '15 at 15:01
@Spacedust_ I think the above code snippet outlines the basic approach enough for you to get going. the `readProc` function is responsible for taking your NSData pointer (via `clientData`) and filling a buffer you pass in. When you call `ExtAudioFileRead` from the parent function, it's up to you to store the data in the returned buffers as you see fit — rmigneco, Feb 25 '15 at 03:27

score 3 · Accepted Answer · answered Jun 27 '13 at 00:14

3

The problem is that you're trying to create a CFURLRef object from the audio bytes (MP3 frames) using the ASCII encoding. CFURLCreateWithBytes is meant to be used with byte strings, not binary data (i.e., "http://www.apple.com" as a char *). To accomplish what you want use AudioFileOpenWithCallbacks, pass your NSData object as the refcon, and handle raw reading/seeking in your custom callbacks operating on the NSData that you passed in.

answered Jun 27 '13 at 00:14

sbooth

16,646
2
55
81

Thanks! I was able to figure it out using AudioFileOpenWithCallbacks. I edited my original question to include the answer. – rmigneco Jun 27 '13 at 20:17
The api doesnt work when there’s an embeded image in the mp3 :( – masaldana2 Apr 10 '18 at 09:36

score -1 · Answer 2 · answered Jun 26 '13 at 20:57

-1

Use Audio Queue Services or AVPlayer for playing audio from a stream or memory.

answered Jun 26 '13 at 20:57

Marcus Adams

53,009
9
91
143

Thanks, I should have been more specific in my request. I require sample level access to the audio. So decoding the mp3 to get linear PCM samples. – rmigneco Jun 26 '13 at 21:05
Audio Queue Services should give you access to audio samples – Khanh Nguyen Jun 27 '13 at 03:20

Decode MP3 File from NSData

2 Answers2

Linked